A method for video jitter detection, an electronic device, and a computer-readable storage medium

Abstract

The present application discloses a video jitter detection method, an electronic device, and a computer-readable storage medium. In the present application, by traversing adjacent frame images in a video stream, the adjacent frame images include a first image and a second image, and the frame order of the first image in the video stream is less than the frame order of the second image in the video stream; matching each first pixel point in the first image with each second pixel point in the second image to obtain a pair of target pixel points that are matched; determining a target adjacent frame image with an offset angle in the video stream according to the coordinate information of each pixel point in the pair of target pixel points and the quantity information of the pair of target pixel points; in response to the number of image pairs of the target adjacent frame images being greater than a preset image quantity threshold, performing video jitter detection processing according to the offset angle corresponding to each target adjacent frame image to obtain a video jitter detection result. Thereby, the jitter condition of the video is detected in real time.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular, to a method for detecting video jitter, an electronic device, and a computer-readable storage medium. Background Art

[0002] In current society, by shooting a preset area through a camera to obtain a video and then analyzing and processing the video, multiple purposes such as security prevention, traffic management, and environmental monitoring can be achieved. Among them, the stability and clarity of the video are very important.

[0003] Currently, in order to obtain a stable and clear video, the commonly adopted method is to conduct anti-shake tests when installing the camera. However, during long-term use, the camera may be affected by the environment, resulting in jitter in the captured image, such as long-term wind blowing, material expansion or contraction caused by temperature changes, vibrations caused by earthquakes or wind, collisions by organisms, etc. Therefore, a method capable of detecting video jitter in real time is needed. Summary of the Invention

[0004] The main technical problem to be solved by the present application is to provide a method for detecting video jitter, an electronic device, and a computer-readable storage medium, which can detect the jitter situation of the video in real time.

[0005] To solve the above technical problem, the present application provides a method for detecting video jitter.

[0006] In one embodiment, a method for detecting video jitter includes: traversing adjacent frame images in a video stream, where the adjacent frame images include a first image and a second image, and the frame order of the first image in the video stream is less than the frame order of the second image in the video stream; performing matching processing on each first pixel point in the first image and each second pixel point in the second image to obtain a pair of target pixel points that are matched; determining a target adjacent frame image with an offset angle in the video stream according to the coordinate information of each pixel point in the pair of target pixel points and the quantity information of the pair of target pixel points; in response to the number of image pairs of the target adjacent frame images being greater than a preset image quantity threshold, performing video jitter detection processing according to the offset angles corresponding to each target adjacent frame image to obtain a video jitter detection result.

[0007] In one embodiment, the target pixel point pair includes a first target pixel point and a second target pixel point that match each other. The step of matching each first pixel point in the first image with each second pixel point in the second image to obtain a matching target pixel point pair includes: selecting first key pixel points from the first image according to the attribute information of each first pixel point in the first image; selecting second key pixel points from the second image according to the attribute information of each second pixel point in the second image; determining the first target pixel point from the first image and determining the second target pixel point that matches the first target pixel point from the second image according to the feature similarity between the pixel features of each first key pixel point in the first image and the pixel features of each second key pixel point in the second image.

[0008] In one embodiment, the attribute information includes a key degree. The step of selecting first key pixel points from the first image according to the attribute information of each first pixel point in the first image includes: determining whether the key degree of each first pixel point in the first image is greater than a key threshold; if so, determining the corresponding first pixel point as the first key pixel point.

[0009] In one embodiment, the step of determining the first target pixel point from the first image and determining the second target pixel point that matches the first target pixel point from the second image according to the feature similarity between the pixel features of each first key pixel point in the first image and the pixel features of each second key pixel point in the second image includes: calculating the feature similarity between the pixel features of each first key pixel point in the first image and the pixel features of each second key pixel point in the second image to obtain at least one feature similarity corresponding to each first key pixel point; performing a sorting process on the at least one feature similarity corresponding to the first key pixel point to obtain a feature similarity sequence; determining the first key pixel point and the second key pixel point corresponding to the feature similarity located in a preset sequence in the feature similarity sequence as the first target pixel point and the second target pixel point, respectively.

[0010] In one embodiment, the quantity information includes a quantity, the target pixel point pair includes a first target pixel point and a second target pixel point that match each other, and the step of determining the target adjacent frame images with an offset angle in the video stream according to the coordinate information of each pixel point in the target pixel point pair and the quantity information of the target pixel point pair includes: determining a coordinate offset value of the second target pixel point relative to the first target pixel point according to the coordinate information of the first target pixel point and the coordinate information of the second target pixel point; in response to the number of target pixel point pairs with a coordinate offset value greater than a preset coordinate offset threshold being greater than a preset pixel quantity threshold, determining an offset angle between the adjacent frame images according to the coordinate offset value; and determining the adjacent frame images corresponding to the offset angle as the target adjacent frame images.

[0011] In one embodiment, the step of determining an offset angle between the adjacent frame images according to the coordinate offset value includes: performing an arctangent function calculation process on the coordinate offset value to obtain an angle value; and determining the angle value as the offset angle between the adjacent frame images.

[0012] In one embodiment, the step of performing video jitter detection processing according to the offset angles corresponding to the respective target adjacent frame images to obtain a video jitter detection result includes: determining an offset direction corresponding to each target adjacent frame image according to the offset angle corresponding to each target adjacent frame image; determining a target offset direction change sequence of the video stream according to the offset directions corresponding to the respective target adjacent frame images; and in response to there being the same change trend in the target offset direction change sequence, determining that the video jitter detection result is that the video stream has jitter.

[0013] In one embodiment, the step of determining a target offset direction change sequence of the video stream according to the offset directions corresponding to the respective target adjacent frame images includes: sorting the offset directions corresponding to the respective target adjacent frame images according to the frame sequence of the first image in the corresponding target adjacent frame image to obtain an offset direction change sequence of the video stream; and performing a duplicate removal process on adjacent and identical offset directions in the offset direction change sequence of the video stream to obtain a target offset direction change sequence of the video stream.

[0014] To solve the above technical problems, the present application provides an electronic device, including a memory and a processor, where the memory stores program instructions, and the processor retrieves the program instructions from the memory to execute the above video jitter detection method.

[0015] To solve the above technical problems, the present application provides a computer-readable storage medium, including: storing program data, where the program data is used to implement the above video jitter detection method when executed by a processor.

[0016] In the above solution, by traversing adjacent frame images in a video stream, where the adjacent frame images include a first image and a second image, and then determining target adjacent frame images with an offset angle in the video stream based on the coordinate information of each pixel point in the matching target pixel point pairs in the first image and the second image and the quantity information of the target pixel point pairs; then, when the number of image pairs in the target adjacent frame images is greater than a preset image quantity threshold, video jitter detection processing is performed based on the offset angles corresponding to each target adjacent frame image to obtain a video jitter detection result. On the one hand, by acquiring the video stream and performing video jitter detection processing based on the offset angles of the target adjacent frame images in the video stream to obtain a video jitter detection result, real-time jitter detection of the video stream can be achieved. On the other hand, the coordinate information of the matching target pixel point pairs in the adjacent frame images and the quantity information of the target pixel point pairs can more accurately determine the target adjacent frame images with an offset angle in the video stream, thereby improving the accuracy of the video jitter detection result. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings, where:

[0018] Figure 1 is a flowchart of an exemplary embodiment of the video jitter detection method shown in the present application;

[0019] Figure 2 is Figure 1 a flowchart of an exemplary embodiment of step S120 in the video jitter detection method shown;

[0020] Figure 3 is Figure 1 a flowchart of an exemplary embodiment of step S130 in the video jitter detection method shown;

[0021] Figure 4 is Figure 1 a flowchart of an exemplary embodiment of step S140 in the video jitter detection method shown;

[0022] Figure 5 is a block diagram of an exemplary embodiment of the video jitter detection device shown in the present application;

[0023] Figure 6 is a schematic structural diagram of an embodiment of an electronic device provided by the present application;

[0024] Figure 7It is a schematic structural diagram of an embodiment of a computer-readable storage medium provided by this application. Detailed implementation manners

[0025] Next, the technical solutions in the embodiments of this application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of this application. It can be understood that the specific embodiments described herein are only used to explain this application, rather than limiting this application. Additionally, it should be noted that for the convenience of description, only parts related to this application rather than all structures are shown in the accompanying drawings. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of this application.

[0026] First of all, it should be noted that in current society, by shooting a preset area with a camera to obtain a video and then analyzing and processing the video, multiple purposes such as security prevention, traffic management, and environmental monitoring can be achieved. Among them, the stability and clarity of the video are very important. Currently, in order to obtain a stable and clear video, the commonly used method is to conduct anti-shake tests when installing the camera. However, during the long-term use of the camera, it may be affected by the environment, resulting in jitter in the captured image, such as long-term wind blowing, material expansion or contraction caused by temperature changes, vibrations caused by earthquakes or wind, collisions by organisms, etc. Therefore, a method capable of detecting video jitter in real time is needed.

[0027] Based on this, this application proposes a video jitter detection method, an electronic device, and a computer-readable storage medium. For details, please refer to Figure 1 , Figure 1 It is a schematic flowchart of an exemplary embodiment of the video jitter detection method shown in this application.

[0028] The execution subject of the video jitter detection method can be a terminal device, a server, or other processing devices. Among them, the terminal device can be a computer, a mobile device, a terminal, a computing device, a vehicle-mounted device, etc. The execution subject of the video jitter detection method can also be a video jitter detection device. In some possible implementation manners, this video jitter detection method can be implemented by a processor calling computer-readable instructions stored in a memory.

[0029] Specifically, a video jitter detection method in this embodiment includes the following steps:

[0030] Step S110: Traverse adjacent frame images in the video stream. The adjacent frame images include a first image and a second image, and the frame order of the first image in the video stream is less than the frame order of the second image in the video stream.

[0031] A video stream refers to continuous video data transmitted over the Internet. The video stream includes a sequence of frame images sorted in order of acquisition time from earliest to latest. For example, the video stream can be a 2-second real-time video received every 2 seconds.

[0032] Adjacent frame images refer to images with adjacent frame numbers in the video stream. Here, the frame number refers to the sequence number of the image in the sequence of frame images of the video stream.

[0033] The video jitter detection device traverses adjacent frame images in the video stream.

[0034] Step S120: Match each first pixel point in the first image with each second pixel point in the second image to obtain matching target pixel point pairs.

[0035] A target pixel pair refers to a first pixel point in the first image and a second pixel point in the second image that match each other. The target pixel pair can include a first target pixel point and a second target pixel point that match each other.

[0036] The video jitter detection device matches each first pixel point in the first image with each second pixel point in the second image to obtain matching target pixel point pairs. Specifically, the video jitter detection device calculates the similarity between each first pixel point in the first image and each second pixel point in the second image, obtains at least one similarity corresponding to each first pixel point, and determines the first pixel point and the second pixel point corresponding to the similarity greater than the preset similarity threshold as the matching target pixel point pair.

[0037] Step S130: Determine the target adjacent frame images with an offset angle in the video stream according to the coordinate information of each pixel point in the target pixel point pair and the quantity information of the target pixel point pair.

[0038] Coordinate information refers to the coordinates of a pixel point in the corresponding image. Coordinate information includes pixel point coordinates.

[0039] The quantity information can include a quantity.

[0040] The offset angle refers to the angular difference in the shooting of the second image relative to the first image.

[0041] Target adjacent frame images refer to adjacent frame images with an offset angle selected from the adjacent frame images of the video stream. The target adjacent frame images can include a first target image and a second target image.

[0042] The video jitter detection device determines target adjacent frame images with an offset angle in the video stream based on the coordinate information of each pixel point in the target pixel point pair and the quantity information of the target pixel point pair. Specifically, the video jitter detection device calculates the coordinate offset value between the pixel point coordinate of the first target pixel point in the first image and the pixel point coordinate of the second target pixel point in the second image, calculates the ratio between the quantity of target pixel point pairs with a non-zero coordinate offset value and the quantity of the target pixel point pair. If the ratio is greater than the preset ratio threshold, the arctangent function value of the coordinate offset value is determined as the offset angle; the adjacent frame images with an offset angle are determined as target adjacent frame images, otherwise the corresponding adjacent frame images have no offset angle. Among them, the preset ratio threshold can be 0.5.

[0043] For example, the video jitter detection device calculates the ratio between the quantity of target pixel point pairs with a non-zero coordinate offset value and the quantity of the target pixel point pair according to the following formula:

[0044]

[0045] Among them, represents the ratio between the quantity of target pixel point pairs with a non-zero coordinate offset value and the quantity of the target pixel point pair in the th target adjacent frame image, represents the coordinate offset of the j th group of target pixel point pairs, represents the summation function, represents the function for finding the length of the summation array, represents the th array corresponding to the target adjacent frame image, which includes the coordinate offset values of all target pixel points in the th target adjacent frame image, the array includes .

[0046] Step S140, in response to the quantity of image pairs of the target adjacent frame images being greater than the preset image quantity threshold, video jitter detection processing is performed according to the offset angle corresponding to each target adjacent frame image, and a video jitter detection result is obtained.

[0047] The quantity of image pairs refers to the quantity of target adjacent frame images composed of the first target image and the second target image. The preset image quantity threshold is the minimum value of the quantity of image pairs of the target adjacent frame images for jitter detection. The preset image quantity threshold can be 5.

[0048] The video jitter detection result can include that the video stream has jitter and the video stream has no jitter.

[0049] The video jitter detection device performs video jitter detection processing based on the offset angles corresponding to each target adjacent frame image, and obtains a video jitter detection result. Specifically, the offset direction corresponding to each target adjacent frame image is determined according to the offset angle corresponding to each target adjacent frame image; the corresponding direction value is determined from a preset value mapping table according to each offset direction, and the preset value mapping table includes the correspondence between the preset offset direction and the preset direction value; the direction values are sorted according to the chronological order of the earliest acquisition time of the corresponding target adjacent frame image, and a direction value sequence is obtained; the direction values that are adjacent in position and have the same value in the direction value sequence are merged, and a merged direction value sequence is obtained; in response to the existence of at least two identical numerical subsequences in the merged direction value sequence, it is determined that the video stream has jitter, and the numerical subsequence includes at least two consecutive direction values in the direction value sequence.

[0050] In one embodiment, the video jitter detection device sorts the offset directions corresponding to each target adjacent frame image according to the frame order of the first image in the corresponding target adjacent frame image, obtains an offset direction change sequence of the video stream, determines the corresponding direction value from a preset value mapping table according to the offset direction in the offset direction change sequence, and obtains a direction value sequence; the direction values that are adjacent in position and have the same value in the direction value sequence are merged, and a merged direction value sequence is obtained; at least one numerical subsequence is selected from the merged direction value sequence, and the numerical subsequence includes at least two adjacent direction values selected from the merged direction value sequence; it is determined whether there is an identical numerical subsequence in at least one numerical subsequence, and if so, it is determined that the video stream has jitter, and if not, it is determined that the video stream has no jitter.

[0051] For example, the offset direction is up, and the corresponding direction value is 1; the offset direction is right, and the corresponding direction value is 2; the offset direction is down, and the corresponding direction value is 3; the offset direction is left, and the corresponding direction value is 4. The offset direction change sequence of the video stream is , and the corresponding direction value is determined from the preset value mapping table according to the offset direction in the offset direction change sequence, and the obtained direction value sequence is ; the direction values that are adjacent in position and have the same value in the direction value sequence are merged, and the obtained merged direction value sequence is , and at least one numerical subsequence selected from the merged direction value sequence is , ; there are duplicates in at least one numerical subsequence , then it is determined that the video stream has jitter. Another example, the merged direction value sequence is , and at least one numerical subsequence selected from the merged direction value sequence is , , , , , , , , ; If there is no repeated numerical subsequence in at least one numerical subsequence, it is determined that the video stream does not have jitter.

[0052] It can be seen that by traversing adjacent frame images in the video stream, the adjacent frame images include a first image and a second image, and then according to the coordinate information of each pixel point in the matching target pixel point pairs in the first image and the second image and the quantity information of the target pixel point pairs, the target adjacent frame images with an offset angle in the video stream are determined; then when the number of image pairs of the target adjacent frame images is greater than a preset image quantity threshold, video jitter detection processing is performed according to the offset angles corresponding to each target adjacent frame image, and a video jitter detection result is obtained. On the one hand, by acquiring the video stream and performing video jitter detection processing according to the offset angles of the target adjacent frame images in the video stream, a video jitter detection result is obtained, so that real-time jitter detection of the video stream can be achieved. On the other hand, the target adjacent frame images with an offset angle in the video stream can be more accurately determined through the coordinate information of the matching target pixel point pairs in the adjacent frame images and the quantity information of the target pixel point pairs, thereby improving the accuracy of the video jitter detection result.

[0053] Based on the above embodiments, please refer to Figure 2 , Figure 2 is Figure 1 a schematic flowchart of an exemplary embodiment of step S120 in the video jitter detection method shown. Specifically, the target pixel point pair includes a mutually matching first target pixel point and a second target pixel point. The process of step S120 for matching each first pixel point in the first image with each second pixel point in the second image to obtain the matching target pixel point pairs includes the following steps:

[0054] Step S210: Select first key pixel points from the first image according to the attribute information of each first pixel point in the first image.

[0055] The attribute information may include criticality. Criticality is a value representing the importance degree of the pixel features of each pixel point in each image. Specifically, the video jitter detection device performs noise reduction processing on the first image and the second image respectively to obtain the first image after noise reduction processing and the second image after noise reduction processing; calculates the criticality of each first pixel point in the first image by using the SuperPoint network on the first image after noise reduction processing; calculates the criticality of each second pixel point in the second image by using the SuperPoint network on the second image after noise reduction processing.

[0056] Specifically, the video jitter detection device performs noise reduction processing on the first image and the second image respectively to obtain the first image after noise reduction processing and the second image after noise reduction processing. For example, the video jitter detection device uses a 3*3 Gaussian kernel to perform Gaussian blur processing on the first image and the second image respectively to obtain the first image after noise reduction processing and the second image after noise reduction processing. Thus, by performing Gaussian blur processing on the first image and the second image with the Gaussian kernel respectively, the noise points of the first image and the second image are reduced, and the interference information of the images is reduced, which is beneficial to improving the accuracy of subsequent matching processing.

[0057] The video jitter detection device selects first key pixel points from the first image according to the attribute information of each first pixel point in the first image. As an example, the video jitter detection device determines whether the criticality of each first pixel point in the first image is greater than the critical threshold; if so, the corresponding first pixel point is determined as the first key pixel point. As another example, the video jitter detection device sorts each first pixel point in descending order of criticality to obtain a first pixel point sequence; determines the first pixel points in the first pixel point sequence whose sorting is before the preset serial number as the first key pixel points.

[0058] For example, the video jitter detection device uses the SuperPoint network to calculate each image after noise reduction processing to obtain the criticality matrix and the feature matrix of each image, and each image can be the first image or the second image; determines the pixel points with criticality greater than the critical threshold as key pixel points, stores the x-axis coordinate, y-axis coordinate, and criticality of the key pixel points in the key pixel matrix, and the dimension of this matrix is , where N represents the number of elements greater than the critical threshold in the criticality matrix; selects the features corresponding to the key pixel points from the feature matrix and stores them in the key feature matrix.

[0059] Step S220: Select second key pixel points from the second image according to the attribute information of each second pixel point in the second image.

[0060] The video jitter detection device selects second key pixels from the second image according to the attribute information of each second pixel in the second image. As an example, the video jitter detection device determines whether the key degree of each second pixel in the second image is greater than the key threshold; if so, the corresponding second pixel is determined as the second key pixel. As another example, the video jitter detection device sorts each second pixel in descending order of key degree to obtain a second pixel sequence; the second pixels in the second pixel sequence whose sorting is before the preset serial number are determined as the second key pixels.

[0061] Step S230: Determine a first target pixel from the first image and a second target pixel matching the first target pixel from the second image according to the feature similarity between the pixel features of each first key pixel in the first image and the pixel features of each second key pixel in the second image.

[0062] The pixel feature refers to the feature of a pixel point in an image. Specifically, the video jitter detection device uses the SuperPoint network to calculate the first image after noise reduction processing to obtain the pixel features of each first pixel in the first image; uses the SuperPoint network to calculate the second image after noise reduction processing to obtain the pixel features of each second pixel in the second image. For example, the pixel features of each pixel point can be represented as a vector , where represents the length of the pixel feature, the value depends on the network structure of the SuperPoint network, the value can be 256, represents the height of the pixel feature, W represents the width of the pixel feature. Extracting the pixel features of the image through the SuperPoint network improves the robustness.

[0063] The feature similarity is a numerical value representing the similarity degree between pixel features. Specifically, the video jitter detection device calculates the cosine similarity between the pixel features of each first key pixel and the pixel features of each second key pixel in the second image, and uses the cosine similarity as the feature similarity.

[0064] The video jitter detection device determines a first target pixel point from the first image and a second target pixel point from the second image that matches the first target pixel point according to the feature similarity between the pixel features of each first key pixel point in the first image and the pixel features of each second key pixel point in the second image. Specifically, the video jitter detection device calculates the feature similarity between the pixel features of each first key pixel point in the first image and the pixel features of each second key pixel point in the second image, and obtains at least one feature similarity corresponding to each first key pixel point; performs a sorting process on the at least one feature similarity corresponding to the first key pixel point to obtain a feature similarity sequence; and determines the first key pixel point and the second key pixel point corresponding to the feature similarity located in the preset sequence in the feature similarity sequence as the first target pixel point and the second target pixel point, respectively.

[0065] The feature similarity sequence can be at least one feature similarity sorted in descending order of numerical value, or at least one feature similarity sorted in ascending order of numerical value.

[0066] Specifically, the video jitter detection device determines the first key pixel point corresponding to the feature similarity located in the preset sequence in the feature similarity sequence as the first target pixel point, and determines the corresponding second key pixel point as the second target pixel point. When the feature similarity sequence is a feature similarity sorted in descending order of numerical value, the preset sequence can be 1, or can also be 2.

[0067] In one embodiment, each adjacent frame image in the video stream is traversed, and the currently traversed adjacent frame images include and , where represents the Kth frame image in the video stream, and represents the (K + 1)th frame image in the video stream. For the pixel feature of each first key pixel point in , the cosine similarity is calculated in turn with the pixel features of each second key pixel point in , and at least one cosine similarity corresponding to the first key pixel point is obtained; the maximum cosine similarity corresponding to the first key pixel point is determined, the second key pixel point corresponding to the pixel feature used to calculate the maximum cosine similarity is determined as the second target pixel point, and the corresponding first key pixel point is determined as the first target pixel point that matches the second target pixel point.

[0068] Based on the above embodiment, please refer to Figure 3 , Figure 3 is Figure 1Schematic diagram of an exemplary embodiment of step S130 in the shown video jitter detection method. Specifically, the quantity information includes a quantity, the target pixel point pair includes a mutually matching first target pixel point and a second target pixel point, and the process of step S130 for determining the target adjacent frame images with an offset angle in the video stream according to the coordinate information of each pixel point in the target pixel point pair and the quantity information of the target pixel point pair includes the following steps:

[0069] Step S310: Determine the coordinate offset value of the second target pixel point relative to the first target pixel point according to the coordinate information of the first target pixel point and the coordinate information of the second target pixel point.

[0070] The coordinate offset value is a numerical value representing the degree of offset of the coordinate of the second target pixel point relative to the coordinate of the first target pixel point.

[0071] The video jitter detection device determines the difference between the coordinate of the first target pixel point and the coordinate of the second target pixel point as the coordinate offset value of the second target pixel point relative to the first target pixel point.

[0072] For example, the coordinate offset can be calculated by the following formula:

[0073]

[0074] Where represents the coordinate offset value of the second target pixel point relative to the first target pixel point in the th group of target pixel point pairs, where the target adjacent frame images include the th frame image and the th frame image, and the th frame image and the th frame image include multiple groups of target pixel point pairs. represents the coordinate of the second target pixel point in the th group of target pixel point pairs in the th frame image. represents the coordinate of the first target pixel point in the th group of target pixel point pairs in the th frame image.

[0075] Step S320: In response to the number of target pixel pairs with a coordinate offset value greater than the preset coordinate offset threshold being greater than the preset pixel quantity threshold, determine the offset angle between the adjacent frame images according to the coordinate offset value.

[0076] The video jitter detection device determines the offset angle between the adjacent frame images according to the coordinate offset value. Specifically, the video jitter detection device performs an arctangent function calculation process on the coordinate offset value to obtain an angle value; and determines the angle value as the offset angle between the adjacent frame images.

[0077] For example, the video jitter detection device performs an arctangent function calculation process on the coordinate offset value to obtain an angle value. Specifically, the video jitter detection device performs an arctangent function calculation process on each coordinate offset value to obtain at least one tangent angle, and determines the average value of the at least one tangent angle as the angle value. The angle value calculation process can be represented by the following formula:

[0078]

[0079] where, represents the angle value, represents the averaging function, represents the arctangent function, represents the coordinate offset value of the second target pixel relative to the first target pixel in the

[0080] Step S330: Determine the adjacent frame images corresponding to the offset angle as the target adjacent frame images.

[0081] The video jitter detection device determines the adjacent frame images corresponding to the offset angle as the target adjacent frame images.

[0082] Based on the above embodiments, please refer to Figure 4 , Figure 4 is Figure 1 a schematic flowchart of an exemplary embodiment of step S140 in the video jitter detection method shown. Specifically, the process of step S140 performing video jitter detection processing according to the offset angles corresponding to the respective target adjacent frame images to obtain a video jitter detection result includes the following steps:

[0083] Step S410: Determine the offset direction corresponding to each target adjacent frame image according to the offset angle corresponding to each target adjacent frame image.

[0084] As an example, the video jitter detection device determines the offset direction corresponding to each target adjacent frame image from a preset direction mapping table according to the offset angle corresponding to each target adjacent frame image. The preset direction mapping table includes the corresponding relationship between the preset offset angle and the preset offset direction. As another example, the video jitter detection device determines the angle range in which the offset angle corresponding to each target adjacent frame image is located, and determines the offset direction corresponding to the angle range as the offset direction corresponding to the target adjacent frame image.

[0085] In one embodiment, the angle range includes a first angle range, a second angle range, a third angle range, and a fourth angle range. The first angle range can be , and the corresponding offset direction is up. The second angle range can be , the corresponding offset direction is to the right, and the third angle range can be , the corresponding offset direction is downward, and the fourth angle range can be or , the corresponding offset direction is to the left.

[0086] Step S420: Determine the target offset direction change sequence of the video stream according to the offset directions corresponding to the target adjacent frame images.

[0087] The video jitter detection device determines the target offset direction change sequence of the video stream according to the offset directions corresponding to the target adjacent frame images. Specifically, the video jitter detection device sorts the offset directions corresponding to the target adjacent frame images according to the frame sequence of the first image in the corresponding target adjacent frame images to obtain the offset direction change sequence of the video stream; and performs duplicate removal processing on the offset directions that are adjacent in position and the same in the offset direction change sequence of the video stream to obtain the target offset direction change sequence of the video stream.

[0088] The video jitter detection device sorts the offset directions corresponding to the target adjacent frame images according to the frame sequence of the first image in the corresponding target adjacent frame images to obtain the offset direction change sequence of the video stream. Specifically, the video jitter detection device sorts the offset directions corresponding to the target adjacent frame images in descending or ascending order according to the frame sequence of the first image in the corresponding target adjacent frame images to obtain the offset direction change sequence of the video stream.

[0089] The video jitter detection device performs duplicate removal processing on the offset directions that are adjacent in position and the same in the offset direction change sequence of the video stream to obtain the target offset direction change sequence of the video stream. For example, the offset direction change sequence can be , after performing duplicate removal on the offset directions that are adjacent in position and the same in the offset direction change sequence of the video stream, the obtained target offset direction change sequence is . Another example, the offset direction change sequence can be , after performing duplicate removal on the offset directions that are adjacent in position and the same in the offset direction change sequence of the video stream, the obtained target offset direction change sequence is .

[0090] Step S430: In response to the existence of the same change trend in the target offset direction change sequence, determine that the video jitter detection result is that the video stream has jitter.

[0091] Before the step of determining that the video jitter detection result is that the video stream has jitter when the video jitter detection device responds to the existence of the same change trend in the target offset direction change sequence, it further includes: judging whether there is the same change trend in the target offset direction change sequence to obtain a judgment result. As an example, the video jitter detection device sequentially selects at least two adjacent offset directions from the target offset direction change sequence to construct a direction subsequence, obtains at least one direction subsequence, and judges whether there are at least two identical direction subsequences in the at least one direction subsequence. If so, it is determined that there is the same change trend in the target offset direction change sequence, and if not, it is determined that there is no same change trend in the target offset direction change sequence. For example, the target offset direction change sequence is , and the direction subsequence has a repeated situation, then it is determined that there is the same change trend in the target offset direction change sequence.

[0092] Figure 5 is a block diagram of the video jitter detection device shown in an exemplary embodiment of the present application. As Figure 5 shown, the exemplary video jitter detection device 500 includes: a traversal module 510, a matching module 520, a target adjacent frame image determination module 530, and a video jitter detection module 540. Specifically:

[0093] The traversal module 510 is configured to traverse adjacent frame images in the video stream. The adjacent frame images include a first image and a second image, and the frame order of the first image in the video stream is less than the frame order of the second image in the video stream.

[0094] The matching module 520 is configured to perform matching processing on each first pixel point in the first image and each second pixel point in the second image to obtain a pair of target pixel points that are matched.

[0095] The target adjacent frame image determination module 530 is configured to determine target adjacent frame images with an offset angle in the video stream according to the coordinate information of each pixel point in the pair of target pixel points and the quantity information of the pair of target pixel points.

[0096] The video jitter detection module 540 is configured to, in response to the number of image pairs of the target adjacent frame images being greater than a preset image quantity threshold, perform video jitter detection processing according to the offset angles corresponding to each target adjacent frame image to obtain a video jitter detection result.

[0097] In the exemplary video jitter detection device, by traversing adjacent frame images in the video stream, the adjacent frame images include a first image and a second image, and then determining the target adjacent frame images with an offset angle in the video stream according to the coordinate information of each pixel point in the matching target pixel point pairs in the first image and the second image and the quantity information of the target pixel point pairs; then when the number of image pairs of the target adjacent frame images is greater than a preset image quantity threshold, performing video jitter detection processing according to the offset angles corresponding to the respective target adjacent frame images to obtain a video jitter detection result. On the one hand, by acquiring the video stream and performing video jitter detection processing according to the offset angles of the target adjacent frame images in the video stream to obtain a video jitter detection result, it is thus possible to achieve real-time jitter detection of the video stream. On the other hand, the coordinate information of the matching target pixel point pairs in the adjacent frame images and the quantity information of the target pixel point pairs can more accurately determine the target adjacent frame images with an offset angle in the video stream, thereby improving the accuracy rate of the video jitter detection result.

[0098] Among them, the functions of each module can be referred to in the embodiments of the video jitter detection method, which will not be elaborated here.

[0099] To implement the video jitter detection method of the above embodiments, the present application proposes another electronic device. For details, please refer to Figure 6 , Figure 6 which is a schematic structural diagram of an embodiment of the electronic device provided by the present application.

[0100] The electronic device 600 includes a memory 601 and a processor 602. Among them, the memory 601 and the processor 602 are coupled.

[0101] The memory 601 is used to store program data, and the processor 602 is used to execute the program data to implement the video jitter detection method of the above embodiments.

[0102] In this embodiment, the processor 602 can also be referred to as a CPU (Central Processing Unit, central processing unit). The processor 602 may be an integrated circuit chip with signal processing capabilities. The processor 602 can also be a general-purpose processor, a digital signal processor, an application-specific integrated circuit, a field-programmable gate array or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor 602 can also be any conventional processor, etc.

[0103] The present application also provides a computer-readable storage medium, as Figure 7 shown, the computer-readable storage medium 700 is used to store program data 701. When the program data 701 is executed by the processor, it is used to implement the video jitter detection method in the method embodiments of the present application.

[0104] In the embodiments of the video jitter detection method of this application, when the method involved exists in the form of a software functional unit and is sold or used as an independent product, it can be stored in a device, such as a computer-readable storage medium. Based on such an understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs.

[0105] The above are only the embodiments of this application, and do not limit the patent scope of this application. Any equivalent structure or equivalent process transformation made by using the content of the specification and drawings of this application, or directly or indirectly applied in other related technical fields, shall be equally included in the patent protection scope of this application.

Claims

1. A video jitter detection method, characterized in that: The method comprises: Traversing adjacent frame images in a video stream, the adjacent frame images include a first image and a second image, and a frame order of the first image in the video stream is smaller than a frame order of the second image in the video stream; Matching each first pixel in the first image with each second pixel in the second image to obtain a matched target pixel pair; Determine a target adjacent frame image having an offset angle in the video stream according to coordinate information of each pixel in the target pixel pair and quantity information of the target pixel pair; In response to the number of image pairs of the target adjacent frame images being greater than a preset image number threshold, performing video jitter detection processing according to the offset angles corresponding to the target adjacent frame images to obtain a video jitter detection result; The step of performing video jitter detection processing according to the offset angles corresponding to adjacent frame images of each target to obtain a video jitter detection result includes: Determine the offset direction corresponding to each target adjacent frame image according to the offset angle corresponding to each target adjacent frame image; Determining a shift direction change sequence of the video stream according to the shift directions corresponding to each target adjacent frame image; Determine the corresponding direction value from a preset value mapping table according to the offset direction in the offset direction change sequence, and obtain a direction value sequence, wherein the preset value mapping table includes a correspondence between preset offset directions and preset direction values; Merging the direction values ​​that are adjacent in position and have the same value in the direction value sequence to obtain a merged direction value sequence; In response to the presence of at least two repeated value subsequences in the merged direction value sequence, it is determined that jitter occurs in the video stream; or, The step of performing video jitter detection processing according to the offset angles corresponding to adjacent frame images of each target to obtain a video jitter detection result includes: Determine the offset direction corresponding to each adjacent frame image of the target according to the offset angle corresponding to each adjacent frame image of the target; Determining a target offset direction change sequence of the video stream according to offset directions corresponding to adjacent frame images of each target; The step of determining the target offset direction change sequence of the video stream according to the offset directions corresponding to each target adjacent frame image comprises: Sorting the offset directions corresponding to the target adjacent frame images according to the frame sequence of the first image in the corresponding target adjacent frame images to obtain the offset direction change sequence of the video stream; Deduplication processing is performed on adjacent and identical offset directions in the offset direction change sequence of the video stream to obtain a target offset direction change sequence of the video stream; If there are at least two identical direction subsequences in the target offset direction change sequence, it is determined that jitter occurs in the video stream.

2. The video jitter detection method according to claim 1, characterized in that: The target pixel pair includes a first target pixel and a second target pixel that match each other, and the step of matching each first pixel in the first image with each second pixel in the second image to obtain a matched target pixel pair includes: Selecting a first key pixel point from the first image according to the attribute information of each first pixel point in the first image; Selecting a second key pixel point from the second image according to the attribute information of each second pixel point in the second image; The first target pixel point is determined from the first image and the second target pixel point matching the first target pixel point is determined from the second image according to the feature similarity between the pixel features of each first key pixel point in the first image and the pixel features of each second key pixel point in the second image.

3. The video jitter detection method according to claim 2, characterized in that: The attribute information includes a key degree, and the step of selecting a first key pixel point from the first image according to the attribute information of each first pixel point in the first image includes: Determining whether a criticality of each first pixel in the first image is greater than a critical threshold; If so, the corresponding first pixel point is determined as the first key pixel point.

4. The video jitter detection method according to claim 2, characterized in that: The step of determining the first target pixel point from the first image according to the feature similarity between the pixel features of each first key pixel point in the first image and the pixel features of each second key pixel point in the second image, and determining the second target pixel point matching the first target pixel point from the second image comprises: Calculating feature similarities between pixel features of each first key pixel in the first image and pixel features of each second key pixel in the second image to obtain at least one feature similarity corresponding to each first key pixel; Sorting at least one feature similarity corresponding to the first key pixel point to obtain a feature similarity sequence; A first key pixel point and a second key pixel point corresponding to the feature similarity in the feature similarity sequence and located in a preset sequence are respectively determined as the first target pixel point and the second target pixel point.

5. The video jitter detection method according to claim 1, characterized in that: The quantity information includes a quantity, the target pixel point pair includes a first target pixel point and a second target pixel point that match each other, and the step of determining the target adjacent frame image with an offset angle in the video stream according to the coordinate information of each pixel point in the target pixel point pair and the quantity information of the target pixel point pair includes: Determine a coordinate offset value of the second target pixel point relative to the first target pixel point according to the coordinate information of the first target pixel point and the coordinate information of the second target pixel point; In response to the number of target pixel pairs whose coordinate offset values ​​are greater than a preset coordinate offset threshold being greater than a preset pixel number threshold, determining an offset angle between the adjacent frame images according to the coordinate offset values; The adjacent frame image corresponding to the offset angle is determined as the target adjacent frame image.

6. The video jitter detection method according to claim 5, characterized in that: The step of determining the offset angle between adjacent frame images according to the coordinate offset value comprises: Performing arc tangent function calculation on the coordinate offset value to obtain an angle value; The angle value is determined as the offset angle between the adjacent frame images.

7. An electronic device, characterized in that: include: A memory and a processor, wherein the memory stores program instructions, and the processor retrieves the program instructions from the memory to execute the method according to any one of claims 1 to 6.

8. A computer-readable storage medium, characterized in that: include: Program data is stored, and when the program data is executed by a processor, it is used to implement the method according to any one of claims 1 to 6.

Images