Video jitter detection method, electronic equipment and computer readable storage medium

Abstract

The invention discloses a video jitter detection method, electronic equipment and a computer readable storage medium. Adjacent frame images in a video stream are traversed, the adjacent frame images comprise a first image and a second image, and the frame sequence of the first image in the video stream is smaller than the frame sequence of the second image in the video stream; performing matching processing on each first pixel point in the first image and each second pixel point in the second image to obtain a matched target pixel point pair; determining a target adjacent frame image with a deviation angle in the video stream according to the coordinate information of each pixel point in the target pixel point pairs and the quantity information of the target pixel point pairs; and in response to the fact that the number of the image pairs of the target adjacent frame images is greater than a preset image number threshold, performing video jitter detection processing according to the deviation angle corresponding to each target adjacent frame image to obtain a video jitter detection result. Therefore, the jitter condition of the video is detected in real time.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a video jitter detection method, an electronic device and a computer-readable storage medium. Background Art

[0002] In today's society, by shooting a preset area with a camera, obtaining a video, and then analyzing and processing the video, multiple purposes such as security prevention, traffic management, and environmental monitoring can be achieved. Among them, the stability and clarity of the video are very important.

[0003] At present, in order to obtain stable and clear videos, the usual method is to perform anti-shake tests when installing the camera. However, the camera may be affected by the environment during long-term use, causing the shooting picture to shake, such as long-term wind blowing, material expansion or contraction caused by temperature changes, vibration caused by earthquakes or wind, collisions of organisms, etc. Therefore, a method that can detect video shaking in real time is needed. Summary of the invention

[0004] The main technical problem solved by the present application is to provide a video jitter detection method, an electronic device and a computer-readable storage medium, which can detect the jitter of the video in real time.

[0005] In order to solve the above technical problems, the present application provides a video jitter detection method.

[0006] In one embodiment, a video jitter detection method includes: traversing adjacent frame images in a video stream, the adjacent frame images including a first image and a second image, the frame order of the first image in the video stream being less than the frame order of the second image in the video stream; matching each first pixel in the first image with each second pixel in the second image to obtain a matched target pixel pair; determining a target adjacent frame image having an offset angle in the video stream based on coordinate information of each pixel in the target pixel pair and information on the number of the target pixel pairs; in response to the number of image pairs of the target adjacent frame images being greater than a preset image number threshold, performing video jitter detection processing based on the offset angle corresponding to each target adjacent frame image to obtain a video jitter detection result.

[0007] In one embodiment, the target pixel pair includes a first target pixel and a second target pixel that match each other, and the step of matching each first pixel in the first image with each second pixel in the second image to obtain a matched target pixel pair includes: selecting a first key pixel from the first image based on the attribute information of each first pixel in the first image; selecting a second key pixel from the second image based on the attribute information of each second pixel in the second image; determining the first target pixel from the first image and determining the second target pixel matching the first target pixel from the second image based on the feature similarity between the pixel features of each first key pixel in the first image and the pixel features of each second key pixel in the second image.

[0008] In one embodiment, the attribute information includes a criticality, and the step of selecting a first key pixel point from the first image based on the attribute information of each first pixel point in the first image includes: determining whether the criticality of each first pixel point in the first image is greater than a critical threshold; if so, determining the corresponding first pixel point as the first key pixel point.

[0009] In one embodiment, the step of determining the first target pixel from the first image and determining the second target pixel matching the first target pixel from the second image based on the feature similarity between the pixel features of each first key pixel in the first image and the pixel features of each second key pixel in the second image includes: calculating the feature similarity between the pixel features of each first key pixel in the first image and the pixel features of each second key pixel in the second image to obtain at least one feature similarity corresponding to each of the first key pixels; sorting the at least one feature similarity corresponding to the first key pixel to obtain a feature similarity sequence; and determining the first key pixel and the second key pixel corresponding to the feature similarity in a preset sequence in the feature similarity sequence as the first target pixel and the second target pixel, respectively.

[0010] In one embodiment, the quantity information includes a quantity, the target pixel point pair includes a first target pixel point and a second target pixel point that match each other, and the step of determining the target adjacent frame image having an offset angle in the video stream based on the coordinate information of each pixel point in the target pixel point pair and the quantity information of the target pixel point pair includes: determining the coordinate offset value of the second target pixel point relative to the first target pixel point based on the coordinate information of the first target pixel point and the coordinate information of the second target pixel point; in response to the number of target pixel point pairs whose coordinate offset values ​​are greater than a preset coordinate offset threshold being greater than a preset pixel quantity threshold, determining the offset angle between the adjacent frame images based on the coordinate offset value; and determining the adjacent frame image corresponding to the offset angle as the target adjacent frame image.

[0011] In one embodiment, the step of determining the offset angle between the adjacent frame images based on the coordinate offset value includes: performing an inverse tangent function calculation on the coordinate offset value to obtain an angle value; and determining the angle value as the offset angle between the adjacent frame images.

[0012] In one embodiment, the step of performing video jitter detection processing according to the offset angles corresponding to each target adjacent frame image to obtain a video jitter detection result includes: determining the offset directions corresponding to each target adjacent frame image according to the offset angles corresponding to each target adjacent frame image; determining a target offset direction change sequence of the video stream according to the offset directions corresponding to each target adjacent frame image; and in response to the presence of the same change trend in the target offset direction change sequence, determining that the video jitter detection result is that jitter occurs in the video stream.

[0013] In one embodiment, the step of determining the target offset direction change sequence of the video stream based on the offset directions corresponding to each target adjacent frame image includes: sorting the offset directions corresponding to each target adjacent frame image according to the frame order of the first image in the corresponding target adjacent frame image to obtain the offset direction change sequence of the video stream; deduplicating adjacent and identical offset directions in the offset direction change sequence of the video stream to obtain the target offset direction change sequence of the video stream.

[0014] In order to solve the above technical problem, the present application provides an electronic device, including a memory and a processor, wherein the memory stores program instructions, and the processor retrieves the program instructions from the memory to execute the above video jitter detection method.

[0015] In order to solve the above technical problems, the present application provides a computer-readable storage medium, including: storing program data, and the program data is used to implement the above video jitter detection method when executed by a processor.

[0016] The above scheme traverses the adjacent frame images in the video stream, the adjacent frame images include the first image and the second image, and then determines the target adjacent frame images with offset angles in the video stream according to the coordinate information of each pixel in the target pixel pairs matched in the first image and the second image and the number information of the target pixel pairs; then when the number of image pairs of the target adjacent frame images is greater than the preset image number threshold, the video jitter detection process is performed according to the offset angles corresponding to each target adjacent frame image to obtain the video jitter detection result. On the one hand, by acquiring the video stream, the video jitter detection process is performed according to the offset angles of the target adjacent frame images in the video stream to obtain the video jitter detection result, so that the jitter detection of the video stream can be realized in real time. On the other hand, the target adjacent frame images with offset angles in the video stream can be more accurately determined through the coordinate information of the target pixel pairs matched in the adjacent frame images and the number information of the target pixel pairs, thereby improving the accuracy of the video jitter detection result. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work, among which: Figure 1 is a flowchart of an exemplary embodiment of a video jitter detection method shown in the present application; Figure 2 yes Figure 1 FIG. 1 is a flow chart of an exemplary embodiment of step S120 in the video jitter detection method; Figure 3 yes Figure 1 FIG. 1 is a flow chart of an exemplary embodiment of step S130 in the video jitter detection method; Figure 4 yes Figure 1 FIG. 1 is a flow chart of an exemplary embodiment of step S140 in the video jitter detection method; Figure 5 is a block diagram of a video jitter detection device shown in an exemplary embodiment of the present application; Figure 6 It is a structural schematic diagram of an embodiment of an electronic device provided by the present application; Figure 7 It is a structural schematic diagram of an embodiment of a computer-readable storage medium provided by the present application. DETAILED DESCRIPTION

[0018] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It will be appreciated that the specific embodiments described herein are only used to explain the present application, rather than to limit the present application. It should also be noted that, for ease of description, only some but not all structures related to the present application are shown in the drawings. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the art without making creative work are within the scope of protection of the present application.

[0019] First of all, it should be noted that in today's society, by shooting a preset area with a camera, obtaining a video, and then analyzing and processing the video, multiple purposes such as security, traffic management, and environmental monitoring can be achieved. Among them, the stability and clarity of the video are very important. At present, in order to obtain stable and clear videos, the usual method is to perform an anti-shake test when installing the camera. However, the camera may be affected by the environment during long-term use, causing the shooting picture to shake, such as long-term wind blowing, material expansion or contraction caused by temperature changes, vibrations caused by earthquakes or wind, collisions of organisms, etc. Therefore, a method that can detect video jitter in real time is needed.

[0020] Based on this, the present application proposes a video jitter detection method, an electronic device and a computer-readable storage medium. Figure 1 , Figure 1 It is a flowchart of an exemplary embodiment of a video jitter detection method shown in the present application.

[0021] The execution subject of the video jitter detection method may be a terminal device or a server or other processing device, wherein the terminal device may be a computer, a mobile device, a terminal, a computing device, a vehicle-mounted device, etc. The execution subject of the video jitter detection method may also be a video jitter detection device. In some possible implementations, the video jitter detection method may be implemented by a processor calling a computer-readable instruction stored in a memory.

[0022] Specifically, a video jitter detection method of this embodiment includes the following steps: Step S110: traverse adjacent frame images in the video stream, where the adjacent frame images include a first image and a second image, and a frame sequence of the first image in the video stream is smaller than a frame sequence of the second image in the video stream.

[0023] Video stream refers to continuous video data transmitted over the Internet. Video streams include multiple frames of sequential frame images that are ordered from the earliest to the latest according to the acquisition time. For example, a video stream can be a 2-second real-time video received every 2 seconds.

[0024] Adjacent frame images refer to images with adjacent frame sequences in a video stream, wherein the frame sequence refers to the sequence number of the images in the video stream in the sequence frame images of the video stream.

[0025] The video jitter detection device traverses adjacent frame images in the video stream.

[0026] Step S120: Match each first pixel in the first image with each second pixel in the second image to obtain a matched target pixel pair.

[0027] The target pixel pair refers to a first pixel point in the first image and a second pixel point in the second image that match each other. The target pixel pair may include a first target pixel point and a second target pixel point that match each other.

[0028] The video jitter detection device matches each first pixel in the first image with each second pixel in the second image to obtain a matched target pixel pair. Specifically, the video jitter detection device calculates the similarity between each first pixel in the first image and each second pixel in the second image to obtain at least one similarity corresponding to each first pixel, and determines the first pixel and the second pixel corresponding to the similarity greater than a preset similarity threshold as a matched target pixel pair.

[0029] Step S130: determining target adjacent frame images having an offset angle in the video stream according to the coordinate information of each pixel in the target pixel pair and the number information of the target pixel pair.

[0030] Coordinate information refers to the coordinates of a pixel point in a corresponding image. Coordinate information includes the coordinates of a pixel point.

[0031] The quantity information may include the quantity.

[0032] The offset angle refers to the angle difference between the shooting of the second image and the shooting of the first image.

[0033] The target adjacent frame image refers to an adjacent frame image with an offset angle selected from adjacent frame images of the video stream. The target adjacent frame image may include a first target image and a second target image.

[0034] The video jitter detection device determines the target adjacent frame images with an offset angle in the video stream based on the coordinate information of each pixel in the target pixel pair and the number information of the target pixel pairs. Specifically, the video jitter detection device calculates the coordinate offset value between the pixel coordinates of the first target pixel in the first image and the pixel coordinates of the second target pixel in the second image, and calculates the ratio between the number of target pixel pairs with non-zero coordinate offset values ​​and the number of target pixel pairs. If the ratio is greater than a preset ratio threshold, the inverse tangent function value of the coordinate offset value is determined as the offset angle; the adjacent frame image with an offset angle is determined as the target adjacent frame image, otherwise the corresponding adjacent frame image does not have an offset angle. Among them, the preset ratio threshold can be 0.5.

[0035] For example, the video jitter detection device calculates the ratio between the number of target pixel pairs whose coordinate offset values ​​are not zero and the number of target pixel pairs as follows:

[0036] in, Characterization The ratio between the number of target pixel pairs with non-zero coordinate offset values ​​and the number of target pixel pairs in adjacent frame images. Characterization j The coordinate offset of the target pixel pair. Represents the summation function, Represents the function of finding the length of an array, Characterization An array corresponding to adjacent frame images of the target, Including The coordinate offset value of all target pixels in the adjacent frame images of the target, The array includes .

[0037] Step S140 , in response to the number of image pairs of target adjacent frame images being greater than a preset image number threshold, performing video jitter detection processing according to the offset angles corresponding to the target adjacent frame images to obtain a video jitter detection result.

[0038] The number of image pairs refers to the number of target adjacent frame images composed of the first target image and the second target image. The preset image number threshold is the minimum value of the number of image pairs of target adjacent frame images for jitter detection. The preset image number threshold can be 5.

[0039] The video jitter detection result may include whether the video stream has jitter or not.

[0040] The video jitter detection device performs video jitter detection processing according to the offset angle corresponding to each target adjacent frame image to obtain a video jitter detection result. Specifically, the offset direction corresponding to each target adjacent frame image is determined according to the offset angle corresponding to each target adjacent frame image; the corresponding direction value is determined from a preset value mapping table according to each offset direction, and the preset value mapping table includes the correspondence between the preset offset direction and the preset direction value; each direction value is sorted according to the order of the earliest acquisition time of the corresponding target adjacent frame image to obtain a direction value sequence; the direction values ​​that are adjacent in position and have the same value in the direction value sequence are merged to obtain a merged direction value sequence; in response to the presence of at least two groups of identical value subsequences in the merged direction value sequence, it is determined that the video stream is jittered, and the value subsequence includes at least two consecutive direction values ​​in the direction value sequence.

[0041] In one embodiment, a video jitter detection device sorts the offset directions corresponding to each target adjacent frame image according to the frame order of the first image in the corresponding target adjacent frame image to obtain an offset direction change sequence of the video stream, determines the corresponding direction value from a preset value mapping table according to the offset direction in the offset direction change sequence to obtain a direction value sequence; merges the direction values ​​in the direction value sequence that are adjacent in position and have the same value to obtain a merged direction value sequence; selects at least one value subsequence from the merged direction value sequence, the value subsequence includes at least two adjacent direction values ​​selected from the merged direction value sequence; determines whether at least one value subsequence has the same value subsequence, and if so, determines that the video stream is jittered, and if not, determines that the video stream is not jittered.

[0042] For example, if the offset direction is up, the corresponding direction value is 1; if the offset direction is right, the corresponding direction value is 2; if the offset direction is down, the corresponding direction value is 3; if the offset direction is left, the corresponding direction value is 4. The offset direction change sequence of the video stream is , according to the offset direction in the offset direction change sequence, the corresponding direction value is determined from the preset value mapping table, and the obtained direction value sequence is ; The direction values ​​in the direction value sequence that are adjacent and have the same value are merged, and the resulting direction value sequence after the merge is: , at least one numerical subsequence selected from the merged direction numerical sequence is , ; There are duplicates in at least one numerical subsequence , it is determined that the video stream is jittering. For another example, the direction value sequence after merging is , at least one numerical subsequence selected from the merged direction numerical sequence is , , , , , , , , ; There is no repeated numerical subsequence in at least one numerical subsequence, determining that the video stream does not experience jitter.

[0043] It can be seen that by traversing the adjacent frame images in the video stream, the adjacent frame images include the first image and the second image, and then determining the target adjacent frame images with offset angles in the video stream according to the coordinate information of each pixel in the target pixel pairs matched in the first image and the second image and the number information of the target pixel pairs; then when the number of image pairs of the target adjacent frame images is greater than the preset image number threshold, the video jitter detection process is performed according to the offset angles corresponding to each target adjacent frame image to obtain the video jitter detection result. On the one hand, by acquiring the video stream, the video jitter detection process is performed according to the offset angles of the target adjacent frame images in the video stream to obtain the video jitter detection result, so that the jitter detection of the video stream can be realized in real time. On the other hand, the target adjacent frame images with offset angles in the video stream can be more accurately determined through the coordinate information of the target pixel pairs matched in the adjacent frame images and the number information of the target pixel pairs, thereby improving the accuracy of the video jitter detection result.

[0044] Based on the above examples, please refer to Figure 2 , Figure 2 yes Figure 1 A flow chart of an exemplary embodiment of step S120 in the video jitter detection method is shown. Specifically, the target pixel pair includes a first target pixel and a second target pixel that match each other. Step S120 matches each first pixel in the first image with each second pixel in the second image. The process of obtaining the matched target pixel pair includes the following steps: Step S210: selecting first key pixels from the first image according to attribute information of each first pixel in the first image.

[0045] The attribute information may include criticality. Criticality is a numerical value that characterizes the importance of the pixel features of each pixel in each image. Specifically, the video jitter detection device performs noise reduction processing on the first image and the second image respectively to obtain the first image after noise reduction processing and the second image after noise reduction processing; uses the SuperPoint network to calculate the first image after noise reduction processing to obtain the criticality of each first pixel in the first image; uses the SuperPoint network to calculate the second image after noise reduction processing to obtain the criticality of each second pixel in the second image.

[0046] Specifically, the video jitter detection device performs noise reduction processing on the first image and the second image respectively to obtain the first image after noise reduction processing and the second image after noise reduction processing. For example, the video jitter detection device uses a 3*3 Gaussian kernel to perform Gaussian blur processing on the first image and the second image respectively to obtain the first image after noise reduction processing and the second image after noise reduction processing. Therefore, by performing Gaussian blur processing on the first image and the second image respectively through the Gaussian kernel, the noise points of the first image and the second image are reduced, and the interference information of the image is reduced, which is conducive to improving the accuracy of subsequent matching processing.

[0047] The video jitter detection device selects a first key pixel point from the first image according to the attribute information of each first pixel point in the first image. As an example, the video jitter detection device determines whether the criticality of each first pixel point in the first image is greater than a critical threshold; if so, the corresponding first pixel point is determined as the first key pixel point. As another example, the video jitter detection device sorts each first pixel point in descending order of criticality to obtain a first pixel point sequence; and the first pixel point in the first pixel point sequence that is sorted before a preset sequence number is determined as the first key pixel point.

[0048] For example, the video jitter detection device uses the SuperPoint network to calculate each image after noise reduction processing to obtain the key matrix and feature matrix of each image. Each image can be the first image or the second image. The pixel point whose key value is greater than the key threshold is determined as the key pixel point, and the x-axis coordinate, y-axis coordinate and key value of the key pixel point are stored in the key pixel matrix. The dimension of the matrix is , N represents the number of pixels in the criticality matrix that are greater than the critical threshold; the features corresponding to the key pixels will be selected from the feature matrix and stored in the key feature matrix.

[0049] Step S220: selecting second key pixels from the second image according to the attribute information of each second pixel in the second image.

[0050] The video jitter detection device selects a second key pixel from the second image according to the attribute information of each second pixel in the second image. As an example, the video jitter detection device determines whether the criticality of each second pixel in the second image is greater than the critical threshold; if so, the corresponding second pixel is determined as the second key pixel. As another example, the video jitter detection device sorts each second pixel in descending order of criticality to obtain a second pixel sequence; and determines the second pixel in the second pixel sequence that is ranked before a preset sequence number as the second key pixel.

[0051] Step S230: determining a first target pixel from the first image and determining a second target pixel matching the first target pixel from the second image based on feature similarity between pixel features of each first key pixel in the first image and pixel features of each second key pixel in the second image.

[0052] Pixel features refer to the features of pixels in an image. Specifically, the video jitter detection device uses the SuperPoint network to calculate the first image after noise reduction processing to obtain the pixel features of each first pixel in the first image; and uses the SuperPoint network to calculate the second image after noise reduction processing to obtain the pixel features of each second pixel in the second image. For example, the pixel features of each pixel can be represented as a vector ,in, Characterizes the length of the pixel feature, The value depends on the network structure of the SuperPoint network. The value can be 256. Characterizes the height of the pixel feature, W Characterizes the width of pixel features. The pixel features of the image are extracted through the SuperPoint network to improve robustness.

[0053] Feature similarity is a value that characterizes the similarity between pixel features. Specifically, the video jitter detection device calculates the cosine similarity between the pixel features of each first key pixel and the pixel features of each second key pixel in the second image, and uses the cosine similarity as the feature similarity.

[0054] The video jitter detection device determines a first target pixel from the first image and determines a second target pixel matching the first target pixel from the second image based on the feature similarity between the pixel features of each first key pixel in the first image and the pixel features of each second key pixel in the second image. Specifically, the video jitter detection device calculates the feature similarity between the pixel features of each first key pixel in the first image and the pixel features of each second key pixel in the second image to obtain at least one feature similarity corresponding to each first key pixel; sorts the at least one feature similarity corresponding to the first key pixel to obtain a feature similarity sequence; and determines the first key pixel and the second key pixel corresponding to the feature similarity in the feature similarity sequence that are located in a preset sequence as the first target pixel and the second target pixel, respectively.

[0055] The feature similarity sequence may be at least one feature similarity sorted in descending order of numerical values, or may be at least one feature similarity sorted in ascending order of numerical values.

[0056] Specifically, the video jitter detection device determines the first key pixel point corresponding to the feature similarity in the feature similarity sequence that is located in the preset sequence as the first target pixel point, and determines the corresponding second key pixel point as the second target pixel point. When the feature similarity sequence is a feature similarity sequence sorted in descending order of numerical values, the preset sequence can be 1 or 2.

[0057] In one embodiment, each adjacent frame image in the video stream is traversed, and the adjacent frame images currently traversed include and ,in, Represents the Kth frame image in the video stream, Represents the K+1th frame image in the video stream. The pixel features of each first key pixel in are sequentially The cosine similarity is calculated between the pixel features of each second key pixel point to obtain at least one cosine similarity corresponding to the first key pixel point; the maximum cosine similarity corresponding to the first key pixel point is determined, the second key pixel point corresponding to the pixel feature used to calculate the maximum cosine similarity is determined as the second target pixel point, and the corresponding first key pixel point is determined as the first target pixel point that matches the second target pixel point.

[0058] Based on the above examples, please refer to Figure 3 , Figure 3 yes Figure 1 A flowchart of an exemplary embodiment of step S130 in the video jitter detection method is shown. Specifically, the quantity information includes the quantity, the target pixel pair includes a first target pixel and a second target pixel that match each other, and the process of step S130 determining the target adjacent frame image with an offset angle in the video stream according to the coordinate information of each pixel in the target pixel pair and the quantity information of the target pixel pair includes the following steps: Step S310: Determine a coordinate offset value of the second target pixel relative to the first target pixel according to the coordinate information of the first target pixel and the coordinate information of the second target pixel.

[0059] The coordinate offset value is a numerical value representing the degree of offset of the coordinates of the second target pixel point relative to the coordinates of the first target pixel point.

[0060] The video jitter detection device determines the difference between the coordinates of the first target pixel point and the coordinates of the second target pixel point as the coordinate offset value of the second target pixel point relative to the first target pixel point.

[0061] For example, the coordinate offset can be calculated by the following formula:

[0062] in, Characterization The coordinate offset value of the second target pixel point relative to the first target pixel point in the target pixel point pair, wherein the target adjacent frame image includes the first Frame image and Frame image, Frame image and The frame image includes multiple groups of target pixel pairs. Characterization The frame image The coordinates of the second target pixel in the target pixel pair, Characterization The frame image The coordinates of the first target pixel in a set of target pixel pairs.

[0063] Step S320 , in response to the number of target pixel pairs whose coordinate offset values ​​are greater than a preset coordinate offset threshold being greater than a preset pixel number threshold, determining an offset angle between adjacent frame images according to the coordinate offset values.

[0064] The video jitter detection device determines the offset angle between adjacent frame images according to the coordinate offset value. Specifically, the video jitter detection device performs arc tangent function calculation processing on the coordinate offset value to obtain an angle value; and determines the angle value as the offset angle between adjacent frame images.

[0065] For example, the video jitter detection device performs an inverse tangent function calculation on the coordinate offset value to obtain an angle value. Specifically, the video jitter detection device performs an inverse tangent function calculation on each coordinate offset value to obtain at least one tangent angle, and determines the average value of the at least one tangent angle as the angle value. The angle value calculation process can be represented by the following formula:

[0066] in, Characterize the angle value, Represents the averaging function, Characterize the inverse tangent function, Characterization The coordinate offset value of the second target pixel point relative to the first target pixel point in the target pixel point pair.

[0067] Step S330: determine the adjacent frame image corresponding to the offset angle as the target adjacent frame image.

[0068] The video jitter detection device determines the adjacent frame images corresponding to the offset angle as target adjacent frame images.

[0069] Based on the above examples, please refer to Figure 4 , Figure 4 yes Figure 1 The flowchart of an exemplary embodiment of step S140 in the video jitter detection method is shown. Specifically, step S140 performs video jitter detection processing according to the offset angles corresponding to each target adjacent frame image, and the process of obtaining the video jitter detection result includes the following steps: Step S410: determining the offset direction corresponding to each target adjacent frame image according to the offset angle corresponding to each target adjacent frame image.

[0070] As an example, the video jitter detection device determines the offset direction corresponding to each target adjacent frame image from a preset direction mapping table according to the offset angle corresponding to each target adjacent frame image, and the preset direction mapping table includes the correspondence between the preset offset angle and the preset offset direction. As another example, the video jitter detection device determines the angle range of the offset angle corresponding to each target adjacent frame image, and determines the offset direction corresponding to the angle range of the offset angle as the offset direction corresponding to the target adjacent frame image.

[0071] In one embodiment, the angle range includes a first angle range, a second angle range, a third angle range and a fourth angle range. The first angle range may be , the corresponding offset direction is upward, and the second angle range can be , the corresponding offset direction is right, and the third angle range can be , the corresponding offset direction is downward, and the fourth angle range can be or , the corresponding offset direction is left.

[0072] Step S420: determining a target offset direction change sequence of the video stream according to the offset directions corresponding to each target adjacent frame image.

[0073] The video jitter detection device determines the target offset direction change sequence of the video stream according to the offset directions corresponding to each target adjacent frame image. Specifically, the video jitter detection device sorts the offset directions corresponding to each target adjacent frame image according to the frame order of the first image in the corresponding target adjacent frame image to obtain the offset direction change sequence of the video stream; and performs deduplication processing on the offset directions that are adjacent and identical in the offset direction change sequence of the video stream to obtain the target offset direction change sequence of the video stream.

[0074] The video jitter detection device sorts the offset directions corresponding to each target adjacent frame image according to the frame sequence of the first image in the corresponding target adjacent frame image, and obtains the offset direction change sequence of the video stream. Specifically, the video jitter detection device sorts the offset directions corresponding to each target adjacent frame image according to the frame sequence of the first image in the corresponding target adjacent frame image from large to small or from small to large, and obtains the offset direction change sequence of the video stream.

[0075] The video jitter detection device removes duplicates of adjacent and identical offset directions in the offset direction change sequence of the video stream to obtain a target offset direction change sequence of the video stream. For example, the offset direction change sequence can be After removing the adjacent and identical offset directions in the offset direction change sequence of the video stream, the target offset direction change sequence is obtained as For another example, the offset direction change sequence can be After removing the adjacent and identical offset directions in the offset direction change sequence of the video stream, the target offset direction change sequence is obtained as .

[0076] Step S430: In response to the existence of the same change trend in the target offset direction change sequence, determining that the video jitter detection result is that jitter occurs in the video stream.

[0077] Before the step of determining that the video jitter detection result is a jitter in the video stream in response to the existence of the same change trend in the target offset direction change sequence, the video jitter detection device further includes: judging whether the same change trend exists in the target offset direction change sequence to obtain a judgment result. As an example, the video jitter detection device sequentially selects at least two adjacent offset directions from the target offset direction change sequence to construct a direction subsequence, obtains at least one direction subsequence, and judges whether there are at least two identical direction subsequences in at least one direction subsequence. If so, it is determined that the same change trend exists in the target offset direction change sequence. Otherwise, it is determined that the same change trend does not exist in the target offset direction change sequence. For example, the target offset direction change sequence is , direction subsequence If there is a repetition, it is determined that there is a same change trend in the target offset direction change sequence.

[0078] Figure 5 FIG. 1 is a block diagram of a video jitter detection device shown in an exemplary embodiment of the present application. Figure 5 As shown, the exemplary video jitter detection device 500 includes: a traversal module 510, a matching module 520, a target adjacent frame image determination module 530 and a video jitter detection module 540. Specifically: The traversal module 510 is used to traverse adjacent frame images in a video stream, where the adjacent frame images include a first image and a second image, and the frame sequence of the first image in the video stream is smaller than the frame sequence of the second image in the video stream.

[0079] The matching module 520 is used to match each first pixel point in the first image with each second pixel point in the second image to obtain a matched target pixel point pair.

[0080] The target adjacent frame image determination module 530 is used to determine the target adjacent frame image with an offset angle in the video stream according to the coordinate information of each pixel point in the target pixel point pair and the number information of the target pixel point pair.

[0081] The video jitter detection module 540 is used to perform video jitter detection processing according to the offset angles corresponding to the target adjacent frame images in response to the number of image pairs of the target adjacent frame images being greater than a preset image number threshold, to obtain a video jitter detection result.

[0082] In this exemplary video jitter detection device, by traversing adjacent frame images in a video stream, the adjacent frame images include a first image and a second image, and then determining the target adjacent frame images with an offset angle in the video stream according to the coordinate information of each pixel in the target pixel pair matched in the first image and the second image and the number information of the target pixel pair; then when the number of image pairs of the target adjacent frame images is greater than the preset image number threshold, video jitter detection processing is performed according to the offset angle corresponding to each target adjacent frame image to obtain a video jitter detection result. On the one hand, by acquiring a video stream, video jitter detection processing is performed according to the offset angle of the target adjacent frame images in the video stream to obtain a video jitter detection result, thereby enabling real-time jitter detection of the video stream. On the other hand, the target adjacent frame images with an offset angle in the video stream can be more accurately determined through the coordinate information of the target pixel pair matched in the adjacent frame images and the number information of the target pixel pair, thereby improving the accuracy of the video jitter detection result.

[0083] The functions of each module can be found in the video jitter detection method embodiment, which will not be described in detail here.

[0084] In order to implement the video jitter detection method of the above embodiment, the present application proposes another electronic device, which is specifically referred to as Figure 6 , Figure 6 It is a structural schematic diagram of an embodiment of an electronic device provided by the present application.

[0085] The electronic device 600 includes a memory 601 and a processor 602 , wherein the memory 601 and the processor 602 are coupled.

[0086] The memory 601 is used to store program data, and the processor 602 is used to execute the program data to implement the video jitter detection method of the above embodiment.

[0087] In this embodiment, the processor 602 may also be referred to as a CPU (Central Processing Unit). The processor 602 may be an integrated circuit chip having signal processing capabilities. The processor 602 may also be a general-purpose processor, a digital signal processor, an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component. A general-purpose processor may be a microprocessor or the processor 602 may also be any conventional processor, etc.

[0088] The present application also provides a computer-readable storage medium, such as Figure 7 As shown, the computer-readable storage medium 700 is used to store program data 701. When the program data 701 is executed by the processor, it is used to implement the video jitter detection method in the method embodiment of the present application.

[0089] The method involved in the embodiment of the video jitter detection method of the present application, when implemented in the form of a software functional unit and sold or used as an independent product, can be stored in a device, such as a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor (processor) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program code.

[0090] The above description is only an implementation method of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the description and drawings of this application, or directly or indirectly used in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A video jitter detection method, characterized in that: The method comprises: Traversing adjacent frame images in a video stream, the adjacent frame images include a first image and a second image, and a frame order of the first image in the video stream is smaller than a frame order of the second image in the video stream; Matching each first pixel in the first image with each second pixel in the second image to obtain a matched target pixel pair; Determine a target adjacent frame image having an offset angle in the video stream according to coordinate information of each pixel in the target pixel pair and quantity information of the target pixel pair; In response to the number of image pairs of the target adjacent frame images being greater than a preset image number threshold, video jitter detection processing is performed according to the offset angles corresponding to the target adjacent frame images to obtain a video jitter detection result.

2. The video jitter detection method according to claim 1, characterized in that: The target pixel pair includes a first target pixel and a second target pixel that match each other, and the step of matching each first pixel in the first image with each second pixel in the second image to obtain a matched target pixel pair includes: Selecting a first key pixel point from the first image according to the attribute information of each first pixel point in the first image; Selecting a second key pixel point from the second image according to the attribute information of each second pixel point in the second image; The first target pixel point is determined from the first image and the second target pixel point matching the first target pixel point is determined from the second image according to the feature similarity between the pixel features of each first key pixel point in the first image and the pixel features of each second key pixel point in the second image.

3. The video jitter detection method according to claim 2, characterized in that: The attribute information includes a key degree, and the step of selecting a first key pixel point from the first image according to the attribute information of each first pixel point in the first image includes: Determining whether a criticality of each first pixel in the first image is greater than a critical threshold; If so, the corresponding first pixel point is determined as the first key pixel point.

4. The video jitter detection method according to claim 2, characterized in that: The step of determining the first target pixel point from the first image according to the feature similarity between the pixel features of each first key pixel point in the first image and the pixel features of each second key pixel point in the second image, and determining the second target pixel point matching the first target pixel point from the second image comprises: Calculating feature similarities between pixel features of each first key pixel in the first image and pixel features of each second key pixel in the second image to obtain at least one feature similarity corresponding to each first key pixel; Sorting at least one feature similarity corresponding to the first key pixel point to obtain a feature similarity sequence; A first key pixel point and a second key pixel point corresponding to the feature similarity in the feature similarity sequence and located in a preset sequence are respectively determined as the first target pixel point and the second target pixel point.

5. The video jitter detection method according to claim 1, characterized in that: The quantity information includes a quantity, the target pixel point pair includes a first target pixel point and a second target pixel point that match each other, and the step of determining the target adjacent frame image with an offset angle in the video stream according to the coordinate information of each pixel point in the target pixel point pair and the quantity information of the target pixel point pair includes: Determine a coordinate offset value of the second target pixel point relative to the first target pixel point according to the coordinate information of the first target pixel point and the coordinate information of the second target pixel point; In response to the number of target pixel pairs whose coordinate offset values ​​are greater than a preset coordinate offset threshold being greater than a preset pixel number threshold, determining an offset angle between the adjacent frame images according to the coordinate offset values; The adjacent frame image corresponding to the offset angle is determined as the target adjacent frame image.

6. The video jitter detection method according to claim 5, characterized in that: The step of determining the offset angle between adjacent frame images according to the coordinate offset value comprises: Performing arc tangent function calculation on the coordinate offset value to obtain an angle value; The angle value is determined as the offset angle between the adjacent frame images.

7. The video jitter detection method according to claim 1, characterized in that: The step of performing video jitter detection processing according to the offset angles corresponding to adjacent frame images of each target to obtain a video jitter detection result includes: Determine the offset direction corresponding to each adjacent frame image of the target according to the offset angle corresponding to each adjacent frame image of the target; Determining a target offset direction change sequence of the video stream according to offset directions corresponding to adjacent frame images of each target; In response to the existence of the same change trend in the target offset direction change sequence, it is determined that the video jitter detection result is that jitter occurs in the video stream.

8. The video jitter detection method according to claim 7, characterized in that: The step of determining the target offset direction change sequence of the video stream according to the offset directions corresponding to each target adjacent frame image comprises: Sorting the offset directions corresponding to the target adjacent frame images according to the frame sequence of the first image in the corresponding target adjacent frame images to obtain the offset direction change sequence of the video stream; Deduplication processing is performed on adjacent and identical offset directions in the offset direction change sequence of the video stream to obtain a target offset direction change sequence of the video stream.

9. An electronic device, characterized in that: include: A memory and a processor, wherein the memory stores program instructions, and the processor retrieves the program instructions from the memory to execute the method according to any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that: include: Program data is stored, and when the program data is executed by a processor, it is used to implement the method according to any one of claims 1 to 8.

Images