Video mosaic detection method based on template matching and terminal

Abstract

The invention discloses a video mosaic detection method and terminal based on template matching, and the method comprises the steps: decoding an obtained video stream to obtain multiple frames of original images, and calculating the gray gradient map of each frame of original image; based on a first threshold value and a second threshold value, carrying out binaryzation on the gray scale gradient image to obtain a first edge image and a second edge image; preprocessing the first edge image and the second edge image to obtain a horizontal shallow edge image and a vertical shallow edge image; generating a mosaic point image according to the horizontal shallow edge image and the vertical shallow edge image, and determining the pixel similarity in a target area corresponding to a target mosaic point in the mosaic point image through a template matching method; and if the pixel similarity is smaller than the preset template similarity, marking the target mosaic point corresponding to the pixel similarity as the target angular point of the mosaic. According to the invention, mosaics caused by poor video coding quality can be effectively detected, and the detection accuracy is improved.

Description

[0001] This case is a divisional application based on the invention patent with application date of 2024-7-26, application number 202411008375.2, and name “A video mosaic detection method and terminal” as the parent case. Technical Field

[0002] The present invention relates to the technical field of video processing, and in particular to a video mosaic detection method and a terminal based on template matching. Background Art

[0003] In existing video processing technologies, mosaic phenomenon is common when video compression is distorted or video quality is reduced. Mosaic phenomenon not only affects the audience's viewing experience, but may also cause trouble in understanding and analyzing the video content. In order to improve the video quality, it is necessary to detect the mosaic area in the video, so as to adopt the corresponding repair method to reduce the mosaic phenomenon in the video. For the detection of mosaic phenomenon, the current technology mainly focuses on detecting mosaics with specific patterns added artificially, while the detection of mosaics caused by poor video encoding quality is relatively rare, and the current mosaic detection method has a high false detection rate for such mosaics. Summary of the invention

[0004] The technical problem to be solved by the present invention is: a video mosaic detection method and terminal based on template matching, which can effectively detect mosaics caused by poor video encoding quality and improve detection accuracy and efficiency.

[0005] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0006] A video mosaic detection method based on template matching, comprising:

[0007] Decoding the acquired video stream to obtain multiple frames of original images, and calculating the grayscale gradient map of each frame of the original image;

[0008] Binarize the grayscale gradient map based on a first threshold and a second threshold respectively to obtain a first edge map and a second edge map;

[0009] Preprocessing the first edge map and the second edge map to obtain a horizontal shallow edge map and a vertical shallow edge map;

[0010] Generate a mosaic point image according to the horizontal shallow edge map and the vertical shallow edge map, and determine the pixel similarity in the target area corresponding to the target mosaic point in the mosaic point image by a template matching method;

[0011] If the pixel similarity is less than the preset template similarity, the target mosaic point corresponding to the pixel similarity is marked as the target corner point of the mosaic.

[0012] In order to solve the above technical problems, another technical solution adopted by the present invention is:

[0013] A video mosaic detection terminal based on template matching includes a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, each step of the above-mentioned video mosaic detection method based on template matching is implemented.

[0014] The beneficial effect of the present invention is that: first, by calculating the grayscale gradient map of each frame of the original image, the degree of change of the pixel grayscale value in the original image is reflected, so as to clarify the edge information in the image. Then, by binarizing the grayscale gradient map, the edge and non-edge information in the image are reflected. Due to the poor quality of video encoding, the mosaic edge gradient is not a particularly clear strong edge, but it is not a texture with a particularly small gradient. Therefore, the present invention performs binarization based on two thresholds and then performs preprocessing, excluding strong edges with particularly clear gradients and textures with particularly small gradients, so as to reflect the shallow edges of the mosaic, thereby improving the accuracy of mosaic detection. Finally, a mosaic point image is generated according to the horizontal shallow edge map and the vertical shallow edge map, that is, the splicing intersection image between the mosaics, and the target corner point is determined by the template matching method. The detection speed is fast and the detection efficiency can be improved. At the same time, the present invention utilizes the pixel characteristics of the mosaic and introduces pixel similarity to determine the target corner point, further ensuring the detection accuracy of the mosaic. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 A flow chart of the steps of a video mosaic detection method based on template matching provided by an embodiment of the present invention;

[0016] Figure 2 A schematic diagram of a mosaic template frame provided by an embodiment of the present invention;

[0017] Figure 3 A schematic diagram of the structure of a video mosaic detection terminal based on template matching provided by an embodiment of the present invention;

[0018] Description of labels:

[0019] 100. A video mosaic detection terminal based on template matching; 101. Memory; 102. Processor. DETAILED DESCRIPTION

[0020] In order to explain the technical content, achieved objectives and effects of the present invention in detail, the following is an explanation in conjunction with the implementation modes and the accompanying drawings.

[0021] An embodiment of the present invention provides a video mosaic detection method based on template matching, comprising:

[0022] Decoding the acquired video stream to obtain multiple frames of original images, and calculating the grayscale gradient map of each frame of the original image;

[0023] Binarize the grayscale gradient map based on a first threshold and a second threshold respectively to obtain a first edge map and a second edge map;

[0024] Preprocessing the first edge map and the second edge map to obtain a horizontal shallow edge map and a vertical shallow edge map;

[0025] Generate a mosaic point image according to the horizontal shallow edge map and the vertical shallow edge map, and determine the pixel similarity in the target area corresponding to the target mosaic point in the mosaic point image by a template matching method;

[0026] If the pixel similarity is less than the preset template similarity, the target mosaic point corresponding to the pixel similarity is marked as the target corner point of the mosaic.

[0027] From the above description, it can be seen that the beneficial effect of the present invention is: first, by calculating the grayscale gradient map of each frame of the original image, the degree of change of the pixel grayscale value in the original image is reflected, so as to clarify the edge information in the image. Then, by binarizing the grayscale gradient map, the edge and non-edge information in the image are reflected. Due to the poor quality of video encoding, the mosaic edge gradient is not a particularly clear strong edge, but it is not a texture with a particularly small gradient. Therefore, the present invention performs binarization based on two thresholds and then performs preprocessing, excluding strong edges with particularly clear gradients and textures with particularly small gradients, so as to reflect the shallow edges of the mosaic, thereby improving the accuracy of mosaic detection. Finally, a mosaic point image is generated according to the horizontal shallow edge map and the vertical shallow edge map, that is, the splicing intersection image between mosaics, and the target mosaic point is determined by the template matching method. The detection speed is fast and the detection efficiency can be improved. At the same time, the present invention utilizes the pixel characteristics of the mosaic and introduces pixel similarity to determine the target corner point, further ensuring the detection accuracy of the mosaic.

[0028] Furthermore, the step of respectively calculating the grayscale gradient map of each frame of the original image includes:

[0029] Convert each frame of the original image into a grayscale image;

[0030] A horizontal grayscale gradient map in the horizontal direction and a vertical grayscale gradient map in the vertical direction of the grayscale image are calculated respectively.

[0031] From the above description, it can be seen that respectively calculating the grayscale gradient map of the grayscale image in the horizontal direction and the vertical direction can better capture the edge information in the image.

[0032] Furthermore, binarizing the grayscale gradient map based on the first threshold and the second threshold to obtain the first edge map and the second edge map includes:

[0033] Binarize the horizontal grayscale gradient map and the vertical grayscale gradient map based on the first threshold to obtain a horizontal first edge map and a vertical first edge map;

[0034] The horizontal grayscale gradient map and the vertical grayscale gradient map are binarized based on the second threshold to obtain a horizontal second edge map and a vertical second edge map.

[0035] From the above description, it can be seen that the mosaic edge gradient caused by poor video encoding quality is not a particularly clear strong edge, nor is it a texture with a particularly small gradient, but a shallow edge with a moderate gradient. Therefore, binarization based on two different thresholds can respectively exclude strong edges with particularly clear grayscale gradients and textures with particularly small grayscale gradients, thereby improving the mosaic edge detection effect caused by poor video encoding quality.

[0036] Further, the preprocessing of the first edge map and the second edge map to obtain a horizontal shallow edge map and a vertical shallow edge map includes:

[0037] Performing an XOR operation on the first horizontal edge map and the second horizontal edge map to obtain a horizontal shallow edge map;

[0038] An XOR operation is performed on the vertical first edge map and the vertical second edge map to obtain a vertical shallow edge map.

[0039] From the above description, it can be seen that the edge maps under different thresholds are XORed, so as to combine the edge characteristics of the two to obtain a shallow edge map with higher accuracy.

[0040] Further, generating a mosaic point image according to the horizontal shallow edge map and the vertical shallow edge map includes:

[0041] Traversing and calculating the row pixel average value of each row of the horizontal shallow edge map, and if the row pixel average value is greater than or equal to a preset third threshold, generating a horizontal edge line corresponding to the row;

[0042] Traversing and calculating the column pixel average of each column of the vertical shallow edge map, if the column pixel average is greater than or equal to the third threshold, generating a vertical edge line corresponding to the column;

[0043] The intersection of the horizontal edge line and the vertical edge line is marked as an original mosaic point to obtain a mosaic point image.

[0044] From the above description, it can be seen that for a row of pixels or a column of pixels whose pixel average value is greater than or equal to the third threshold, a corresponding edge line is generated, and the possible intersection points between mosaics are determined based on the intersection between the horizontal edge line and the vertical edge line, and then the next step of detection is performed, thereby improving the accuracy of mosaic detection.

[0045] Furthermore, the determining the pixel similarity in the target area corresponding to the target mosaic point in the mosaic point image by the template matching method includes:

[0046] Determine a target mosaic point in the mosaic point image and a mosaic template frame corresponding to the target mosaic point according to pixel values ​​of the horizontal shallow edge map and the vertical shallow edge map in the horizontal direction and the vertical direction respectively;

[0047] In the original image, the target mosaic point is used as a template corner point of the mosaic template frame, the pixel area selected by the mosaic template frame is marked as the target area, and the pixel similarity of the target area is calculated.

[0048] From the above description, it can be seen that based on the pixel values ​​of the horizontal shallow edge map and the vertical shallow edge map, some erroneous original mosaic points are screened out to obtain the target mosaic points, thereby improving the accuracy of subsequent template matching. At the same time, the pixel characteristics of the mosaic are used to introduce pixel similarity to determine the target corner points of the mosaic. When the pixel similarity selected by the mosaic template frame is high, it indicates that the pixels in the area are the mosaic area. When the pixel similarity selected by the mosaic template is low, it indicates that the pixels in the area are not the mosaic area, further ensuring the accuracy of mosaic detection.

[0049] Further, determining the target mosaic point in the mosaic point image and the mosaic template frame corresponding to the target mosaic point according to the pixel values ​​of the horizontal shallow edge map and the vertical shallow edge map in the horizontal direction and the vertical direction respectively includes:

[0050] Creating a standard coordinate system using each original mosaic point in the mosaic point image as a coordinate origin;

[0051] Counting the number of first pixels of the horizontal shallow edge image whose pixel values ​​are continuously edge values ​​in the horizontal positive direction of the standard coordinate system;

[0052] Counting the number of second pixels of the vertical shallow edge image whose pixel values ​​are continuously edge values ​​in the vertical negative direction of the standard coordinate system;

[0053] If the first number of pixels and the second number of pixels are both greater than a preset fourth threshold, the original mosaic point is marked as a target mosaic point, and a mosaic template frame corresponding to the target mosaic point is generated with the minimum value of the first number of pixels and the second number of pixels as the side length.

[0054] From the above description, it can be seen that by counting the number of pixels whose pixel values ​​are continuously edge values, the size of the edge area corresponding to all original mosaic points is determined. When the number of pixels is too small, it means that the current area size does not meet the size of the mosaic, so the original mosaic point is excluded. At the same time, the minimum value of the number of pixels is set to the side length of the mosaic template frame to ensure that the area selected by the mosaic template frame belongs to the edge area, thereby improving the detection accuracy.

[0055] Further, the calculating the pixel similarity of the target area includes:

[0056] Calculate the R channel pixel variance, the G channel pixel variance and the B channel pixel variance of the original image in the target area respectively;

[0057] The average value of the R channel pixel variance, the G channel pixel variance and the B channel pixel variance is calculated to obtain the pixel similarity of the target area.

[0058] Furthermore, it also includes:

[0059] The mosaic level of the video stream is detected according to a preset classification threshold and the number of the target corner points.

[0060] From the above description, we can know that the color brightness of the three RGB channels is used to characterize the pixel similarity. The smaller the variance value, the higher the pixel similarity. Therefore, when the pixel similarity is less than the preset threshold, the target mosaic point is identified as the target corner point of the mosaic. The pixel similarity is used to further confirm the mosaic pixel information and improve the detection accuracy.

[0061] Another embodiment of the present invention provides a video mosaic detection terminal based on template matching, including a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, each step of the above-mentioned video mosaic detection method based on template matching is implemented.

[0062] From the above description, it can be seen that the beneficial effect of the present invention is: first, by calculating the grayscale gradient map of each frame of the original image, the degree of change of the pixel grayscale value in the original image is reflected, so as to clarify the edge information in the image. Then, by binarizing the grayscale gradient map, the edge and non-edge information in the image are reflected. Due to the poor quality of video encoding, the mosaic edge gradient is not a particularly clear strong edge, but it is not a texture with a particularly small gradient. Therefore, the present invention performs binarization based on two thresholds and then performs preprocessing, excluding strong edges with particularly clear gradients and textures with particularly small gradients, so as to reflect the shallow edges of the mosaic, thereby improving the accuracy of mosaic detection. Finally, a mosaic point image is generated according to the horizontal shallow edge map and the vertical shallow edge map, that is, the splicing intersection image between mosaics, and the target mosaic point is determined by the template matching method. The detection speed is fast and the detection efficiency can be improved. At the same time, the present invention utilizes the pixel characteristics of the mosaic, introduces pixel similarity to determine the target mosaic point, and further ensures the detection accuracy of the mosaic.

[0063] The embodiments of the present invention provide a video mosaic detection method and terminal based on template matching, which can be applied to the video processing module of a camera and can effectively detect whether a video shot by the camera has mosaics caused by poor video encoding quality. The method is described below through specific embodiments:

[0064] Please refer to Figure 1 , Embodiment 1 of the present invention is:

[0065] A video mosaic detection method based on template matching, comprising:

[0066] S110, decoding the acquired video stream to obtain multiple frames of original images, and calculating the grayscale gradient map of each frame of the original image.

[0067] Specifically, in step S110, respectively calculating the grayscale gradient map of each frame of the original image includes:

[0068] S1101, converting each frame of the original image into a grayscale image.

[0069] In some embodiments, if the original image is in jpg format, before step S1101, the original image is converted from jpg format to yuv format, where y represents brightness, that is, grayscale value. u and v represent chroma, which is used to describe the color and saturation of the image and is used to specify the color of the pixel. u represents the chroma component of the blue part minus the brightness, and v represents the chroma component of the red part minus the brightness. In yuv format, only the y component needs to be retained to obtain the grayscale image corresponding to the original image.

[0070] S1102, respectively calculating a horizontal grayscale gradient map in the horizontal direction and a vertical grayscale gradient map in the vertical direction of the grayscale image.

[0071] In some embodiments, the horizontal grayscale gradient map in the horizontal direction and the vertical grayscale gradient map in the vertical direction of the grayscale image can be calculated respectively by using the Sobel operator or the Scharr operator. Among them, since the Scharr operator responds more strongly to the edge of the image than the Sobel operator, it has high accuracy in image edge detection and is more sensitive to changes in edge details, and can better capture edge information in the image.

[0072] In some embodiments, the Scharr convolution kernel in the horizontal direction is specifically:

[0073]

[0074] The Scharr convolution kernel in the vertical direction is specifically:

[0075]

[0076] Then step S1102 is specifically as follows: traverse each pixel in the grayscale image, perform convolution operation through the convolution kernel of the Scharr operator in the horizontal direction, and perform weighted summation based on the grayscale value of each pixel and then take the absolute value processing to obtain the grayscale gradient value of the pixel in the horizontal direction; after obtaining the grayscale gradient values ​​of all pixels in the grayscale image in the horizontal direction, the horizontal grayscale gradient map GradImgX can be obtained. Similarly, traverse each pixel in the grayscale image, perform convolution operation through the convolution kernel of the Scharr operator in the vertical direction, and perform weighted summation based on the grayscale value of each pixel and then take the absolute value processing to obtain the grayscale gradient value of the pixel in the vertical direction; after obtaining the grayscale gradient values ​​of all pixels in the grayscale image in the vertical direction, the vertical grayscale gradient map GradImgY can be obtained.

[0077] S120 , binarizing the grayscale gradient map based on a first threshold and a second threshold respectively to obtain a first edge map and a second edge map.

[0078] In some embodiments, the value interval of the first threshold a and the second threshold b is [10, 25] to ensure that strong edges and textures in the image can be excluded and only shallow edges are retained.

[0079] Specifically, the step S120 includes:

[0080] S1201. Binarize the horizontal grayscale gradient map and the vertical grayscale gradient map based on the first threshold to obtain a horizontal first edge map and a vertical first edge map.

[0081] In some embodiments, step S1201 is specifically as follows: traverse each pixel in the horizontal grayscale gradient map GradImgX and the vertical grayscale gradient map GradImgY, and if the grayscale gradient value of the pixel is greater than the first threshold a, mark the pixel as an edge pixel, that is, set the pixel value to an edge value, and the edge value is 255; if the grayscale gradient value of the pixel is less than or equal to the first threshold a, mark the pixel as a non-edge pixel, that is, set the pixel value to a non-edge value, and the non-edge value is 0. Thus, the horizontal first edge map Cx and the vertical first edge map Cy are obtained.

[0082] S1202: Binarize the horizontal grayscale gradient map and the vertical grayscale gradient map based on the second threshold to obtain a horizontal second edge map and a vertical second edge map.

[0083] In some embodiments, similar to the binarization by the first threshold, step S1202 is specifically as follows: traverse each pixel in the horizontal grayscale gradient map GradImgX and the vertical grayscale gradient map GradImgY, and if the grayscale gradient value of the pixel is greater than the second threshold b, mark the pixel as an edge pixel, that is, set the pixel value to an edge value, and the edge value is 255; if the grayscale gradient value of the pixel is less than or equal to the second threshold b, mark the pixel as a non-edge pixel, that is, set the pixel value to a non-edge value, and the non-edge value is 0. Thus, the horizontal second edge map Dx and the vertical second edge map Dy are obtained.

[0084] S130: Preprocess the first edge map and the second edge map to obtain a horizontal shallow edge map and a vertical shallow edge map.

[0085] Specifically, the step S130 includes:

[0086] S1301. Perform an XOR operation on the first horizontal edge map and the second horizontal edge map to obtain a horizontal shallow edge map.

[0087] In some embodiments, pixels of the first horizontal edge map Cx and pixels of the second horizontal edge map Dx are subjected to an XOR operation bit by bit to obtain a horizontal shallow edge map Ex.

[0088] S1302: Perform an XOR operation on the vertical first edge map and the vertical second edge map to obtain a vertical shallow edge map.

[0089] In some embodiments, pixels of the vertical first edge map Cy and pixels of the vertical second edge map Dy are subjected to an XOR operation bit by bit to obtain a vertical shallow edge map Ey.

[0090] S140, generating a mosaic point image according to the horizontal shallow edge map and the vertical shallow edge map, and determining the pixel similarity in the target area corresponding to the target mosaic point in the mosaic point image by a template matching method.

[0091] S150: If the pixel similarity is less than a preset template similarity, mark the target mosaic point corresponding to the pixel similarity as a target corner point of the mosaic.

[0092] In some embodiments, the template similarity r has a value interval of [10, 20].

[0093] In some embodiments, the smaller the value of the pixel similarity is, the higher the pixel similarity of the target area is. The higher the pixel similarity is, the higher the target mosaic point is, which indicates that the target mosaic point is a target corner point of the mosaic.

[0094] In an optional embodiment, the method further includes:

[0095] S160: Detect the mosaic level of the video stream according to a preset classification threshold and the number of the target corner points.

[0096] In some embodiments, multiple level thresholds are set, including a first level threshold l1, a second level threshold l2, and a third level threshold l3, wherein the first level threshold l1 is less than the second level threshold l2, and the second level threshold l2 is less than the third level threshold l3. When the number of target corner points in the video stream is greater than the first level threshold l1 and less than the second level threshold l2, it indicates that the mosaic severity level of the video stream is level one; similarly, when the number of target corner points in the video stream is greater than the second level threshold l2 and less than the third level threshold l3, it indicates that the mosaic severity level of the video stream is level two. In this way, the mosaic severity of the current video stream caused by the video encoding quality is detected.

[0097] Please refer to Figure 2 , Embodiment 2 of the present invention is:

[0098] A video mosaic detection method based on template matching is different from the first embodiment in that the specific steps of the above step S140 are limited.

[0099] Specifically, the step S140 includes:

[0100] S210, traversing and calculating the row pixel average value of each row of the horizontal shallow edge map, and if the row pixel average value is greater than or equal to a preset third threshold, generating a horizontal edge line corresponding to the row.

[0101] In some embodiments, the value interval of the third threshold value f is between 15% and 40% of the edge value, that is, [255*15%, 255*40%].

[0102] In some embodiments, on the horizontal shallow edge map Ex, the pixel average of all pixels located in the same row in the horizontal direction is calculated from left to right. If the pixel average of the row is greater than or equal to the third threshold f, a horizontal line with the same traversal direction is drawn on the row. The horizontal line is the horizontal edge line of the pixels in the row.

[0103] S220, traversing and calculating the column pixel average of each column of the vertical shallow edge map, if the column pixel average is greater than or equal to the third threshold, generating a vertical edge line corresponding to the column.

[0104] In some embodiments, on the vertical shallow edge map Ey, the pixel average of all pixels located in the same column in the vertical direction is calculated from top to bottom. If the pixel average of the column is greater than or equal to a third threshold f, a vertical line with the same traversal direction is drawn on the column. The vertical line is the vertical edge line of the pixels in the column.

[0105] In some embodiments, after the horizontal edge lines and the vertical edge lines are generated, a horizontal edge line map Fx corresponding to the horizontal shallow edge map Ex and a vertical edge line map Fy corresponding to the vertical shallow edge map Ey can be obtained.

[0106] S230: Mark the intersection of the horizontal edge line and the vertical edge line as an original mosaic point to obtain a mosaic point image.

[0107] In some embodiments, the mosaic point image includes a horizontal mosaic point image and a vertical mosaic point image. Specifically, the vertical edge lines in the vertical edge line image Fy are first superimposed on the horizontal edge line image Fx, thereby marking the original mosaic points in the horizontal shallow edge image Ex, and obtaining the horizontal mosaic point image Hx. At the same time, the horizontal edge lines in the horizontal edge line image Fx are superimposed on the vertical edge line image Fy, thereby marking the original mosaic points in the vertical shallow edge image Ey, and obtaining the vertical mosaic point image Hy.

[0108] S240, determining a target mosaic point in the mosaic point image and a mosaic template frame corresponding to the target mosaic point according to the pixel values ​​of the horizontal shallow edge map and the vertical shallow edge map in the horizontal direction and the vertical direction respectively.

[0109] Specifically, the step S240 includes:

[0110] S2401. Create a standard coordinate system using each original mosaic point in the mosaic point image as a coordinate origin.

[0111] In some embodiments, all original mosaic points are traversed starting from the upper left corner of the mosaic point image, and the horizontal positive direction of the standard coordinate system is the right side of the original mosaic point, and the vertical positive direction of the standard coordinate system is the top of the original mosaic point.

[0112] S2402. Count the number of first pixels of the horizontal shallow edge image whose pixel values ​​are continuously edge values ​​in the horizontal positive direction of the standard coordinate system.

[0113] In some embodiments, after a standard coordinate system is created in the horizontal mosaic point image Hx with each original mosaic point as a coordinate origin, the number of first pixels Gx whose pixel values ​​are continuously 255 is counted along the right side of the original mosaic point.

[0114] S2403. Count the number of second pixels in the vertical shallow edge image whose pixel values ​​are continuously edge values ​​in the vertical negative direction of the standard coordinate system.

[0115] In some embodiments, after a standard coordinate system is created in the vertical mosaic point image Hy with each original mosaic point as the coordinate origin, a second number of pixels Gy having consecutive pixel values ​​of 255 is counted along the bottom of the original mosaic point.

[0116] S2404. If the first number of pixels and the second number of pixels are both greater than a preset fourth threshold, the original mosaic point is marked as a target mosaic point, and a mosaic template frame corresponding to the target mosaic point is generated with the minimum value of the first number of pixels and the second number of pixels as the side length.

[0117] In some embodiments, the fourth threshold h is in the range of [10, 20]. If both the first pixel number Gx and the second pixel number Gy are greater than the fourth threshold h, the original mosaic point is marked as the target mosaic point Q(i, j).

[0118] In some embodiments, the mosaic template frame is a square, so the side length of the mosaic template frame is set to the minimum value of the first number of pixels Gx(i,j) corresponding to the target mosaic point Q(i,j) and the second number of pixels Gy(i,j), that is, the side length of the mosaic template frame corresponding to the target mosaic point Q(i,j) regionEdgeLength(i,j) = min(Gx(i,j),Gy(i,j)).

[0119] S250. In the original image, use the target mosaic point as a template corner point of the mosaic template frame, mark the pixel area selected by the mosaic template frame as a target area, and calculate the pixel similarity of the target area.

[0120] In some embodiments, after marking the target mosaic point in the horizontal mosaic point image Hx or the vertical mosaic point image Hy, the coordinates of the target mosaic point in the image Hx or Hy are determined to be Q(i, j). After marking the corresponding target mosaic point in the original image by the coordinates Q(i, j), the target mosaic point in the original image is used as the template corner point of the mosaic template frame, and the pixel area selected by the mosaic template frame is marked as the target area.

[0121] In some embodiments, Figure 2 As shown, the target mosaic point Q(i, j) is used as the upper left corner point of the mosaic template frame, and then the mosaic template frame corresponding to the target mosaic point Q(i, j) can be generated based on the side length obtained in step S2404, wherein the pixel area located in the middle of the mosaic template frame is the target area.

[0122] Specifically, in step S250, calculating the pixel similarity of the target area includes:

[0123] S2501 , respectively calculating the R channel pixel variance, the G channel pixel variance, and the B channel pixel variance of the original image in the target area.

[0124] S2502: Calculate the average value of the R channel pixel variance, the G channel pixel variance, and the B channel pixel variance to obtain the pixel similarity of the target area.

[0125] In some embodiments, step 2502 is specifically:

[0126] Pixel similarity RegionSimilarity(i,j)=[R.StdDev(region(i,j))+G.StdDev(region(i,j))+B.StdDev(region(i,j))] / 3. R.StdDev, G.StdDev and B.StdDev represent the pixel variance of R channel, pixel variance of G channel and pixel variance of B channel respectively, and region(i,j) represents the target region corresponding to the target mosaic point Q(i,j).

[0127] Please refer to Figure 3 , Embodiment 3 of the present invention is:

[0128] A video mosaic detection terminal 100 based on template matching includes a memory 101, a processor 102, and a computer program stored in the memory 101 and running on the processor 102. When the processor 102 executes the computer program, each step of a video mosaic detection method based on template matching in embodiment 1 is implemented.

[0129] In summary, the present invention provides a video mosaic detection method and terminal based on template matching, which calculates the grayscale gradient map of the original image based on the Scharr operator, so as to better capture the edge information in the image. At the same time, based on two different thresholds, binarization is performed respectively and then an XOR operation is performed to obtain a shallow edge map, so as to exclude strong edges with particularly clear gradients and textures with particularly small gradients, so as to capture the shallow edges of the mosaic caused by poor video encoding quality in the image, thereby improving the accuracy of mosaic detection. According to the pixel average values ​​of the horizontal shallow edge map and the vertical shallow edge map, edge lines in different directions are generated to determine the original mosaic point image of the mosaic, that is, the splicing intersection image between the mosaics. The mosaic template frame is determined based on the mosaic point image, and the target corner point is determined by the template matching method. The detection speed is fast and the resource occupation is small, which can improve the detection efficiency, thereby meeting the needs of real-time video detection. At the same time, the present invention utilizes the pixel characteristics of the mosaic, introduces pixel similarity to determine the target corner point, and further ensures the detection accuracy of the mosaic. In addition, while detecting video mosaics, the present invention also determines the severity of mosaics by the number of target corner points to evaluate the current video quality.

[0130] The above descriptions are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent transformations made using the contents of the present invention's specification and drawings, or directly or indirectly applied in related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A video mosaic detection method based on template matching, characterized in that: include: Decoding the acquired video stream to obtain multiple frames of original images, and calculating the grayscale gradient map of each frame of the original image; Binarize the grayscale gradient map based on a first threshold and a second threshold respectively to obtain a first edge map and a second edge map; Preprocessing the first edge map and the second edge map to obtain a horizontal shallow edge map and a vertical shallow edge map; Generate a mosaic point image according to the horizontal shallow edge map and the vertical shallow edge map, and determine the pixel similarity in the target area corresponding to the target mosaic point in the mosaic point image by a template matching method; If the pixel similarity is less than the preset template similarity, marking the target mosaic point corresponding to the pixel similarity as the target corner point of the mosaic; The mosaic point image includes a horizontal mosaic point image and a vertical mosaic point image; Determining the pixel similarity in the target area corresponding to the target mosaic point in the mosaic point image by the template matching method includes: Determine a target mosaic point and a mosaic template frame corresponding to the target mosaic point according to the pixel values ​​of the horizontal mosaic point image and the vertical mosaic point image in the horizontal direction and the vertical direction respectively; In the original image, the target mosaic point is used as a template corner point of the mosaic template frame, the pixel area selected by the mosaic template frame is marked as the target area, and the pixel similarity of the target area is calculated.

2. The video mosaic detection method based on template matching according to claim 1, characterized in that: The step of respectively calculating the grayscale gradient map of each frame of the original image comprises: Convert each frame of the original image into a grayscale image; A horizontal grayscale gradient map in the horizontal direction and a vertical grayscale gradient map in the vertical direction of the grayscale image are calculated respectively.

3. The video mosaic detection method based on template matching according to claim 2, characterized in that: The step of binarizing the grayscale gradient map based on the first threshold and the second threshold to obtain the first edge map and the second edge map comprises: Binarize the horizontal grayscale gradient map and the vertical grayscale gradient map based on the first threshold to obtain a horizontal first edge map and a vertical first edge map; The horizontal grayscale gradient map and the vertical grayscale gradient map are binarized based on the second threshold to obtain a horizontal second edge map and a vertical second edge map.

4. The video mosaic detection method based on template matching according to claim 3 is characterized in that: The preprocessing of the first edge map and the second edge map to obtain a horizontal shallow edge map and a vertical shallow edge map comprises: Performing an XOR operation on the first horizontal edge map and the second horizontal edge map to obtain a horizontal shallow edge map; An XOR operation is performed on the vertical first edge map and the vertical second edge map to obtain a vertical shallow edge map.

5. The video mosaic detection method based on template matching according to claim 1, characterized in that: Generating a mosaic point image according to the horizontal shallow edge map and the vertical shallow edge map comprises: Traversing and calculating the row pixel average value of each row of the horizontal shallow edge map, if the row pixel average value is greater than or equal to a preset third threshold, generating a horizontal edge line corresponding to the row, and obtaining a horizontal edge line map corresponding to the horizontal shallow edge map; Traversing and calculating the column pixel average of each column of the vertical shallow edge map, if the column pixel average is greater than or equal to the third threshold, generating a vertical edge line corresponding to the column, and obtaining a vertical edge line map corresponding to the vertical shallow edge map; The intersection of the horizontal edge line and the vertical edge line is marked as an original mosaic point to obtain a mosaic point image.

6. The video mosaic detection method based on template matching according to claim 5, characterized in that: The step of marking the intersection of the horizontal edge line and the vertical edge line as an original mosaic point to obtain a mosaic point image comprises: Superimposing the vertical edge lines in the vertical edge line image on the horizontal edge line image, and marking the intersections of the horizontal edge lines and the vertical edge lines as original mosaic points, to obtain a horizontal mosaic point image; The horizontal edge lines in the horizontal edge line image are superimposed on the vertical edge line image, and the intersections of the horizontal edge lines and the vertical edge lines are marked as original mosaic points to obtain a vertical mosaic point image.

7. The video mosaic detection method based on template matching according to claim 1, characterized in that: In the original image, taking the target mosaic point as the template corner point of the mosaic template frame, marking the pixel area selected by the mosaic template frame as the target area comprises: Determine the coordinates of the target mosaic point in the horizontal mosaic point image or the vertical mosaic point image, mark the corresponding target mosaic point in the original image using the coordinates, then use the target mosaic point in the original image as the template corner point of the mosaic template frame, and mark the pixel area selected by the mosaic template frame as the target area.

8. The video mosaic detection method based on template matching according to claim 1, characterized in that: The step of determining a target mosaic point and a mosaic template frame corresponding to the target mosaic point according to the pixel values ​​of the horizontal mosaic point image and the vertical mosaic point image in the horizontal direction and the vertical direction respectively comprises: After creating a standard coordinate system in the horizontal mosaic point image with each original mosaic point as a coordinate origin, the number of first pixels whose pixel values ​​are continuously edge values ​​is counted along the horizontal positive direction of the original mosaic point; After creating a standard coordinate system in the vertical mosaic point image with each original mosaic point as a coordinate origin, the number of second pixels whose pixel values ​​are continuously edge values ​​is counted along the vertical negative direction of the original mosaic point; If the first number of pixels and the second number of pixels are both greater than a preset fourth threshold, the original mosaic point is marked as a target mosaic point, and a mosaic template frame corresponding to the target mosaic point is generated with the minimum value of the first number of pixels and the second number of pixels as the side length.

9. The video mosaic detection method based on template matching according to claim 1, characterized in that: Also includes: The mosaic level of the video stream is detected according to a preset classification threshold and the number of the target corner points.

10. A video mosaic detection terminal based on template matching, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, each step of the video mosaic detection method based on template matching described in any one of claims 1 to 9 is implemented.

Images