Method and system for detecting static pixel point in video image
A technology of video image and detection method, which is applied in the field of multimedia, and can solve the problems of inability to detect still objects and broken objects in video images
Active Publication Date: 2017-06-13
INST OF AUTOMATION CHINESE ACAD OF SCI
6 Cites 2 Cited by
AI-Extracted Technical Summary
Problems solved by technology
[0003] In order to solve the above-mentioned problems in the prior art, that is, in order to solve the problem that the prior art cannot detect the still objects in the video image, thus caus...
Method used
Because a lot of continuous frames of large-area background are in static state, so the count value can be accumulated to a very high value, if a moving object breaks into suddenly at this moment, because the count value cannot drop below the threshold T in time, so the current frame moving object The corresponding pixels at ...
Abstract
The invention provides a method and a system for detecting a static pixel point in a video image. The method comprises the following steps of computing characteristic values of corresponding neighborhoods of a to-be-processed pixel point in adjacent frames; determining a confidence coefficient rel1 for representing the to-be-processed pixel point as a non-marginal pixel point in a static object and a confidence coefficient rel2 for representing the to-be-processed pixel point as a pixel point in a semitransparent static object according to the characteristic values; selecting a corresponding value from the characteristic values to judge whether the to-be-processed pixel point is the static pixel point according to the rel1 and the rel2; adding one to a count value corresponding to the to-be-processed pixel point when the to-be-processed pixel point is judged as the static pixel point, and subtracting one from the count value when the to-be-processed pixel point is judged to be not the static pixel point; and comparing the count value with a set threshold T, and determining the to-be-processed pixel point as the static pixel point if the count value is greater than T. According to the method and the system, the static pixel point in the video image can be accurately detected, so that the object fragmentation phenomenon in frame rate conversion of the video image is avoided.
Application Domain
Television system detailsImage analysis +1
Technology Topic
Numeric ValueBand counts +2
Image
Examples
- Experimental program(3)
Example
[0024] First embodiment
[0025] Refer to figure 1 , figure 1 The flow chart of the method for detecting still pixels in a video image in the first embodiment of the present invention is shown, and the specific steps are as follows:
[0026] S101: Calculate the feature value of the neighborhood corresponding to the pixel to be processed in the adjacent frame.
[0027] In this embodiment, the above-mentioned feature values include: the normalized cross-correlation coefficient NCC in the neighborhood of the pixel to be processed, the absolute brightness error and SAD used to characterize the similarity of pixels in adjacent frames, and whether the pixel has a boundary The brightness variance varY, the mean MV of the motion vector used to characterize whether the pixel is stationary, and the variance of the motion vector varMV used to characterize whether the motion of the pixel is consistent.
[0028] The calculation formula of brightness SAD is shown in formula (1), where x i Is the brightness value of the pixel in the image block X centered on the pixel to be processed in the current frame curY, y i It is the brightness value of the pixel in the image block Y centered on the pixel at the same position of the pixel to be processed in the previous frame preY.
[0029] SAD(X,Y)=∑ i |x i -y i | (1)
[0030] The mean value of the motion vector meanMV=(meanMVx, meanMVy). meanMVx and meanMVy are the mean values of the motion vector MV (MotionVector) in the neighborhood in the horizontal and vertical directions, respectively. MV is obtained through motion estimation. Motion estimation can use any existing motion estimation method, such as full search, 3DSR and other methods. The motion vector variance varMV is calculated in the horizontal and vertical directions respectively, and the variances in the two directions are added together as the final motion vector variance.
[0031] The normalized cross-correlation number NCC. With the pixel to be processed as the center, the calculation formula is shown in formula (2), where u x And u y It is the mean value of brightness in the calculation window of two adjacent frames.
[0032]
[0033] S102: Determine the confidence rel1 and the confidence rel2 according to the characteristic value.
[0034] In this embodiment, the confidence rel1 is used to characterize that the pixel to be processed is a non-edge pixel in a stationary object, and the confidence rel2 is used to characterize that the pixel to be processed is a pixel in a semi-transparent static object. The specific method to determine the confidence rel1 and the confidence rel2 is:
[0035] When the SAD is less than the set threshold and the meanMV horizontal and vertical components are both 0, rel1 is determined to be 1, and when the SAD is not less than the set threshold or at least one of the meanMV horizontal and vertical components is not 0, rel1 is determined to be 0 ; When the NCC value is greater than the set threshold, rel2 is determined to be 1, and when the NCC value is not greater than the set threshold, rel2 is determined to be 0.
[0036] S103: According to the values of rel1 and rel2, a corresponding value is selected from the characteristic values to determine whether the pixel to be processed is a stationary pixel.
[0037] The above steps are to filter the characteristic values according to the values of rel1 and rel2, and select the corresponding characteristic values to determine whether the pixel to be processed is a stationary pixel. The specific determination process is divided into the following situations:
[0038] When both rel1 and rel2 are 0, judge according to SAD, meanMV, varY and varMV. The judgment method is that if SAD is less than the set threshold, the horizontal and vertical components of meanMV are 0, varY is greater than the set threshold, and varMV is greater than The threshold is set, the pixel to be processed is judged to be a static pixel.
[0039] When rel1 is 1 and rel2 is 0, it means that the pixel to be processed may be a non-edge pixel inside a stationary object. Therefore, in order to correctly determine whether the pixel to be processed is a stationary pixel, the luminance variance varY in the feature value needs to be removed , Judging by the remaining feature value, the judging method is, if the SAD is less than the set threshold, the meanMV horizontal and vertical components are both 0, and varMV is greater than the set threshold, then the pixel to be processed is judged to be a static pixel.
[0040] When rel1 is 0 and rel2 is 1, it means that the pixel to be processed may be a semi-transparent pixel. Therefore, in order to correctly determine whether the pixel to be processed is a static pixel, it is necessary to remove the meanMV of the motion vector in the feature value. Judge by the remaining characteristic value. This is because when a stationary object is in a semi-transparent state, the motion estimation is prone to errors. At this time, the average value of the motion vector of the stationary object in the horizontal and vertical directions may not be zero. If the judgment condition that the motion vector is zero is used, the semi-transparent stationary Objects will be missed. However, the texture of adjacent frames of stationary objects still maintains the characteristic of high similarity. The judgment method is that if the SAD is less than the set threshold, varY is greater than the set threshold, and varMV is greater than the set threshold, then the pixel to be processed is judged to be a static pixel.
[0041] When both rel1 and rel2 are 1, remove the luminance variance varY and the mean MV of the motion vector in the feature value, and judge with the remaining feature value. The judgment method is that if SAD is less than the set threshold, varMV is greater than the set threshold, Then it is determined that the pixel to be processed is a static pixel.
[0042] S104: When it is determined that the pixel to be processed is a static pixel, the count value corresponding to the pixel to be processed is increased by 1, and when it is determined that the pixel to be processed is not a static pixel, the count is decreased by 1.
[0043] For the accumulation of the count value, a set arbitrary constant can be added each time. In this embodiment, 1 is added. Of course, it can be understood that this embodiment does not limit the specific value of the accumulation constant. The principle of subtraction is the same.
[0044] S105: The count value is compared with a set threshold T, and if the count value is greater than T, it is determined that the pixel to be processed is a stationary pixel.
[0045] Since the large-area background is in a static state for many consecutive frames, the count value will accumulate to a very high value. If a moving object is suddenly broken into at this time, because the count value cannot drop below the threshold T in time, the corresponding moving object in the current frame Pixels will be mistakenly detected as static points. In consideration of this kind of situation, in this embodiment, the upper and lower accumulation range limits are added to the count value, and the maximum value and the minimum value of the count value are restricted. Avoid the occurrence of the above situation and improve the accuracy of detection.
[0046] In this embodiment, the feature values of the neighborhoods corresponding to the pixels to be processed in adjacent frames are calculated; according to the feature values, the confidence rel1 and the confidence rel2 are determined; according to the values of rel1 and rel2, the corresponding values are selected from the feature values to determine the Whether the pixel to be processed is a static pixel; when it is determined that the pixel to be processed is a static pixel, the count value corresponding to the pixel to be processed is increased by 1, and when it is determined that the pixel to be processed is not a static pixel, the count is decreased by 1 ; Compare the count value with the set threshold T, if the count value is greater than T, the pixel to be processed is determined to be a stationary pixel. Thereby, the static pixels in the video image can be accurately detected, and the phenomenon of object fragmentation in the video image frame rate conversion can be avoided.
Example
[0047] Second embodiment
[0048] Refer to figure 2 , figure 2 It shows a schematic flowchart of a method for detecting still pixels in a video image in the second embodiment of the present invention, and the specific steps are as follows:
[0049] S201: Calculate the feature value of the neighborhood corresponding to the pixel to be processed in the adjacent frame, and then enter step S202.
[0050] In this embodiment, the above-mentioned feature values include: the normalized cross-correlation coefficient NCC in the neighborhood of the pixel to be processed, the absolute brightness error and SAD used to characterize the similarity of pixels in adjacent frames, and whether the pixel has a boundary The brightness variance varY, the mean MV of the motion vector used to characterize whether the pixel is stationary, and the variance of the motion vector varMV used to characterize whether the motion of the pixel is consistent.
[0051] The calculation formula of brightness SAD is shown in formula (1), where x i Is the brightness value of the pixel in the image block X centered on the pixel to be processed in the current frame curY, y i It is the brightness value of the pixel in the image block Y centered on the pixel at the same position of the pixel to be processed in the previous frame preY.
[0052] SAD(X,Y)=∑ i |x i -y i |(1)
[0053] The mean value of the motion vector meanMV=(meanMVx, meanMVy). meanMVx and meanMVy are the mean values of the motion vector MV (MotionVector) in the neighborhood in the horizontal and vertical directions, respectively. MV is obtained through motion estimation. Motion estimation can use any existing motion estimation method, such as full search, 3DSR and other methods. The motion vector variance varMV is calculated in the horizontal and vertical directions respectively, and the variances in the two directions are added together as the final motion vector variance.
[0054] The normalized cross-correlation number NCC. Taking the pixel to be processed as the center, the calculation formula is shown in formula (2), where u x And u y It is the mean value of brightness in the calculation window of two adjacent frames.
[0055]
[0056] S202: Determine the confidence rel1, the confidence rel2, and the confidence rel3 according to the feature value, and then enter S203.
[0057] In this embodiment, the confidence rel1 is used to characterize the pixel to be processed as a non-edge pixel in a stationary object, the confidence rel2 is used to characterize the pixel to be processed as a pixel in a semi-transparent static object, and the confidence rel3 is to characterize a moving object. The confidence level of a sudden break into a static background. The specific methods for determining the confidence rel1, the confidence rel2, and the confidence rel3 are:
[0058] When the SAD is less than the set threshold and the meanMV horizontal and vertical components are both 0, rel1 is determined to be 1, and when the SAD is not less than the set threshold or at least one of the meanMV horizontal and vertical components is not 0, rel1 is determined to be 0 ; When the NCC value is greater than the set threshold, rel2 is determined to be 1, and when the NCC value is not greater than the set threshold, rel2 is determined to be 0. When the SAD is greater than the set threshold and the varMV is greater than the set threshold, the confidence rel3 is determined to be 1, and when the SAD is not greater than the set threshold or varMV is not greater than the set threshold, it is determined that rel3 is 0.
[0059] S203: According to the values of rel1 and rel2, select a corresponding value from the characteristic values to determine whether the pixel to be processed is a stationary pixel, and then enter S204.
[0060] The above steps are to filter the characteristic values according to the values of rel1 and rel2, and select the corresponding characteristic values to determine whether the pixel to be processed is a stationary pixel. The specific determination process is divided into the following situations:
[0061] When both rel1 and rel2 are 0, judge according to SAD, meanMV, varY and varMV. The judgment method is that if SAD is less than the set threshold, the horizontal and vertical components of meanMV are 0, varY is greater than the set threshold, and varMV is greater than The threshold is set, the pixel to be processed is judged to be a static pixel.
[0062] When rel1 is 1 and rel2 is 0, it means that the pixel to be processed may be a non-edge pixel inside a stationary object. Therefore, in order to correctly determine whether the pixel to be processed is a stationary pixel, the luminance variance varY in the feature value needs to be removed , Judging by the remaining feature value, the judging method is, if the SAD is less than the set threshold, the meanMV horizontal and vertical components are both 0, and varMV is greater than the set threshold, then the pixel to be processed is judged to be a static pixel.
[0063] When rel1 is 0 and rel2 is 1, it means that the pixel to be processed may be a semi-transparent pixel. Therefore, in order to correctly determine whether the pixel to be processed is a static pixel, it is necessary to remove the meanMV of the motion vector in the feature value. Judge by the remaining characteristic value. This is because when a stationary object is in a semi-transparent state, the motion estimation is prone to errors. At this time, the average value of the motion vector of the stationary object in the horizontal and vertical directions may not be zero. If the judgment condition that the motion vector is zero is used, the semi-transparent stationary Objects will be missed. However, the texture of adjacent frames of stationary objects still maintains the characteristic of high similarity. The judgment method is that if the SAD is less than the set threshold, varY is greater than the set threshold, and varMV is greater than the set threshold, then the pixel to be processed is judged to be a static pixel.
[0064] When both rel1 and rel2 are 1, remove the luminance variance varY and the mean MV of the motion vector in the feature value, and judge with the remaining feature value. The judgment method is that if SAD is less than the set threshold, varMV is greater than the set threshold, Then it is determined that the pixel to be processed is a static pixel.
[0065] S204: Determine whether rel3 is 1, if rel3 is 1, go to S205, and if rel3 is not 1, go to S206.
[0066] S205: Decrease the count value corresponding to the pixel to be processed below the T value, and then enter S209.
[0067] Since the large-area background is in a static state for many consecutive frames, the count value will accumulate to a very high value. If a moving object is suddenly broken into at this time, because the count value cannot drop below the threshold T in time, the corresponding moving object in the current frame Pixels will be mistakenly detected as static points. Taking into account this situation, the count value in this embodiment will change according to the value of rel3. When the value of rel3 is 1, it means that a moving object suddenly enters. At this time, the count value directly drops below the T value, thereby effectively avoiding false detections. When the situation occurs, the T value is a preset value. When the count value is greater than the T value, the pixel to be processed is determined to be a stationary pixel, and when the count value is less than the T value, the pixel to be processed is determined to be a non-static pixel.
[0068] S206: Determine whether the count value corresponding to the pixel to be processed is within a preset interval, if the count value is within the preset interval, then proceed to S208, and if the count value is not within the preset interval, then proceed to S207.
[0069] S207: When it is determined that the pixel to be processed is a static pixel, keep the count value unchanged; when it is determined that the pixel to be processed is not a static pixel, keep the count value unchanged, and then enter S209.
[0070] The above steps add upper and lower accumulation limits to the count value, and limit the maximum and minimum count values, so as to further avoid the count value accumulating to a very high value due to the large area background being in a static state for many consecutive frames.
[0071] S208: When it is determined that the pixel to be processed is a stationary pixel, the count value is increased by 1, and when it is determined that the pixel to be processed is not a stationary pixel, the count is decreased by 1, and then the process proceeds to S209.
[0072] S209: Compare the count value with a set threshold value T. If the count value is greater than T, it is determined that the pixel to be processed is a stationary pixel, and then enters S210.
[0073] S210: Mark all pixels in the setting neighborhood of the pixel to be processed as static pixels.
[0074] In this embodiment, when the pixel to be processed is a static pixel, if the SAD is greater than the set threshold, meanMV is less than the set threshold, and the number of static pixels in the set neighborhood of the pixel to be processed is greater than the set threshold. When the threshold is set, all the pixels in the setting neighborhood of the pixel to be processed are marked as static pixels.
[0075] In this embodiment, the feature values of the neighborhoods corresponding to the pixels to be processed in adjacent frames are calculated; the confidence rel1, the confidence rel2, and rel3 are determined according to the feature values; the corresponding values are selected from the feature values according to the values of rel1 and rel2 Determine whether the pixel to be processed is a stationary pixel; and determine the corresponding count value of the pixel to be processed according to the determination result and rel3; finally compare the count value with the set threshold T, if the count value is greater than T, determine the pixel to be processed The dots are static pixels. Thereby, the static pixels in the video image can be accurately detected, and the phenomenon of object fragmentation in the video image frame rate conversion can be avoided.
Example
[0076] The third embodiment
[0077] Refer to image 3 , image 3 It shows a schematic diagram of a system for detecting static pixels in a video image in the third embodiment of the present invention, and the system includes:
[0078] The feature value module 301 is used to calculate the feature value of the neighborhood corresponding to the pixel to be processed in the adjacent frame.
[0079] The confidence module 302 is configured to determine the confidence rel1, the confidence rel2, and the confidence rel3 according to the feature value.
[0080] The judging module 303 is configured to select corresponding values from the characteristic values according to the values of rel1 and rel2 to judge whether the pixel to be processed is a stationary pixel.
[0081] The counting module 304 is configured to add 1 to the count value corresponding to the pixel to be processed when it is determined that the pixel to be processed is a static pixel, and to subtract 1 when it is determined that the pixel to be processed is not a static pixel.
[0082] The determination module 305 is configured to compare the count value with a set threshold T, and if the count value is greater than T, determine that the pixel to be processed is a stationary pixel.
[0083] Further, see Figure 4 , The system also includes: an expansion module 306, for if the pixel to be processed is a static pixel, when the SAD is greater than the set threshold, the meanMV is less than the set threshold, and the pixel to be processed is a static pixel in the set neighborhood When the number of points is greater than the set threshold, all pixels in the set neighborhood are marked as static pixels.
[0084] In this embodiment, the feature values of the neighborhoods corresponding to the pixels to be processed in adjacent frames are calculated; the confidence rel1, the confidence rel2, and rel3 are determined according to the feature values; the corresponding values are selected from the feature values according to the values of rel1 and rel2 Determine whether the pixel to be processed is a stationary pixel; and determine the corresponding count value of the pixel to be processed according to the determination result and rel3; finally compare the count value with the set threshold T, if the count value is greater than T, determine the pixel to be processed The dots are static pixels. Thereby, the static pixels in the video image can be accurately detected, and the phenomenon of object fragmentation in the video image frame rate conversion can be avoided.
PUM
Description & Claims & Application Information
We can also present the details of the Description, Claims and Application information to help users get a comprehensive understanding of the technical details of the patent, such as background art, summary of invention, brief description of drawings, description of embodiments, and other original content. On the other hand, users can also determine the specific scope of protection of the technology through the list of claims; as well as understand the changes in the life cycle of the technology with the presentation of the patent timeline. Login to view more.