Improved adaptive de-interlacing algorithm based on motion estimation

Abstract

The present invention relates to the technical field of computer vision, and in particular to an adaptive deinterlacing algorithm based on motion estimation improvement, comprising the following steps: Step 1: using motion blocks to determine the motion state of pixels; performing motion detection on a four-field motion detection algorithm, respectively calculating a class of absolute difference and SAD of inter-field pixel blocks and a class of absolute pixel difference diff, which are used to detect the degree of motion of pixel blocks, and defining a class of absolute difference and the motion state of targets; Step 2: if the pixel is judged to be in motion, then introducing the intra-field interpolation method and the inter-field interpolation method in the non-motion interpolation compensation method to restore the current field information, and if the pixel is judged to be stationary, directly entering Step 3; Step 3: smoothing the layered artifact effect caused by interpolation through mean filtering. The problem of inaccurate motion judgment caused by environmental changes is solved, and mean filtering is introduced to remove layered artifacts, so that the purpose of deinterlacing can be achieved, and better restoration of interlaced video can be realized.

Description

Technical Field

[0001] The invention relates to the technical field of computer vision, and in particular to an improved adaptive de-interlacing algorithm based on motion estimation. Background Art

[0002] In the field of video technology in surveillance and security, the current mainstream technologies are digital video technology and analog video technology. Although analog video technology is widely used, since analog video data uses interlaced scanning, the odd and even occasions are not consistent, resulting in image tailing, edge feathering, jagged edges and other phenomena.

[0003] At present, the simplest way of deinterlacing algorithm is to use odd field or even field line field replication. Although it can restore the image to a certain extent, since there is only single field information, it is easy to lose image details, resulting in poor resolution. Deinterlacing based on filtering and other methods can also achieve the effect of deinterlacing, but the filtering method will cause edge blur and unsatisfactory image quality restoration. Patent (Sun Guangpei, Chen Wei, Chen Shu, et al., A median filtering video deinterlacing method, CN113852830A, 2021) designs a median filtering video deinterlacing method, which uses the current field, previous field and next field data to perform median filtering in the spatial domain and time domain. Compared with the method that only uses single field data, it can restore more detail information, and at the same time, it can better suppress the generation of screen flickering and comb stripes, and the image quality is significantly improved. Patent (Yang Xingliang, Ma Xiao, Guo Yaosong, et al., A deinterlaced image processing method and device, CN107517356B, 2020) adds the judgment of the horizontal boundary threshold, so it can reduce the misjudgment of the edge direction. Since the motion state information of the previous and next frames is referenced, the noise at the boundaries and corners can be reduced, thereby improving the picture clarity and thus improving the image quality. Patent (Zhou Hong, Wei Hu, Luo Xiaowei, et al., a method and device for processing television signal deinterlacing, CN105744201B, 2019) uses an improved interpolation method to achieve deinterlacing, which can effectively improve the accuracy of the pixel value of the interpolated point. Patent (Yang Changshui, Chen Rui, Jia Huizhu, et al., a video deinterlacing method, CN106027943B, 2019) improves the edge line average through precise motion detection, thereby improving the effect of deinterlacing processing. The above mentioned are all video deinterlacing methods. The deinterlacing effect is improved by improving the pixel motion judgment and interpolation method. However, it is impossible to accurately and adaptively judge the motion state for environmental changes such as lighting changes and random target changes, resulting in unsatisfactory deinterlacing effect. Summary of the invention

[0004] In view of the shortcomings of the prior art, the present invention discloses an adaptive de-interlacing algorithm improved based on motion estimation, that is, an improved non-motion interpolation compensation algorithm for adaptive determination of inter-field motion, which obtains the motion state of pixels through inter-field motion blocks, and then uses intra-field interpolation and inter-field interpolation methods to restore field information, and removes layered artifact effects through mean filtering, thereby achieving the purpose of video de-interlacing and improving video quality.

[0005] The present invention is achieved through the following technical solutions:

[0006] The adaptive de-interlacing algorithm improved based on motion estimation includes the following steps:

[0007] Step 1: Use motion blocks to determine the motion state of pixels;

[0008] The four-field motion detection algorithm is used for motion detection. The absolute difference and SAD of the pixel blocks between the fields and the absolute pixel difference diff are calculated respectively to detect the motion degree of the pixel blocks. At the same time, the absolute difference and the motion state of the target are defined.

[0009] Step 2: If the pixel is judged to be moving, the intra-field interpolation method and inter-field interpolation method in the non-motion interpolation compensation method are introduced to restore the current field information. If the pixel is judged to be still, go directly to step 3;

[0010] Step 3: Smooth out the layering artifacts caused by interpolation through mean filtering.

[0011] Preferably, in step 1, a formula for defining a type of absolute difference and SAD is as follows:

[0012]

[0013]

[0014] Preferably, in step 1, the formula for defining the motion state of the target block using absolute pixel difference diff detection is as follows:

[0015] diff1=|F n (x, y-1)-F n (x, y+1) /

[0016] diff2=|F n-1 (x, y)-F n-1 (x,y+2) /

[0017] diff3=|F n (x,y+1)-F n-1 (x, y)|

[0018] diff4=|F n(x,y+1)-F n-1 (x, y+2)|.

[0019] Preferably, the step 1 further comprises: when the scene is affected by environmental changes resulting in motion state detection errors, introducing a Sobel gradient operator combined with SAD to measure the motion state between fields;

[0020] The specific process is as follows: first, use SAD to find the information of similar motion blocks between fields, then use the Sobel gradient operator to calculate the edge gradient map of the block, calculate the Euclidean distance between the gradient maps, introduce the THR threshold to determine the similarity of the target block, and accurately define the motion state of the target block;

[0021] The formula for the Euclidean distance between edge gradient images is as follows:

[0022]

[0023] Preferably, the specific process of determining the motion state of the point to be detected in step 2 is as follows:

[0024] 1) SAD1≥Th or SAD2≥Th;

[0025] 2) diff1≤Th1 and diff2≤Th1 and diff3≤Th2 and diff4≤Th2, Th1<Th2;

[0026] 3) 1<THR and E2<THR;

[0027] 4) Other situations are judged as static state;

[0028] Where: x, y represent the coordinates of the pixel point, n represents the field number, Fn(x, y) represents the brightness value of the pixel point (x, y), E1, E2 represent the Euclidean distance of the pixel blocks in the nth field, n-2 field, n+1 field, and n-1 field respectively, and Th, Th1, Th2, and THR are pre-set thresholds.

[0029] Preferably, in step 2, if the pixel is judged to be in motion, the current pixel value is restored by using the median operation of the interpolation method, as follows:

[0030]

[0031] f cur (x+1,y),f nex (x, y)),

[0032] Where med is the median symbol, f cur (x, y) is the pixel value at (x, y) in the current frame, f pre (x, y) is the pixel value at (x, y) in the previous frame of the current frame, fnex (x, y) is the pixel value at (x, y) in the next frame of the current frame; conversely, the pixel value of the current frame f cur The (x, y) value remains unchanged; ultimately image deinterlacing is achieved.

[0033] Preferably, in step 3, after executing the deinterlacing algorithm, the obtained image has layered artifacts, and a mean filter is used to smooth the moving pixel area, thereby improving the overall image quality while removing the artifacts.

[0034] The present invention has the following beneficial effects:

[0035] The present invention uses motion blocks to determine the motion state of pixels, then introduces intra-field interpolation and inter-field interpolation to restore the current field information, and finally smoothes the layered artifact effect caused by interpolation through mean filtering. Since the algorithm of the present invention can accurately complete the motion estimation of pixels, it solves the problem of inaccurate motion determination caused by environmental changes, and introduces mean filtering to remove layered artifacts, thereby achieving the purpose of deinterlacing and realizing better restoration of interlaced video.

[0036] The present invention realizes video deinterlacing. Compared with the traditional deinterlacing algorithm, the present invention can not only adaptively and accurately judge the motion state of pixels by using a visual algorithm, but also accurately deinterlace and realize image restoration. The advantages of the present invention are: 1) isomorphic to a class of absolute difference sums, defining the motion state of the target, absolute pixel difference detection and a method based on gradient map to resist environmental interference define the motion state; 2) using mean filtering to eliminate layered artifacts after interpolation. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0038] Figure 1 The diagram is a schematic diagram of the non-motion interpolation compensation algorithm for the improved inter-field motion determination of the present invention. DETAILED DESCRIPTION

[0039] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0040] like Figure 1 In view of the interlaced problem existing in the output of the PAL video format, an adaptive deinterlacing algorithm framework with improved motion estimation is established. The embodiment of the present invention provides an adaptive deinterlacing algorithm based on improved motion estimation, which uses motion blocks to determine the motion state of pixels, then introduces intra-field interpolation and inter-field interpolation to restore the current field information, and finally smoothes the layered artifact effect caused by interpolation through mean filtering. Since the algorithm of the present invention can accurately complete the motion estimation of pixels, it solves the problem of inaccurate motion determination caused by environmental changes, and introduces mean filtering to remove layered artifacts, thereby achieving the purpose of deinterlacing and realizing better restoration of interlaced video.

[0041] The embodiment of the present invention specifically includes the following steps:

[0042] Step 1: Use motion blocks to determine the motion state of pixels;

[0043] The four-field motion detection algorithm is used for motion detection. The absolute difference and SAD of the pixel blocks between the fields and the absolute pixel difference diff are calculated respectively to detect the motion degree of the pixel blocks. At the same time, the absolute difference and the motion state of the target are defined.

[0044] The formula for defining a type of absolute difference and SAD is as follows:

[0045]

[0046]

[0047] Due to the motion of the target block spatial periodic pattern, the formula for defining the motion state of the target block using absolute pixel difference diff detection is as follows:

[0048] diff1=|F n (x,y-1)-F n (x,y+1) /

[0049] diff2=|F n-1 (x, y)-F n-1 (x,y+2) /

[0050] diff3=|F n (x,y+1)-F n-1 (x, y)|

[0051] diff4=| F n (x,y+1)-F n-1 (x, y+2)|.

[0052] The step 1 also includes: when the scene is affected by environmental changes (illumination changes, random target changes, etc.), which may cause motion state detection errors, since the gradient detection operator has a certain robustness to changes such as illumination, the Sobel gradient operator is introduced in combination with SAD to measure the motion state between fields;

[0053] The specific process is as follows: first, use SAD to find the information of similar motion blocks between fields, then use the Sobel gradient operator to calculate the edge gradient map of the block, calculate the Euclidean distance between the gradient maps, introduce the THR threshold to determine the similarity of the target block, and accurately define the motion state of the target block;

[0054] The formula for the Euclidean distance between edge gradient images is as follows:

[0055]

[0056] Step 2: If the pixel is judged to be moving, the intra-field interpolation method and inter-field interpolation method in the non-motion interpolation compensation method are introduced to restore the current field information. If the pixel is judged to be still, go directly to step 3;

[0057] The specific process of determining the motion state of the detection point is as follows:

[0058] 1) SAD1≥Th or SAD2≥Th;

[0059] 2) diff1≤Th1 and diff2≤Th1 and diff3≤Th2 and diff4≤Th2, Th1<Th2;

[0060] 3) 1<THR and E2<THR;

[0061] 4) Other situations are judged as static state;

[0062] Where: x, y represent the coordinates of the pixel point, n represents the field number, Fn(x, y) represents the brightness value of the pixel point (x, y), E1, E2 represent the Euclidean distance of the pixel blocks in the nth field, n-2 field, n+1 field, and n-1 field respectively, and Th, Th1, Th2, and THR are pre-set thresholds.

[0063] If the pixel is judged to be in motion, the current pixel value is restored by taking the median operation using the interpolation method, as follows:

[0064]

[0065] f cur (x+1,y),f nex (x, y)),

[0066] Where med is the median symbol, f cur(x, y) is the pixel value at (x, y) in the current frame, f pre (x, y) is the pixel value at (x, y) in the previous frame. nex (x, y) is the pixel value at (x, y) in the next frame of the current frame; conversely, the pixel value of the current frame f cur The (x, y) value remains unchanged; finally, the image is deinterlaced by using the intra-field interpolation method and the inter-field interpolation method in the non-motion interpolation compensation method.

[0067] Step 3: Use mean filtering to smooth the layered artifacts caused by interpolation. After executing the deinterlacing algorithm, the image obtained has layered artifacts. The moving pixel area is smoothed by using mean filtering (filter window 7*7). The layered artifacts are removed by using mean filtering, which improves the overall image quality while removing the artifacts.

[0068] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An adaptive de-interlacing algorithm based on motion estimation improvement, characterized in that: It includes the following steps: Step 1: Use motion blocks to determine the motion state of pixels; Perform motion detection on the four-field motion detection algorithm, calculate the first type of absolute difference sum and SAD of pixel blocks between fields and the first type of absolute pixel difference diff respectively to detect the motion degree of pixel blocks, and at the same time define the first type of absolute difference sum and the motion state of the target; Step 2: If the pixel is judged to be in motion, introduce the intra-field interpolation method and the inter-field interpolation method in the non-motion interpolation compensation method to restore the current field information. If the pixel is judged to be stationary, directly go to Step 3; Step 3: Smooth the layering artifact effect brought by interpolation through mean filtering; In Step 2, if the pixel is judged to be in motion, use the median operation of the interpolation method to restore the current pixel value, specifically as follows: ， ， Where med is the median symbol, For the current frame , ), The frame before the current frame is , ), The next frame for the current frame is , ) pixel value; otherwise, the current frame pixel value The value remains unchanged; ultimately the image is deinterlaced.

2. The improved adaptive de-interlacing algorithm based on motion estimation as claimed in claim 1, characterized in that: In Step 1, the formula for defining the first type of absolute difference sum SAD is as follows: 。 3. The improved adaptive de-interlacing algorithm based on motion estimation as claimed in claim 2, characterized in that: In Step 1, the formula for using the absolute pixel difference diff to detect and define the motion state of the target block is as follows: | | | |。 4. The improved adaptive de-interlacing algorithm based on motion estimation as claimed in claim 3, characterized in that: Step 1 further includes: when the scene is affected by environmental changes resulting in incorrect motion state detection, introduce the combination of the Sobel gradient operator and SAD to measure the motion state between fields; The specific process is as follows: First, use SAD to find the inter-field similar motion block information, then use the Sobel gradient operator to calculate the edge gradient map of the block, calculate the Euclidean distance between the gradient maps, introduce the THR threshold to determine the similarity of the target block, and accurately define the motion state of the target block.

5. The improved adaptive de-interlacing algorithm based on motion estimation as claimed in claim 4, characterized in that: The specific process of determining the motion state of the point to be detected in Step 2 is as follows: 1) SAD1 ≥ Th or SAD2 ≥ Th; 2) diff1 ≤ Th1 and diff2 ≤ Th1 and diff3 ≤ Th2 and diff4 ≤ Th2, Th1 < Th2; 3) 1 < THR and E2 < THR; 4) In other cases, it is judged as the stationary state; Where: x, y represent the coordinates of the pixel point, n represents the field number, Fn(x, y) represents the luminance value of the pixel point (x, y), E1 and E2 respectively represent the Euclidean distances of the pixel blocks in the nth field, the n - 2th field and the n + 1th field, the n - 1th field, and Th, Th1, Th2, and THR are preset thresholds.

6. The improved adaptive de-interlacing algorithm based on motion estimation as claimed in claim 1, characterized in that: In Step 3, after performing the deinterlacing algorithm, the obtained image has layering artifacts. The mean filtering is used to smooth the motion pixel area, improving the overall image quality while removing the artifacts.

Images