A motion detection method based on empirical mode decomposition improved frame difference method

Abstract

The application provides a motion detection method based on an empirical mode decomposition improved frame difference method, and the method comprises the following steps: S1, inputting a previous frame image and a current frame image respectively; S2, performing the following steps on the input images respectively: calculating local extreme points; constructing an envelope surface; calculating local average values; extracting high-frequency information to obtain high-frequency information; and returning mean value images; S3, using the local mean value images and the method of step S2, obtaining medium-frequency information and low-frequency information respectively; S4, performing high, medium and low frequency band frame difference between the current frame and the previous frame; S5, performing dynamic and static judgment on the three frequencies respectively; S6, performing motion fusion; and S7, finally obtaining motion information. The application can effectively solve the problems existing in the current frame difference method, and makes the motion detection more accurate.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, and specifically relates to a motion detection method based on an improved frame difference method of empirical mode decomposition. Background Technology

[0002] In existing technologies, frame differencing is a simple motion detection method based on pixel differences between video frames. Its basic principle is to extract moving regions from an image by using pixel-based temporal differences between two or three adjacent frames. It is widely used in video surveillance, behavior analysis, and motion detection. Its main advantages are its simple computation and fast update speed.

[0003] Among them, the three-frame difference method is an improved method for video motion detection. It reduces the impact of noise and improves detection accuracy by utilizing the differences between three consecutive frames. Compared with the simple two-frame difference method, the three-frame difference method can handle illumination changes and small noise more effectively. Empirical Mode Decomposition (EMD) is an adaptive method for analyzing nonlinear and non-stationary time series signals. Proposed by Huang et al. in 1998, it does not rely on preset basis functions and transforms complex signals into a set of intrinsic mode functions (IMFs) and a residual component through decomposition.

[0004] However, while frame differencing offers simplicity and effectiveness in motion detection, it also has some problems and limitations. Frame differencing is susceptible to noise. Random noise in the image can lead to false positives or false negatives, especially in ISP image processing where noise levels are high. Furthermore, frame differencing relies on a set threshold to determine motion. Noise significantly affects threshold settings, and fixed thresholds are difficult to adapt to dynamically changing environments, resulting in motion holes for large, uniformly colored moving targets.

[0005] Furthermore, the terminology used in this art includes:

[0006] Empirical Mode Decomposition (EMD): Empirical Mode Decomposition (EMD) is a signal processing method based on local characteristics and adaptability. The core idea of ​​EMD is to decompose a complex signal into a superposition of several Intrinsic Mode Functions (IMFs), each IMF representing a local oscillation mode or frequency component in the signal. Frame Difference: Frame difference is a simple and effective video processing technique, commonly used for motion detection and moving target tracking. It detects motion or change based on the differences between consecutive video frames. Summary of the Invention

[0007] In order to solve the above problems, the purpose of this application is:

[0008] 1. Solve the problem of poor noise robustness of the frame difference method.

[0009] 2. Solve the motion hole problem in the frame difference method.

[0010] Specifically, the present invention provides a motion detection method based on an improved frame difference method using empirical mode decomposition, the method comprising:

[0011] S1, input the previous frame image and the current frame image respectively;

[0012] S2 performs the following steps on the input image: calculates local extrema; constructs the envelope surface; calculates the local mean; extracts high-frequency information; and returns the mean image.

[0013] S3, using the local mean image and the method in step S2, obtain the mid-frequency information and low-frequency information respectively;

[0014] S4, the frame difference between the current frame and the previous frame in the high, medium and low frequency bands;

[0015] S5, performs dynamic and static judgments on the three frequencies respectively;

[0016] S6, to perform motion fusion;

[0017] S7 ultimately obtains motion information.

[0018] Step S2 further includes:

[0019] S2.1. Finding local extrema: In an image, local extrema are points where the value of each pixel is greater than or less than the values ​​of all other pixels in its neighborhood. The detection formula is as follows:

[0020] maxima(i,j) if

[0021] minima(i,j) if

[0022] Where I(i,j) represents the extreme point of a pixel; I(m,n) represents the pixel within the neighborhood range;

[0023] S2.2. Constructing the envelope surfaces: Using bilinear interpolation, connect the local maxima and minima to construct the upper envelope surface U(x,y) and the lower envelope surface L(x,y).

[0024] U(x,y)=Interpolate{(x i ,y i ),I(x i ,y i ))}

[0025] L(x,y)=Interpolate{(x j ,y j ),I(x j ,y j ))}

[0026] Let the image be I(x,y), and the local minimum point be (x,y). j ,y j The local maximum point is (x) i ,y i Interpolate represents the interpolation function;

[0027] S2.3. Calculate the local average: The local average m(x,y) is the average of the upper and lower envelope surfaces.

[0028]

[0029] S2.4. Extracting high-frequency information: Subtract the average value m(x,y) from the original image I(x,y) to obtain the high-frequency component h(x,y):

[0030] h(x,y)=I(x,y)-m(x,y).

[0031] Step S3 further includes:

[0032] The local mean image m(x,y) is processed through steps S2.1 and S2.3 to obtain the local mean image of the image. The local mean image is used as the low frequency component hl(x,y). m(x,y) and hl(x,y) are processed through S2.4 to obtain the mid frequency component hm(x,y).

[0033] Step S4, multi-frequency frame difference, further includes:

[0034] Calculate the differences between the high, mid, and low frequency components of the current frame and the previous frame, respectively.

[0035] hdiff(x,y)=abs(h cur (x,y)-h last (x,y))

[0036] hmdiff(x,y)=abs(h cur m(x,y)-h last m(x,y))

[0037] hldiff(x,y)=abs(h cur l(x,y)-h last l(x,y))

[0038] Where hcur represents the high-frequency information of the current frame, hlast represents the high-frequency information of the previous frame; hcurm represents the mid-frequency information of the current frame, hlastm represents the mid-frequency information of the previous frame; hcurl represents the low-frequency information of the current frame, hlastl represents the low-frequency information of the previous frame; hdiff, hmdiff and hldiff represent the motion frame differences of the high, medium and low frequencies, respectively.

[0039] The motion / static determination step S5 further includes:

[0040] High, medium, and low frequencies are used to independently determine motion and stillness, with different thresholds assigned to different frequency bands to determine motion.

[0041] If hdiff(x,y) > thr1 and hmove(x,y) = 1, then hmove(x,y) = 0.

[0042] If hmdiff(x,y) > thr 2 and hmmove(x,y) = 1, then hmmove(x,y) = 0.

[0043] If hldiff(x,y) > thr3 and hlmove(x,y) = 1, then hlmove(x,y) = 0. Here, hmove represents high-frequency motion information; hmmove represents mid-frequency motion information; and hlmove represents low-frequency motion information.

[0044] The motion fusion step S6 further includes:

[0045] The three types of motion information are fused with different weights, and high-frequency information is given the lowest weight because it is easily affected by noise.

[0046]

[0047] Where w1, w2, and w3 represent adjustable motion information weights for different frequency bands.

[0048] The method uses empirical mode decomposition to obtain information in different frequency domains (high, medium, and low) of the current frame and the previous frame in the video. It extracts detail information from the high frequency, edge information from the medium frequency, and light and shadow information from the low frequency. By calculating the frame difference between the three frequencies, the motion of details, edge motion, and light and shadow in the image can be obtained.

[0049] Therefore, the advantages of this application are as follows: This invention uses Empirical Mode Decomposition (EMD) to obtain information from different frequency domains (high, medium, and low) in the current and previous frames of a video image. It extracts detail information from high frequencies, edge information from mid-frequency frequencies, and light and shadow information from low frequencies. By calculating the frame difference between these three frequencies, detail motion, edge motion, and light and shadow motion in the image can be obtained. Simultaneously, three different thresholds are set to distinguish noise, improving the noise robustness of the frame difference method and addressing the motion hole problem to some extent. In summary, this invention effectively solves the problems existing in current frame difference methods, making motion detection more accurate. Attached Figure Description

[0050] The accompanying drawings, which are provided to further illustrate the invention and form part of this application, are not intended to limit the scope of the invention.

[0051] Figure 1 This is a schematic diagram of the method flow of the present invention. Detailed Implementation

[0052] To better understand the technical content and advantages of the present invention, the present invention will now be described in further detail with reference to the accompanying drawings.

[0053] This invention proposes a motion detection method that improves the frame difference method by utilizing empirical mode decomposition; the process is as follows: Figure 1 As shown:

[0054] S1, input the previous frame image and the current frame image respectively;

[0055] S2, for the input image, respectively performs the following: calculate local extrema; construct the envelope surface; calculate the local mean; extract high-frequency information to obtain high-frequency information; and return the mean image; further including: S2.1. Finding the local extrema.

[0056] In an image, a local extremum is a pixel whose value is either greater than or less than the values ​​of all other pixels in its neighborhood. The detection formula is as follows:

[0057]

[0058] Where I(i,j) represents the extreme point of a pixel; I(m,n) represents the pixel within the neighborhood.

[0059] S2.2. Constructing the envelope surface

[0060] The local maxima and minima are connected using bilinear interpolation to construct the upper envelope U(x,y) and lower envelope L(x,y).

[0061] U(x,y)=Interpolate{(x i ,y i ),I(x i ,y i ))}

[0062] L(x,y)=Interpolate{(x j ,y j ),I(x j ,y j ))}

[0063] Let the image be I(x,y), and the local minimum point be (x,y). j ,y j The local maximum point is (x) i ,y i Interpolate represents the interpolation function.

[0064] S2.3. Calculate the local average: The local average m(x,y) is the average of the upper and lower envelope surfaces.

[0065]

[0066] S2.4. Extracting high-frequency information: Subtract the average value m(x,y) from the original image I(x,y) to obtain the high-frequency component h(x,y).

[0067] h(x,y)=I(x,y)-m(x,y).

[0068] S3, using the local mean image and the method of step S2, obtains mid-frequency information and low-frequency information respectively; further including:

[0069] The local mean image m(x,y) is processed through steps S2.1 and S2.3 to obtain the local mean image of the image. The local mean image is used as the low frequency component hl(x,y). m(x,y) and hl(x,y) are processed through S2.4 to obtain the mid frequency component hm(x,y).

[0070] S4, the frame difference between the current frame and the previous frame in the high, medium, and low frequency bands; further including: calculating the difference images of the high, medium, and low frequency components of the current frame and the previous frame respectively.

[0071] hdiff(x,y)=abs(h cur (x,y)-hlast (x,y))

[0072] hmdiff(x,y)=abs(h cur m(x,y)-h last m(x,y))

[0073] hldiff(x,y)=abs(h cur l(x,y)-h last l(x,y))

[0074] Where hcur represents the high-frequency information of the current frame, hlast represents the high-frequency information of the previous frame; hcurm represents the mid-frequency information of the current frame, hlastm represents the mid-frequency information of the previous frame; hcurl represents the low-frequency information of the current frame, hlastl represents the low-frequency information of the previous frame; hdiff, hmdiff and hldiff represent the motion frame differences of the high, medium and low frequencies, respectively.

[0075] S5 performs dynamic and static judgments on the three frequencies respectively; further including:

[0076] High, medium, and low frequencies are used to independently determine motion and stillness, with different thresholds assigned to different frequency bands to determine motion.

[0077] If hdiff(x,y) > thr1, hmove(x,y) = 1; otherwise, hmove(x,y) = 0.

[0078] If hmdiff(x,y) > thr2, then hmmove(x,y) = 1; otherwise, hmmove(x,y) = 0.

[0079] If hldiff(x,y) > thr3, hlmove(x,y) = 1; otherwise, hlmove(x,y) = 0. Here, hmove represents high-frequency motion information; hmmove represents mid-frequency motion information; and hlmove represents low-frequency motion information.

[0080] S6, for motion fusion; further includes:

[0081] The three types of motion information are fused with different weights, and high-frequency information is given the lowest weight because it is easily affected by noise.

[0082]

[0083] Where w1, w2, and w3 represent adjustable motion information weights for different frequency bands.

[0084] S7 ultimately obtains motion information.

[0085] The method uses empirical mode decomposition to obtain information in different frequency domains (high, medium, and low) of the current frame and the previous frame in the video. It extracts detail information from the high frequency, edge information from the medium frequency, and light and shadow information from the low frequency. By calculating the frame difference between the three frequencies, the motion of details, edge motion, and light and shadow in the image can be obtained.

[0086] In summary, this application obtains frame difference information of different frequency bands of the image through empirical mode decomposition, which makes the obtained information more comprehensive and richer, and the motion detection more accurate; by setting different thresholds for motion and stillness judgment, and setting different weights for fusion, the noise robustness of motion detection is better.

[0087] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, various modifications and variations can be made to the embodiments of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A motion detection method based on an improved frame difference method using empirical mode decomposition, characterized in that, The method includes: S1, input the previous frame image and the current frame image respectively; S2 performs the following steps on the input image: calculates local extrema; constructs the envelope surface; calculates the local mean; extracts high-frequency information; and returns the mean image. S3, using the local mean image and the method in step S2, obtain the mid-frequency information and low-frequency information respectively; S4, the frame difference between the current frame and the previous frame in the high, medium and low frequency bands; S5, performs dynamic and static judgments on the three frequencies respectively; S6, to perform motion fusion; S7 ultimately obtains motion information.

2. The motion detection method based on empirical mode decomposition and improved frame difference method according to claim 1, characterized in that, Step S2 further includes: S2.

1. Finding local extrema: In an image, local extrema are points where the value of each pixel is greater than or less than the values ​​of all other pixels in its neighborhood. The detection formula is as follows: maxima(i,j) if minima(i,j) if Where I(i,j) represents the extreme point of a pixel; I(m,n) represents the pixel within the neighborhood range; S2.

2. Constructing the envelope surfaces: Using bilinear interpolation, connect the local maxima and minima to construct the upper envelope surface U(x,y) and the lower envelope surface L(x,y). U(x,y)=Interpolate{(x i ,y i ),I(x i ,y i ))} L(x,y)=Interpolate{(x j ,y j ),I(x j ,y j ))} Let the image be I(x,y), and the local minimum point be (x,y). j ,y j The local maximum point is (x) i ,y i Interpolate represents the interpolation function; S2.

3. Calculate the local average: The local average m(x,y) is the average of the upper and lower envelope surfaces. S2.

4. Extracting high-frequency information: Subtract the average value m(x,y) from the original image I(x,y) to obtain the high-frequency component h(x,y): h(x,y)=I(x,y)-m(x,y).

3. The motion detection method based on empirical mode decomposition and improved frame difference method according to claim 2, characterized in that, Step S3 further includes: The local mean image m(x,y) is processed through steps S2.1 and S2.3 to obtain the local mean image of the image. The local mean image is used as the low frequency component hl(x,y). m(x,y) and hl(x,y) are processed through S2.4 to obtain the mid frequency component hm(x,y).

4. The motion detection method based on empirical mode decomposition and improved frame difference method according to claim 1, characterized in that, Step S4, multi-frequency frame difference, further includes: Calculate the differences between the high, mid, and low frequency components of the current frame and the previous frame, respectively. hdiff(x,y)=abs(h cur (x,y)-h last (x,y)) hmdiff(x,y)=abs(h cur m(x,y)-h last m(x,y)) hldiff(x,y)=abs(h cur l(x,y)-h last l(x,y)) Where hcur represents the high-frequency information of the current frame, hlast represents the high-frequency information of the previous frame; hcurm represents the mid-frequency information of the current frame, hlastm represents the mid-frequency information of the previous frame; hcurl represents the low-frequency information of the current frame, hlastl represents the low-frequency information of the previous frame; hdiff, hmdiff and hldiff represent the motion frame differences of the high, medium and low frequencies, respectively.

5. The motion detection method based on empirical mode decomposition and improved frame difference method according to claim 1, characterized in that, The motion / static determination step S5 further includes: High, medium, and low frequencies are used to independently determine motion and stillness, with different thresholds assigned to different frequency bands to determine motion. If hdiff(x,y) > thr1 and hmove(x,y) = 1, then hmove(x,y) = 0. If hmdiff(x,y) > thr 2 and hmmove(x,y) = 1, then hmmove(x,y) = 0. If hldiff(x,y) > thr3 and hlmove(x,y) = 1, then hlmove(x,y) = 0. Among them, hmove represents high-frequency motion information; hmmove represents mid-frequency motion information; and hlmove represents low-frequency motion information.

6. The motion detection method based on empirical mode decomposition and improved frame difference method according to claim 1, characterized in that, The motion fusion step S6 further includes: The three types of motion information are fused with different weights, and high-frequency information is given the lowest weight because it is easily affected by noise. Where w1, w2, and w3 represent adjustable motion information weights for different frequency bands.

7. The motion detection method based on empirical mode decomposition and improved frame difference method according to claim 1, characterized in that, The method uses empirical mode decomposition to obtain information in different frequency domains (high, medium, and low) of the current frame and the previous frame in the video. It extracts detail information from the high frequency, edge information from the medium frequency, and light and shadow information from the low frequency. By calculating the frame difference between the three frequencies, the motion of details, edge motion, and light and shadow in the image can be obtained.

Images