Mobile detection method and system based on continuous frame difference method in combination with CNN (Convolutional Neural Network)
By combining continuous frame difference method and lightweight CNN in the motion detection system, the potential motion areas in the video frame sequence are extracted and deep feature extraction is performed, and the problems of insufficient real-time and computational efficiency in the prior art are solved, and efficient motion detection and abnormal behavior recognition are achieved.
Patent Information
- Application Number
- CN202411919386.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-25
- Publication Date
- 2025-04-18
AI Technical Summary
The existing CNN-based motion detection methods have shortcomings in real-time and computing efficiency, and are difficult to effectively apply in mobile devices and embedded systems, and are difficult to effectively extract motion information in video sequences and identify abnormal behaviors.
The motion detection method based on the continuous frame difference method combined with the lightweight convolutional neural network (CNN) is adopted. The video frame sequence is collected in real time through the camera, and the potential motion area is extracted using the continuous frame difference method, and input it into the lightweight MobileNetV3_small convolutional neural network for deep feature extraction and recognition, to determine whether the motion is an abnormal behavior, and trigger an alarm.
The rapidity of the frame difference method reduces the computational amount of CNN processing, improves the system's real-time processing capabilities, and uses a lightweight CNN structure to ensure the efficient operation of the system on embedded devices, significantly improves processing speed and maintains high detection accuracy and robustness.
Smart Images
Figure CN119850978A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of digital image processing, and in particular relates to a motion detection method and system based on a continuous frame difference method combined with CNN. Background Art
[0002] In the field of digital image processing, motion detection technology is widely used in security monitoring, traffic flow statistics, intelligent transportation systems and other aspects; traditional motion detection methods mainly rely on background subtraction, optical flow method, etc., but these methods often have poor effects when dealing with complex scenes and are easily affected by environmental changes. In recent years, with the development of deep learning technology, motion detection methods based on convolutional neural networks (CNNs) have gradually become a research hotspot. CNNs can automatically extract features from images and have better adaptability and accuracy for motion detection in complex scenes. However, existing CNN-based motion detection methods still have shortcomings in real-time and computational efficiency. For example, some methods require a lot of computing resources, which limits their application in mobile devices and embedded systems. In addition, how to effectively extract motion information from video sequences and accurately identify abnormal behaviors is also a problem that current technology needs to solve. Summary of the invention
[0003] In order to solve the above problems existing in the prior art, the present invention provides a motion detection method and system based on continuous frame difference method combined with CNN;
[0004] The purpose of the present invention can be achieved through the following technical solutions:
[0005] A motion detection method based on continuous frame difference method combined with CNN, comprising:
[0006] The camera collects video frame sequences in real time, performs differential operation on continuous video frames of the video frame sequence to extract potential motion areas; the extracted potential motion areas are input into a convolutional neural network for deep feature extraction and recognition; the extracted deep features are analyzed to determine whether the motion is an abnormal behavior. If abnormal behavior is detected, the system automatically triggers an alarm and records relevant data.
[0007] Specifically, the image data of the video frame sequence used for differential calculation is calculated using basic image data of the green channel.
[0008] Specifically, the difference operation method is:
[0009] Two frame difference images are obtained by performing continuous frame difference comparison on the current frame image and the previous two frame images in the video frame sequence, and the frame difference images are again continuously compared on the frame difference images to obtain a final continuous frame difference image.
[0010] Specifically, in the process of continuous frame difference comparison, the range of the pixel difference value of the frame difference image to be compared is determined by setting the upper and lower limits of the filtering threshold, and the calculation formula of the frame difference value is:
[0011]
[0012] Where I(k) represents the image pixel value after frame difference is taken, TH h is the upper limit of the filtering threshold, TH l is the lower bound of the filtering threshold, G(i,k) represents the pixel of the i-th frame image, G(i-1,k) represents the pixel of the i-1-th frame image, i is the image frame number, and k represents the pixel of the image.
[0013] Specifically, the method for extracting the potential motion area is:
[0014] The image is divided into area blocks of different sizes through preset grid area blocks. During the extraction process, intra-block operations are performed on the area blocks of the image one by one. If the first non-zero pixel appears in the area block during the calculation process of the intra-block operation, the intra-block operation is jumped out and the starting coordinates corresponding to the area block currently calculated are recorded; the continuity relationship of the image blocks of the video frames in the video frame sequence is judged according to the starting coordinates and the image block size of the area block, and the potential motion area is finally determined by scanning the video frame sequence block by block to obtain the continuous area.
[0015] Specifically, the convolutional neural network adopts a lightweight MobileNetV3_small convolutional neural network structure for application in mobile devices and embedded systems. The standard convolution is decomposed into deep convolution and point-by-point convolution through deep separable convolution to reduce computational overhead, and the feature dimension is reduced through a linear bottleneck structure. The linear bottleneck structure expands the dimension of the input image features through an expansion factor, and then processes the features through the deep separable convolution layer. The activation function used in the processing process adopts a nonlinear hard activation function, and finally the dimension is compressed through a 1×1 convolution layer and a linear activation function.
[0016] Specifically, the deep feature extraction and recognition also include a post-processing process, and the post-processing process uses morphological operations to eliminate noise and fill holes.
[0017] Specifically, the upper and lower limits of the filtering threshold are dynamically adjusted according to factors such as changes in ambient light, the size and speed of the target object, etc. through an adaptive threshold adjustment mechanism.
[0018] Specifically, the abnormal behavior recognition method adopts multi-scale feature fusion technology to capture abnormal behaviors in potential motion areas by integrating feature information of abnormal behavior samples at different scales.
[0019] A motion detection system based on continuous frame difference method combined with CNN, including: image acquisition module, feature extraction module, anomaly detection module;
[0020] The image acquisition module acquires a video frame sequence in real time through a camera, and performs differential operation on continuous video frames of the video frame sequence to extract potential motion areas;
[0021] The feature extraction module inputs the extracted potential motion area into a convolutional neural network for deep feature extraction and recognition;
[0022] The anomaly detection module analyzes the extracted depth features to determine whether the movement is an abnormal behavior. If it is detected as an abnormal behavior, the system automatically triggers an alarm and records relevant data.
[0023] The beneficial effects of the present invention are:
[0024] The rapidity of the frame difference method allows a large number of redundant areas to be excluded, reducing the amount of computation required for CNN processing, thereby improving the real-time processing capability of the system. The lightweight convolutional neural network structure is used to reduce model parameters and computing requirements, ensuring efficient operation of the system on embedded devices. It can quickly determine the type of objects moving in the area, greatly improving the object type information of the motion detection part, and reducing the situation where the movement of objects such as flying insects or leaves causes false triggering of motion detection within the monitoring range. Compared with directly using CNN for full-frame analysis, the frame difference method reduces the scope of CNN processing and significantly improves the processing speed. In various complex environments, such as light changes, dynamic backgrounds, noise, etc., it can still maintain high detection accuracy and robustness. BRIEF DESCRIPTION OF THE DRAWINGS
[0025] In order to facilitate understanding by those skilled in the art, the present invention is further described below with reference to the accompanying drawings.
[0026] Figure 1 It is a flow chart of a motion detection method based on continuous frame difference method combined with CNN of the present invention;
[0027] Figure 2 This is a flow chart of the continuous frame difference effect in the present invention;
[0028] Figure 3 This is a flow chart of continuous frame difference combined with CNN classification in the present invention. DETAILED DESCRIPTION
[0029] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the specific implementation methods, structures, features and effects of the present invention are described in detail below in conjunction with the accompanying drawings and preferred embodiments.
[0030] See also Figure 1-3 , a motion detection method based on continuous frame difference method combined with CNN, including:
[0031] The camera collects video frame sequences in real time, performs differential operation on continuous video frames of the video frame sequence to extract potential motion areas; the extracted potential motion areas are input into a convolutional neural network for deep feature extraction and recognition; the extracted deep features are analyzed to determine whether the motion is an abnormal behavior. If abnormal behavior is detected, the system automatically triggers an alarm and records relevant data.
[0032] Specifically, the image data of the video frame sequence used for differential calculation is calculated using basic image data of the green channel.
[0033] In this embodiment, according to the imaging principle of the camera sensor, the G channel (green channel) data of the image is selected as the basic image data for calculating the frame difference operation. The method for calculating the continuous frame difference is to compare the current frame i with the previous frame (i-1) and the previous frame (i-2) to obtain two frame difference images (j and j-1), and then perform frame difference operation on the two frame difference images to obtain the final image frame difference image I. Through the defined grid area block (the image is divided into M*N area blocks of different sizes), each pixel in the image I is performed with an AND operation. When the first non-zero pixel appears in the block, the operation in the image block is jumped out, and the starting coordinates corresponding to the current block are recorded. The image block size must be smaller than the minimum detection target size. The image block determines whether the previous and next image blocks have a continuous relationship based on the coordinates and width and height. Continuous operations are performed to obtain a continuous area block. According to the block-by-block scanning, a continuous block of a larger area is obtained. When the obtained continuous block area is smaller than the minimum detection target, the image cropping function of the current image block will be abandoned. Based on the advantage of CNN in image feature extraction, the original image of the moving area block is further extracted to identify the detailed features of the moving target, such as shape, texture, etc., and classify them. The multi-scale features of the image are extracted layer by layer through the convolution layer and pooling layer of CNN, which can effectively eliminate noise interference in the environment, such as dynamic background (swaying leaves, etc.) or lighting changes.
[0034] Specifically, the difference operation method is:
[0035] Two frame difference images are obtained by performing continuous frame difference comparison on the current frame image and the previous two frame images in the video frame sequence, and the frame difference images are again continuously compared on the frame difference images to obtain a final continuous frame difference image.
[0036] Specifically, in the process of continuous frame difference comparison, the range of the pixel difference value of the frame difference image to be compared is determined by setting the upper and lower limits of the filtering threshold, and the calculation formula of the frame difference value is:
[0037]
[0038] Where I(k) represents the image pixel value after frame difference is taken, TH h is the upper limit of the filtering threshold, TH l is the lower bound of the filtering threshold, G(i,k) represents the pixel of the i-th frame image, G(i-1,k) represents the pixel of the i-1-th frame image, i is the image frame number, and k represents the pixel of the image.
[0039] Specifically, the method for extracting the potential motion area is:
[0040] The image is divided into area blocks of different sizes through preset grid area blocks. During the extraction process, intra-block operations are performed on the area blocks of the image one by one. If the first non-zero pixel appears in the area block during the calculation process of the intra-block operation, the intra-block operation is jumped out and the starting coordinates corresponding to the area block currently calculated are recorded; the continuity relationship of the image blocks of the video frames in the video frame sequence is judged according to the starting coordinates and the image block size of the area block, and the potential motion area is finally determined by scanning the video frame sequence block by block to obtain the continuous area.
[0041] Specifically, the convolutional neural network adopts a lightweight MobileNetV3_small convolutional neural network structure for application in mobile devices and embedded systems. The standard convolution is decomposed into deep convolution and point-by-point convolution through deep separable convolution to reduce computational overhead, and the feature dimension is reduced through a linear bottleneck structure. The linear bottleneck structure expands the dimension of the input image features through an expansion factor, and then processes the features through the deep separable convolution layer. The activation function used in the processing process adopts a nonlinear hard activation function, and finally the dimension is compressed through a 1×1 convolution layer and a linear activation function.
[0042] Specifically, the deep feature extraction and recognition also include a post-processing process, and the post-processing process uses morphological operations to eliminate noise and fill holes.
[0043] Specifically, the upper and lower limits of the filtering threshold are dynamically adjusted according to factors such as changes in ambient light, the size and speed of the target object, etc. through an adaptive threshold adjustment mechanism.
[0044] Specifically, the abnormal behavior recognition method adopts multi-scale feature fusion technology to capture abnormal behaviors in potential motion areas by integrating feature information of abnormal behavior samples at different scales.
[0045] A motion detection system based on continuous frame difference method combined with CNN, including: image acquisition module, feature extraction module, anomaly detection module;
[0046] The image acquisition module acquires a video frame sequence in real time through a camera, and performs differential operation on continuous video frames of the video frame sequence to extract potential motion areas;
[0047] The feature extraction module inputs the extracted potential motion area into a convolutional neural network for deep feature extraction and recognition;
[0048] The anomaly detection module analyzes the extracted depth features to determine whether the movement is an abnormal behavior. If it is detected as an abnormal behavior, the system automatically triggers an alarm and records relevant data.
[0049] The computer storage medium of the embodiment of the present invention may adopt any combination of one or more computer-readable media. The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device.
[0050] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.
[0051] The program code included on the computer readable medium can be transmitted with any appropriate medium, including but not limited to wireless, electric wire, optical cable, RF, etc., or any suitable combination of the above. The computer program code for performing the operation of the present invention can be written in one or more programming languages or their combinations, and the programming language includes object-oriented programming languages-such as Java, Smalltalk, C++, and also includes conventional procedural programming languages-such as "C" language or similar programming languages. The program code can be executed completely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on the remote computer, or completely on the remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (for example, using an Internet service provider to connect through the Internet).
[0052] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Although the present invention has been disclosed as a preferred embodiment as above, it is not used to limit the present invention. Any technical personnel in this field can make some changes or modify the technical contents disclosed above into equivalent embodiments without departing from the scope of the technical solution of the present invention. However, any simple modification, equivalent change and modification made to the above embodiments according to the technical essence of the present invention without departing from the content of the technical solution of the present invention still fall within the scope of the technical solution of the present invention.
Claims
1. A motion detection method based on continuous frame difference method combined with CNN, characterized in that: include: The camera collects a video frame sequence in real time, and performs differential operation on the continuous video frames of the video frame sequence to extract the potential motion area; The extracted potential motion area is input into a convolutional neural network for deep feature extraction and recognition; the extracted deep features are analyzed to determine whether the motion is an abnormal movement behavior. If it is detected as an abnormal movement behavior, the system automatically triggers an alarm and records relevant data.
2. The method according to claim 1, characterized in that The image data used for differential calculation in the video frame sequence is calculated using basic image data of the green channel.
3. The method according to claim 1, characterized in that The method of the differential operation is: Two frame difference images are obtained by performing continuous frame difference comparison on the current frame image and the previous two frame images in the video frame sequence, and the frame difference images are again continuously compared on the frame difference images to obtain a final continuous frame difference image.
4. The method according to claim 3, characterized in that In the process of continuous frame difference comparison, the range of the pixel difference value of the frame difference image to be compared is determined by setting the upper and lower limits of the filtering threshold. The calculation formula of the frame difference value is: Where I(k) represents the image pixel value after frame difference is taken, TH h is the upper limit of the filtering threshold, TH l is the lower bound of the filtering threshold, G(i,k) represents the pixel of the i-th frame image, G(i-1,k) represents the pixel of the i-1-th frame image, i is the image frame number, and k represents the pixel of the image.
5. The method according to claim 3, characterized in that: The method for extracting the potential motion area is: The image is divided into area blocks of different sizes through preset grid area blocks. During the extraction process, intra-block operations are performed on the area blocks of the image one by one. If the first non-zero pixel appears in the area block during the calculation process of the intra-block operation, the intra-block operation is jumped out and the starting coordinates corresponding to the area block currently calculated are recorded; the continuity relationship of the image blocks of the video frames in the video frame sequence is judged according to the starting coordinates and the image block size of the area block, and the potential motion area is finally determined by scanning the video frame sequence block by block to obtain the continuous area.
6. The method according to claim 1, characterized in that The convolutional neural network adopts a lightweight MobileNetV3_small convolutional neural network structure for application in mobile devices and embedded systems. The standard convolution is decomposed into deep convolution and point-by-point convolution through deep separable convolution to reduce computational overhead, and the feature dimension is reduced through a linear bottleneck structure. The linear bottleneck structure expands the dimension of the input image features through an expansion factor, and then processes the features through the deep separable convolution layer. The activation function used in the processing process adopts a nonlinear hard activation function, and finally the dimension is compressed through a 1×1 convolution layer and a linear activation function.
7. The method according to claim 1, characterized in that The deep feature extraction and recognition also include a post-processing process, in which morphological operations are used to eliminate noise and fill holes.
8. The method according to claim 4, characterized in that The upper and lower limits of the filtering threshold are dynamically adjusted according to factors such as changes in ambient light, the size and speed of the target object, etc. through an adaptive threshold adjustment mechanism.
9. The method according to claim 1, characterized in that: The abnormal behavior recognition method adopts multi-scale feature fusion technology to capture abnormal behaviors in potential motion areas by integrating feature information of abnormal behavior samples at different scales.
10. A motion detection system based on continuous frame difference method combined with CNN, used to execute the method according to claims 1-9, characterized in that: include: Image acquisition module, feature extraction module, anomaly detection module; The image acquisition module acquires a video frame sequence in real time through a camera, and performs a differential operation on continuous video frames of the video frame sequence to extract a potential motion area; The feature extraction module inputs the extracted potential motion area into a convolutional neural network for deep feature extraction and recognition; The anomaly detection module analyzes the extracted depth features to determine whether the movement is an abnormal behavior. If it is detected as an abnormal behavior, the system automatically triggers an alarm and records relevant data.