A moving object detection method based on deep learning

Abstract

The present invention relates to a method for detecting moving objects based on deep learning, comprising: step S1, using a convolutional neural network to extract features of two original images to be compared to obtain a feature map; step S2, passing the feature map through a difference network to obtain a difference feature map, then extracting the area exceeding a threshold as a difference area, and after extraction, passing the difference area through two or more fully connected networks to obtain a feature difference value; step S3, passing the difference area through one or more convolutional neural network layers to obtain a deep feature map, and after obtaining, passing the deep feature values ​​through one or more fully connected layers; step S4, performing a difference comparison to obtain a feature similarity value of the object in the difference area; and step S5, combining the feature difference value and the feature similarity value of the object in the difference area to comprehensively determine whether the object is moving. Thus, by using a smaller network than deep learning target detection to extract features and using the difference network to determine whether there is a moving object.

Description

Technical Field

[0001] The present invention relates to the field of moving object detection, and in particular to a moving object detection method based on deep learning. Background Art

[0002] Traditional moving object detection uses background subtraction, which statistically models each pixel in an image. The average RGB value of each pixel over a period of time is obtained by statistically analyzing multiple consecutive frames. This average RGB value of all pixels in an image constitutes the background of the entire image. This is called a background subtraction method. If the value of a pixel at the next moment exceeds or falls below the average value plus or minus a threshold, the pixel is considered foreground. When the number of foreground pixels exceeds a certain threshold, a moving object is detected. A fatal weakness of background subtraction is that it is almost impossible to detect moving objects when the moving object and the background have similar colors. In this case, the difference between the RGB value of the moving object and the extracted RGB value of the background is within the threshold, resulting in the foreground being classified as background or the number of pixels classified as foreground being limited, making it impossible to capture the moving object. Moving object detection methods based on target detection use convolutional neural networks such as YOLO and SSD to detect bounding boxes for specific object categories within the image. If the bounding boxes corresponding to these object categories do not change in the next frame, the object is considered to have no movement. If the bounding boxes of the corresponding object categories change, the object is considered to have moved. Moving object detection methods require a relatively large convolutional neural network to accurately identify the bounding box of an object. This requires strong CPU or GPU computing power and cannot be deployed on devices with weaker computing power, such as ARM processors. Furthermore, these methods cannot detect objects for which they have not been trained, or for which only partial features exist that cannot accurately determine the category. This can also lead to the loss of some moving objects.

[0003] Chinese patent CN201380072134.8 discloses a method and system for detecting a moving object, comprising: identifying a plurality of feature points based on a plurality of video frames; selecting from the plurality of feature points based on the correlation between the plurality of feature points to form a first feature point group and a second feature point group; checking whether the first feature point group and the second feature point group have common feature points; if so, identifying a first reference point from the first feature point group and a second reference point from the second feature point group; classifying the common feature points into one of the first feature point group and the second feature point group to obtain an updated first feature point group and an updated second feature point group, wherein the reference point of the one of the first feature point group and the second feature point group has the greatest correlation with the common feature point; and identifying two segments in at least one video frame as detected moving objects based on the updated first feature point group and the updated second feature point group, respectively, wherein the correlation between the two feature points includes a distance component and a motion difference component, wherein the distance component is related to the distance between the two feature points, and the motion difference component is related to the difference between the corresponding motions of the two feature points.

[0004] Currently, there are some moving object detection methods based on deep learning, but they generally cannot extract features by using a smaller network than deep learning target detection and determine whether there is a moving object through a differential network. Summary of the Invention

[0005] To this end, the present invention provides a moving object detection method based on deep learning, which can effectively solve the technical problem in the prior art that it is not possible to extract features by using a network smaller than the deep learning target detection and determine whether there is a moving object through a differential network.

[0006] To achieve the above objectives, the present invention provides a method for detecting moving objects based on deep learning, comprising:

[0007] Step S1: using a convolutional neural network to extract features of the two original images to be compared to obtain an original feature map and a comparison feature map;

[0008] Step S2, the original feature map and the comparison feature map are passed through a differential network to obtain a feature difference map; after obtaining, the regions of the original feature map and the comparison feature map in S1 where the corresponding feature difference maps exceed a threshold are extracted as original feature difference regions and comparison feature difference regions, respectively. After extraction, the feature difference maps are passed through two or more fully connected networks to obtain feature difference values;

[0009] Step S3, passing the original feature difference region and the compared feature difference region through one or more convolutional neural network layers to obtain an original deep feature map and a compared deep feature map, respectively. After obtaining the original deep feature map and the compared deep feature map, passing the original deep feature map and the compared deep feature map through one or more fully connected layers to obtain an original deep feature value and a compared deep feature value, respectively.

[0010] Step S4, comparing the original deep feature value and the compared deep feature value to obtain similar feature values ​​of objects in the difference area;

[0011] In step S5, a comprehensive judgment is made as to whether the object has moved by combining the feature difference value obtained in step S2 and the feature similarity value of the object in the difference area obtained in step S4.

[0012] Furthermore, the specific calculation for comprehensively judging whether the object is moving in step S5 is as follows:

[0013] A = diff × (1.0-simi);

[0014] Throughout, A represents the final judgment value, diff represents the feature difference value, and simi represents the object similarity value in the difference area.

[0015] Furthermore, after the final determination value A is obtained, it is compared with a threshold value. If the final determination value A ≥ the threshold value, it indicates that the object has moved; if the final determination value A < the threshold value, it indicates that the object has not moved.

[0016] Furthermore, the feature difference value and the feature similarity value of the object in the difference area in step S5 need to be output through a Sigmoid activation function before comprehensive judgment.

[0017] Furthermore, the feature difference value in step S2 is the degree of difference between the original feature difference region and the compared feature difference region.

[0018] Furthermore, the object similarity value of the different regions in step S4 is determined by whether the different regions are the same object at different positions.

[0019] Furthermore, the convolutional neural network used in step S1 is extracted by selecting the existing mobile open source network MobileNet series, and the number of network layers is appropriately adjusted to meet the computing power and memory requirements of the target ARM device to be deployed.

[0020] Furthermore, after obtaining the original feature map and the comparison feature map in step S1, the original feature map and the comparison feature map can be passed through a differential network to obtain a differential feature map. After obtaining, the differential feature map is passed through one or more layers of a fully connected network to finally obtain a differential value.

[0021] Furthermore, after the difference value is obtained, the difference value is directly passed through a Sigmoid activation function to determine whether there is a moving object in the picture.

[0022] Compared to existing technologies, the present invention offers the following advantages: it uses a smaller network than deep learning target detection to extract features of objects in the image, and then uses a differential network to determine whether there is a moving object. This approach requires less CPU / GPU computing power, making it suitable for deployment on edge devices like ARM, and offers greater applicability. Furthermore, it eliminates the need to identify specific object categories; it simply determines whether the texture features extracted through deep learning have changed to determine whether an object is moving.

[0023] Furthermore, the present invention determines whether the object has moved by comparing the final determination value A with a threshold value, wherein the final determination value A is accurately calculated by a preset formula, thereby being able to accurately determine whether the object has moved.

[0024] Furthermore, the feature difference value and the feature similarity value of the object in the difference area of ​​the present invention need to be output through a Sigmoid activation function before comprehensive judgment, which can improve the accuracy of comprehensive judgment of whether the object is moving.

[0025] Furthermore, the present invention directly determines whether an object is moving by differentiating two feature maps and then performing full connection; the feature extraction of the difference area is not refined, and the judgment of the moving object can still be achieved, but there may be false alarms. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 Schematic diagram of the process of a moving object detection method based on deep learning according to an embodiment of the present invention;

[0027] Figure 2 Schematic diagram A of a moving object detection method based on deep learning according to an embodiment of the present invention;

[0028] Figure 3 FIG4 is a schematic diagram B of a moving object detection method based on deep learning according to an embodiment of the present invention. DETAILED DESCRIPTION

[0029] In order to make the objects and advantages of the present invention more clearly understood, the present invention is further described below in conjunction with embodiments; it should be understood that the specific embodiments described herein are merely used to explain the present invention and are not intended to limit the present invention.

[0030] The preferred embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood by those skilled in the art that these embodiments are only used to explain the technical principles of the present invention and are not intended to limit the scope of protection of the present invention.

[0031] It should be noted that, in the description of the present invention, terms such as "up", "down", "left", "right", "inside", and "outside" indicating directions or positional relationships are based on the directions or positional relationships shown in the accompanying drawings. This is only for the convenience of description and does not indicate or imply that the device or element must have a specific orientation, be constructed and operated in a specific orientation. Therefore, it cannot be understood as a limitation on the present invention.

[0032] Furthermore, it should be noted that, in the description of the present invention, unless otherwise expressly specified or limited, the terms "mounted," "connected," and "connected" should be understood in a broad sense. For example, they may refer to fixed connections, detachable connections, or integral connections; mechanical connections or electrical connections; direct connections or indirect connections through an intermediate medium; and internal communication between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on specific circumstances.

[0033] See also Figure 1 and Figure 2 As shown, Figure 1 Schematic diagram of a flow chart of a method for detecting moving objects based on deep learning according to an embodiment of the present invention. Figure 2 FIG. 1 is a schematic diagram of a method for detecting moving objects based on deep learning according to an embodiment of the present invention. The present invention provides a method for detecting moving objects based on deep learning, comprising:

[0034] Step S1: using a convolutional neural network to extract features of the two original images to be compared to obtain an original feature map and a comparison feature map;

[0035] Step S2, the original feature map and the comparison feature map are passed through a differential network to obtain a feature difference map; after obtaining, the regions of the original feature map and the comparison feature map in S1 where the corresponding feature difference maps exceed a threshold are extracted as original feature difference regions and comparison feature difference regions, respectively. After extraction, the feature difference maps are passed through two or more fully connected networks to obtain feature difference values;

[0036] Step S3, passing the original feature difference region and the compared feature difference region through one or more convolutional neural network layers to obtain an original deep feature map and a compared deep feature map, respectively. After obtaining the original deep feature map and the compared deep feature map, passing the original deep feature map and the compared deep feature map through one or more fully connected layers to obtain an original deep feature value and a compared deep feature value, respectively.

[0037] Step S4, comparing the original deep feature value and the compared deep feature value to obtain similar feature values ​​of objects in the difference area;

[0038] In step S5, a comprehensive judgment is made as to whether the object has moved by combining the feature difference value obtained in step S2 and the feature similarity value of the object in the difference area obtained in step S4.

[0039] Specifically, this method uses a smaller network than deep learning target detection to extract features of objects in the image. It then uses a differential network to determine whether there is movement. This approach requires low CPU / GPU computing power, making it suitable for deployment on edge devices like ARM. Furthermore, it doesn't require extracting specific object categories; it only needs to determine whether the texture features extracted by deep learning have changed to determine whether an object is moving.

[0040] Specifically, the specific calculation for comprehensively judging whether the object is moving in step S5 is as follows:

[0041] A = diff × (1.0-simi);

[0042] Throughout, A represents the final judgment value, diff represents the feature difference value, and simi represents the object similarity value in the difference area.

[0043] Specifically, after the final determination value A is obtained, it is compared with a threshold value. If the final determination value A ≥ the threshold value, it indicates that the object has moved. If the final determination value A < the threshold value, it indicates that the object has not moved.

[0044] Specifically, the present invention determines whether the object has moved by comparing the final determination value A with a threshold value, wherein the final determination value A is accurately calculated using a preset formula, thereby being able to accurately determine whether the object has moved.

[0045] Specifically, the feature difference value and the feature similarity value of the object in the difference area in step S5 need to be output through a Sigmoid activation function before comprehensive judgment.

[0046] Specifically, the feature difference value and the feature similarity value of the object in the difference area need to be output through a Sigmoid activation function before comprehensive judgment, which can improve the accuracy of comprehensive judgment of whether the object is moving.

[0047] Specifically, the feature difference value in step S2 is the degree of difference between the original feature difference region and the compared feature difference region. In this embodiment, the degree of difference is the degree of image difference.

[0048] Specifically, the object similarity value of the different regions in step S4 is determined by whether the different regions are the same object at different positions.

[0049] Specifically, the convolutional neural network used in step S1 uses the existing open-source MobileNet series for mobile network extraction, and the number of network layers is appropriately adjusted to meet the computing power and memory requirements of the target ARM device for deployment. In this embodiment, the convolutional neural network can use the first four layers of MobileNet V1 as the backbone network for feature extraction, or the first six layers of MobileNet V2 as the backbone network for feature extraction, and can be flexibly combined according to user needs.

[0050] Specifically, see Figure 3 As shown, it is a schematic diagram B of the moving object detection method based on deep learning according to an embodiment of the present invention. After obtaining the original feature map and the comparison feature map in step S1, the original feature map and the comparison feature map can be passed through a differential network to obtain a differential feature map. After obtaining the differential feature map, the differential feature map is passed through one or more layers of a fully connected network to finally obtain a differential value.

[0051] Specifically, after the difference value is obtained, the difference value is directly passed through a Sigmoid activation function to determine whether there is a moving object in the picture.

[0052] Specifically, the present invention directly differentiates two feature maps and then fully connects them to determine whether an object has moved; without refining the features of the difference area, the judgment of the moving object can still be achieved, but false alarms may occur.

[0053] Thus far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art may make equivalent changes or substitutions to the relevant technical features, and the technical solutions after such changes or substitutions will fall within the scope of protection of the present invention.

Claims

1. A moving object detection method based on deep learning, characterized in that: include: Step S1: using a convolutional neural network to extract features of the two original images to be compared to obtain an original feature map and a comparison feature map; Step S2, obtaining a feature difference map by applying a differential network to the original feature map and the comparison feature map; After obtaining, the regions where the corresponding feature difference maps in the original feature map and the contrast feature map in S1 exceed the threshold are extracted as the original feature difference region and the contrast feature difference region respectively. After extraction, the feature difference maps are passed through two or more fully connected networks to obtain feature difference values; Step S3, passing the original feature difference region and the compared feature difference region through one or more convolutional neural network layers to obtain an original deep feature map and a compared deep feature map, respectively. After obtaining the original deep feature map and the compared deep feature map, passing the original deep feature map and the compared deep feature map through one or more fully connected layers to obtain an original deep feature value and a compared deep feature value, respectively. Step S4, comparing the original deep feature value and the compared deep feature value to obtain similar feature values ​​of objects in the difference area; In step S5, a comprehensive judgment is made as to whether the object has moved by combining the feature difference value obtained in step S2 and the feature similarity value of the object in the difference area obtained in step S4.

2. The method for detecting moving objects based on deep learning according to claim 1, wherein: The specific calculation for comprehensively judging whether the object is moving in step S5 is as follows: A = diff × (1.0-simi); Throughout, A represents the final judgment value, diff represents the feature difference value, and simi represents the object similarity value in the difference area.

3. The method for detecting moving objects based on deep learning according to claim 2, wherein: After the final determination value A is obtained, it is compared with the threshold. If the final determination value A ≥ the threshold, it indicates that the object has moved. If the final determination value A < the threshold, it indicates that the object has not moved.

4. The method for detecting moving objects based on deep learning according to claim 1, wherein: The feature difference value and the feature similarity value of the object in the difference area in step S5 need to be output through a Sigmoid activation function before comprehensive judgment.

5. The method for detecting moving objects based on deep learning according to claim 1, wherein: The feature difference value in step S2 is the degree of difference between the original feature difference region and the compared feature difference region.

6. The method for detecting moving objects based on deep learning according to claim 1, wherein: The object similarity value of the different regions in step S4 is determined by whether the different regions are the same object at different positions.

7. The method for detecting moving objects based on deep learning according to claim 1, wherein: The convolutional neural network used in step S1 is extracted using the existing mobile open source network MobileNet series, and the number of network layers is appropriately adjusted to meet the computing power and memory requirements of the target ARM device to be deployed.

8. The method for detecting moving objects based on deep learning according to claim 1, wherein: After obtaining the original feature map and the comparison feature map in step S1, the original feature map and the comparison feature map can also be passed through a differential network to obtain a differential feature map. After obtaining, the differential feature map is passed through one or more layers of a fully connected network to finally obtain a differential value.

9. The method for detecting moving objects based on deep learning according to claim 8, wherein: After the difference value is obtained, it is directly passed through the Sigmoid activation function to determine whether there is a moving object in the picture.

Images