Moving object detection method based on data distribution difference and multi-scale feature fusion

Abstract

The present invention is a motion target detection method based on the fusion of data distribution differences and multi-scale features. It combines a fast background subtraction algorithm with a foreground segmentation algorithm to perform motion target detection, so as to achieve the effect of not only finding the motion target but also accurately segmenting it. The present invention first uses the frame difference method and background subtraction method to detect the motion target, preliminarily extracts the motion features, and then uses a multi-scale foreground extraction network to perform foreground extraction. Finally, the feature results of the two are subjected to IOU calculation to obtain a binary mask of the motion target. Secondly, the present invention improves the multi-scale foreground extraction network, adds a channel attention mechanism to its decoder, and enables it to assign different weights to different channels of the feature map, remove the fused features and noise, and obtain a cleaner foreground. The method of the present invention was experimented on the mainstream dataset CDNet2014 for motion target detection. The results show that the algorithm can accurately detect moving targets in various scenarios and has good comprehensive performance.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision technology, and in particular relates to a moving target detection method based on data distribution difference and multi-scale feature fusion. Background Art

[0002] The ultimate goal of computer vision research is to use computers and other intelligent devices to replace humans in perceiving, interpreting, and responding to their surroundings. With the advancement of science and technology, moving object detection in computer vision is finding application in an increasing number of fields. It plays a fundamental role in intelligent video surveillance, robotic visual navigation, virtual reality, and medical diagnosis (cell state tracking), making it a crucial and challenging task.

[0003] Motion target detection is the process of finding areas of motion. The detected motion areas can provide reference areas for subsequent tasks such as target recognition and tracking, and behavioral analysis. It is generally assumed that the background of a video captured by a stationary camera is stationary and the foreground is moving. However, in real-world scenarios, foreground objects may exhibit intermittent motion, and the background may contain local motion, such as swaying leaves or rippling water. Therefore, detecting moving targets in the presence of complex backgrounds, changing lighting, noise interference, intermittent motion, and camouflage is a very challenging task. Motion target detection aims to automatically generate accurate and consistent pixel masks for foreground objects.

[0004] Research on moving object detection algorithms has made significant progress in recent years. However, detecting moving objects in videos shot under uncontrolled weather conditions, varying lighting conditions, or dynamic backgrounds is a challenging task. Consequently, researchers have conducted extensive research on this issue. This research can be broadly categorized into machine learning-based methods and deep learning-based methods.

[0005] Machine learning-based methods have provided significant performance improvements for many computer vision applications. Currently, the most commonly used machine learning-based motion target detection methods can be roughly divided into five categories: frame difference method, optical flow method, background difference method, probability model method and method combining time and space information.

[0006] Although these methods have achieved certain results, their applicability is limited. In outdoor conditions, due to the influence of lighting, occlusion and shadow, the appearance of the detected moving objects will change, which will affect the accuracy of moving object detection.

[0007] Deep learning-based methods have been a hot topic in motion target detection research in recent years, achieving promising results in multiple image and video segmentation applications. They achieve high detection accuracy and overcome practical challenges such as intermittent object motion, lighting changes, shadows, and camouflage. Deep learning-based algorithms, incorporating optical flow, frame subtraction, background subtraction, and various other methods, also achieve high accuracy. By integrating a machine learning-based data distribution difference algorithm with a motion target detection method based on a deep convolutional neural network, a motion target detection model has been developed that performs well even in dynamic backgrounds. This model is of great significance to computer vision applications. However, these methods cannot fully and accurately segment foreground targets while simultaneously excluding stationary foreground targets.

[0008] In existing research, most highly accurate deep learning algorithms segment all foreground objects without adding motion information. Consequently, their detection results include not only moving objects but also stationary objects. The few algorithms that do include motion information also suffer from a decrease in segmentation accuracy. Therefore, the goal of this paper is to propose a robust moving object detection network that can accurately identify moving objects within foreground objects while excluding stationary foreground objects. This network is adaptable to varying weather and lighting conditions, and performs well in environments with dynamic backgrounds and camera shake. Summary of the Invention

[0009] In response to the shortcomings of traditional moving target detection methods, this paper combines the advantages of strong interpretability of machine learning and efficient and accurate segmentation results of deep learning, and proposes a moving target detection method based on data distribution difference and multi-scale feature fusion to improve the accuracy of moving target detection.

[0010] The scientific problem that the present invention intends to solve is: how to detect moving targets and accurately segment them under complex backgrounds, lighting changes, noise interference, intermittent motion and camouflage. The present invention combines a fast background subtraction algorithm with a foreground segmentation algorithm to detect moving targets, so as to achieve the effect of both finding moving targets and accurately segmenting them. On the other hand, the present invention is a moving target detection algorithm based on the fusion of data distribution differences and multi-scale features, that is, first using the frame difference method and the background difference method to detect moving targets, preliminarily extract motion features, and then using a multi-scale foreground extraction network to extract foreground, and finally performing IOU calculation on the feature results of the two to obtain a binary mask of the moving target. Secondly, the present invention improves the multi-scale foreground extraction network, adds a channel attention mechanism to its decoder, and allows it to assign different weights to different channels of the feature map, removes the fused features and noise, and obtains a cleaner foreground. The specific steps are as follows:

[0011] The first step is efficient background modeling based on sparse representation and iterative outlier removal:

[0012] In the fast background extraction part based on machine learning, the present invention adopts an efficient background modeling algorithm based on sparse representation and iterative removal of outliers. This algorithm does not use all frames to simulate the background, but simply uses the identification frame, uses sparse representation to reduce the size of the video, and explores its identification frame. This can effectively improve the speed of background modeling. Then, a cyclic iterative algorithm based on the PCP model is used to extract the background from the identification frame. The cyclic iterative process consists of a tensor model and pixel processing. Among them, the tensor model considers the overall information, and the pixel processing focuses on specific information. After the cyclic iterative process, an accurate background image can be quickly obtained;

[0013] The second step is to combine the background difference method with the frame difference method to perform preliminary moving target detection:

[0014] After obtaining the background image (i.e., there are no people or other objects of interest in the image) through the first step, it is input into the network, and the current frame and the background image are used to perform background subtraction to preliminarily estimate the moving target. The background subtraction method is to perform a difference between the current image and the known background model and calculate the degree of matching between the current image and the background model. For the Frame image, For the background image, is the coordinate point position, is the threshold value, and the binary image obtained after the background difference method is the moving target detection result. The formula is shown in (1):

[0015] ;

[0016] However, if only the background subtraction method is used to obtain the moving target, the segmentation accuracy is often unsatisfactory. Therefore, the present invention also uses the frame difference method, that is, the current frame and the previous frame are differentiated to obtain the outer contour of the moving target. The frame difference method is to first convert the color image into a grayscale image, and then perform a differential operation on the grayscale image. The difference method between consecutive images in a video image sequence is to compare the pixel size differences of corresponding pixel positions of adjacent images, thereby detecting the moving target. The pixel size of the position where the moving target is located will change accordingly, while the background area of the non-target remains unchanged or changes less. Finally, the present invention combines the results obtained by the background difference method and the frame difference method to obtain a moving target that is as complete as possible.

[0017] The third step is to use the encoder network to extract low-level features:

[0018] The present invention comprises a triplet CNN that encodes features at three different scales. The first four blocks of the encoder network are modified copies of a pre-trained VGG-16 network, i.e., the third and fourth maximum pooling layers are removed from the VGG-16 network, and dropout is inserted between each layer of the fourth convolutional block to prevent overfitting. The input of the encoder of the present invention is a single image transformed into three images of different scales. These three images are simultaneously fed to the triplet CNN. The architecture of the triplet CNN is identical and shares weights. After feature extraction by the triplet CNN, feature maps F1, F2, and F3 of three different scales are obtained. F2 and F3 are scaled to match the scale of F1, and then concatenated along the depth axis to form a combined feature image, i.e., feature map F.

[0019] The fourth step is to use the multi-scale feature extraction method of the fusion attention mechanism to extract the foreground of the current frame:

[0020] The present invention adopts a multi-scale feature extraction method that integrates the attention mechanism to extract the foreground of the current frame. After the multi-scale encoder network extracts the low-level features in the third step, a feature map F is obtained. The feature map F is input into the attention transposed convolutional neural network, and the weights are learned for decoding. The final output is a segmentation mask with the same size as the original input image.

[0021] Step 5: Calculate the IOU between the moving targets obtained by machine learning and all foreground targets (including moving targets and stationary targets) obtained by deep convolutional neural networks:

[0022] Motion detection results from machine learning-based methods are often inaccurate, providing only an approximate region containing a moving target. Foreground object detection methods can fully detect the target, but lack motion information. Therefore, the present invention uses IOU to add motion information for moving object detection. An image may contain multiple objects, including both moving and non-moving ones. When the IOU value exceeds a threshold, a segmentation mask for the foreground target is output.

[0023] The beneficial effects of the present invention are as follows: Compared with the prior art, the present invention has the following outstanding substantial features and significant improvements:

[0024] (1) The present invention combines a fast background subtraction algorithm with an image segmentation algorithm to detect moving targets, which can achieve the effect of both finding moving targets and accurately segmenting them.

[0025] (2) The method of the present invention is a study on a motion target detection algorithm based on the fusion of data distribution differences and multi-scale features. That is, the frame difference method and the background difference method are first used to detect the motion target, and then the multi-scale foreground extraction network is used to extract the foreground. Finally, the IOU calculation is performed on the results of the two to obtain a binary mask of the motion target, which improves the accuracy of motion target recognition and reduces the situation where the static foreground is mistakenly identified as a static target.

[0026] (3) The present invention improves the multi-scale foreground extraction network by adding a channel attention mechanism to its decoder, so that it assigns different weights to different channels of the feature map, removes the integrated features and noise, and can obtain a cleaner foreground. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The present invention will be further described below with reference to the accompanying drawings and examples.

[0028] Figure 1 This is the overall model diagram of the method of the present invention.

[0029] Figure 2 This is a diagram of the fast background modeling algorithm framework.

[0030] Figure 3 Flowchart of the background difference method. DETAILED DESCRIPTION

[0031] Figure 1 The illustrated embodiment demonstrates the process of the method of the present invention: first, sparse representation is introduced into the task of large-scale stable background modeling, and the size of the video is reduced by exploring its recognizable frames. Then, a looping iterative process is used to quickly extract the background from the discriminant frame set. Then, moving targets are detected using frame difference and background difference methods, and the detection results obtained by the two methods are combined. Then, foreground segmentation is performed using an attention convolutional neural network. The foreground extraction of the attention convolutional neural network is combined with the moving target detection method based on machine learning to finally obtain the moving target.

[0032] Figure 2 The illustrated embodiment illustrates the process of the present invention's rapid background extraction method. The background modeling method employed in this invention is a highly efficient algorithm based on sparse representation and outlier removal. The specific process involves applying a sparse representation to the original video sequence consisting of all frames to reduce the video size and identify the discriminant frames, thereby improving background modeling speed. Then, a cyclic iterative algorithm based on the PCP model is applied to the selected discriminant frame set. This algorithm, which combines a tensor model and pixel-level processing, yields a precise background image after multiple iterations.

[0033] Figure 3The illustrated embodiment illustrates a flowchart of the background subtraction method of the present invention, which uses multiple video frames to combine local and global pixel intensity information for background estimation. After estimating the clean background, a simple pixel-level subtraction is performed between the calculated background image and the input video frame to estimate the foreground object. The local and global information of the current frame are combined to classify the pixel (foreground or background) using the pixel values at that location and surrounding locations.

[0034] Example 1

[0035] The moving target detection method based on data distribution difference and multi-scale feature fusion in this embodiment is a method that combines the background difference method and frame difference method based on machine learning with the fusion of multi-scale depth features. The specific steps are as follows:

[0036] The first step is background extraction based on sparse representation and outlier iteration points:

[0037] (1.1) Discriminative search using sparse representation

[0038] This embodiment refines the frame sequence to obtain a new information set consisting of selected distinguishable frames. By solving the maximum linear independent group of video frames, sparse representation is used to explore discriminative frames. The sparse representation process based on video content is highly robust to noise. Once a frame is represented by other frames, its content is no longer distinguishable. Therefore, this embodiment uses some frames to represent all frames:

[0039] ;

[0040] in, is the Frobenius norm, which is equal to the square root of the sum of the squares of the tensor terms. is a coefficient matrix, To balance the two parts. yes norm, is The sum of all rows in . No. Line record Frames represent coefficients of other frames, No. The other frames in the column represent The coefficients of the frame can then be observed by The corresponding rows in are used to infer the role of each frame. Frames with coefficients of zero are considered redundant. The non-zero rows in correspond to the discriminant frames. Form a new set To include all the discriminant frames. After sparsely representing the video, the original video Refine to form a new set of selected discriminant frames .

[0041] (1.2) Background extraction using iterative loop method

[0042] This embodiment uses a cyclic iterative background extraction design. In each iteration, this embodiment uses a PCP model to extract the selected discriminant frame set. In the pixel-level outlier removal strategy, this embodiment uses the purified mean framework to improve the selected discriminant framework set. Then continue to iterate until the purified average frame converges to a fixed frame, which is the background of the current frame.

[0043] The second step is to combine the background difference method with the frame difference method to perform preliminary moving target detection:

[0044] (2.1) Using background difference method to detect moving targets:

[0045] The background obtained in the first step is used as the reference background for the background difference method in this step. The pixel difference is divided into foreground and background using a threshold. Pixels with a difference greater than the threshold are classified as foreground, and pixels with a difference less than the threshold are classified as background. The formula is as follows:

[0046] ;

[0047] in,

[0048] ;

[0049] Among them, the pixel points in the image sequence In the The grayscale values of the frame foreground and background are and , is the absolute value of the difference between the two, is the threshold value, distinguishing pixels Foreground or background.

[0050] (2.2) Using frame difference method to detect moving targets:

[0051] Two adjacent frames of images, Frame image position The pixel size at , No. The pixel size of the frame image at the same position is , then the difference in pixel sizes between adjacent image frames is as shown in formula (5).

[0052] ;

[0053] Obviously, the part with a larger difference is the moving target. To extract the detection target, it is necessary to use threshold processing to distinguish the background from the moving target. When the value is less than the threshold, it is judged as a background pixel, otherwise it is a moving target, as shown in formula (6).

[0054] ;

[0055] In the above formula, T is the threshold value, It is the result after “binarization”, and the result of 1 represents a moving target.

[0056] The third step is to use the multi-scale feature extraction method of the fusion attention mechanism to extract the foreground of the current frame:

[0057] In this example, the input image is first converted into images of three different scales, and then a triplet convolutional neural network is used to extract input features. The triplet CNN contains three copies of the CNN, running in parallel with the same input at three different scales. The first four blocks of these networks are modified copies of the pre-trained VGG-16 Net; in this example, the third and fourth max pooling layers are removed, and dropout is inserted between each layer of the fourth convolutional block to prevent overfitting.

[0058] The input of each CNN is a raw RGB image of different sizes. Assume that the input image size is ,in is the image width, is the image height, 3 is the RGB color channel, and at the end of the first convolution block it is converted to 64 dimensions. The feature maps are then multiplied by 2. The max pooling layer downsamples and converts to 128-bit size at the end of the second block. Similarly, these feature maps are multiplied by 2. The maximum pooling layer downsamples and converts to 256-bit size at the end of the third block. Finally, at the end of the fourth block, these feature maps are converted into 512 feature maps of size feature map.

[0059] To avoid overfitting, this example applies dropout regularization after each convolutional layer in the fourth convolutional block and applies zero padding to all convolutional layers in the network to preserve the spatial dimensions of the input in the output.

[0060] The output F of the encoding network is a concatenation of feature maps at three different scales. This map is input to the TCNN, which learns the weights of the decoded feature map and outputs a probability mask. In the network of this embodiment, F has a large depth of 1536 layers, which is due to the concatenation of features at three different scales. For computational efficiency and to increase the nonlinearity of the decision function in the network, we use The convolution layer is transposed to project the high-dimensional feature map depth to the low-dimensional feature map depth. In the 5th block of TCNN, a stride of 1 is used. Transposed convolution, the shape is The connection feature F is mapped to , and finally, project it onto The layer structures of the 6th and 7th blocks are similar to those of the 5th block. In the 8th block, this embodiment uses a stride of 2. The transposed convolution enlarges the feature map to match the original size of the input image. In the 9th block, the feature map is enlarged by operating with a stride of 1. The transposed convolution maps the 64 feature maps of the 8th block into 1 feature map. Finally, the sigmoid function is applied to the last layer to generate a probability mask for each pixel, encoding the probability of the foreground pixel as a value between 0 and 1.

[0061] The fourth step is to use IOU calculation to fuse the results of the second and third steps:

[0062] Neither the preliminary moving target obtained in the second step nor the accurate segmentation result obtained in the third step can guarantee that the target obtained is an accurate moving target. Therefore, the IOU calculation is performed on the two results to obtain the binary mask of the moving target. The IOU calculation formula is shown in formula (7).

[0063] ;

[0064] in is the target obtained by motion detection, is the segmentation mask of the foreground detection part.

[0065] This embodiment conducts experiments on the CDNet database. The CDnet2014 dataset contains 11 categories: baseline, camera shake, bad weather, dynamic background, intermittent object motion, low frame rate, night video, PTZ (pan tilt zoom), shadows, thermal airflow, and turbulence. Covering diverse targets such as cars, pedestrians, and boats, as well as complex environments, the dataset provides a large number of ground truth test frames. Each category contains 4 to 6 sequences. There are 53 different video sequences in total. The spatial resolution of the video frames ranges from 320×240 to 720×576 pixels. In addition, a video sequence can contain 900 to 7000 frames. Almost all video sequences contain different challenging scenes, which makes this dataset suitable for measuring the robustness of the model in each case.

[0066] The following will be analyzed in combination with the experimental results.

[0067] BadWeather is a rainy and snowy scene. Raindrops and snowflakes can be considered dynamic background and global interference. Based on the evaluation results of the four videos, as shown in Tables 1 to 11, the F-measure of our method mostly reached 0.99.

[0068] The Baseline class includes simple scenarios. The effects of the present invention in this scenario are shown in Table 1.

[0069] Table 1 F-measure comparison results of this method in the baseline scenario

[0070] Video Name The method of the present invention (%) Highway 98.65 Office 99.59 Pedestrians 99.61 PETS2006 99.40

[0071] The BadWeather scene is a video shot in rainy and snowy weather. The effects of the present invention in this scene are shown in Table 2.

[0072] Table 2 F-measure comparison results of this method in the BadWeather scenario

[0073] Video Name The method of the present invention (%) Skating 98.32 SnowFall 98.72 Blizzard 99.01 WetSnow 99.53

[0074] CameraJitter is a camera jitter class, and the video it contains is shot by a jittering camera. The effects of the present invention in this scenario are shown in Table 3.

[0075] Table 3 F-measure comparison results of this method in CameraJitter scene

[0076] Video Name The method of the present invention (%) Badminton 98.18 Traffic 98.72 boulevard 99.73 Sidewalk 99.66

[0077] Dynamic-Background includes dynamic backgrounds with different interference levels. The effects of the present invention in this scenario are shown in Table 4.

[0078] Table 4 F-measure comparison results of this method in DynamicBackground scene

[0079] Video Name The method of the present invention (%) Boats 99.95 Canoe 99.84 Fall 99.35 Fountain01 99.96 Fountain02 99.96 Overpass 99.85

[0080] The video in the LowFramerate class is a low frame rate video. The effects of the present invention in this scenario are shown in Table 5.

[0081] Table 5 F-measure comparison results of this method in Low Framerate scenario

[0082] Video Name The method of the present invention (%) Port_0_17fps 99.95 TramCrossroad_1fps 99.69 TunnelExit_0_35fps 99.10 Turnpike_0_5fps 97.26

[0083] In the IntermittentObjectMotion scene, the foreground moves intermittently. The effects of the present invention in this scene are shown in Table 6.

[0084] Table 6 F-measure comparison results of this method in Intermittent Object Motion scenario

[0085] Video Name The method of the present invention (%) AbandonedBox 99.83 Parking 99.96 Sofa 99.15 StreetLight 99.87 Tramstop 99.54 Winter Driveway 99.71

[0086] The video in the NightVideos scene is a video shot at night. The effects of the present invention in this scene are shown in Table 7.

[0087] Table 7 F-measure comparison results of this method in NightVideos scene

[0088] Video Name The method of the present invention (%) BridgeEntry 99.75 BusyBoulvard 99.23 FluidHighway 99.54 StreetCornerAtNight 99.91 TramStation 99.58 Winter Street 99.17

[0089] PTZ is a pan-tilt-zoom video. The effects of the present invention in this scenario are shown in Table 8.

[0090] Table 8 F-measure comparison results of this method in PTZ scene

[0091] Video Name The method of the present invention (%) ContinuousPan 99.95 IntermittentPan 97.59 TwoPositionPTZCam 97.48 ZoomInZoomOut 99.82

[0092] The foreground in the Shadow scene has strong shadow interference, making segmentation more difficult. The effects of the present invention in this scene are shown in Table 9.

[0093] Table 9 F-measure comparison results of this method in Shadow scene

[0094] Video Name The method of the present invention (%) Backdoor 99.26 Bungalows 99.20 BusStation 99.29 CopyMachine 99.21 Cubicle 99.75 PeopleInShade 99.66

[0095] Thermal is a scene captured by a thermal imaging device. The color or grayscale features of its video frame are not obvious compared to common images. The effect of the present invention in this scene is shown in Table 10.

[0096] Table 10 Comparison of F-measure results of this method in the Thermal scenario

[0097] Video Name The method of the present invention (%) Corridor 99.76 DiningRoom 99.64 LakeSide 99.65 Library 99.42 Park 99.52

[0098] The strong airflow disturbance in the turbulence scene causes the video image to be blurred. The camera is far away from the foreground, making the foreground relatively small and difficult to capture. The effect of the present invention in this scene is shown in Table 11.

[0099] Table 11 F-measure comparison results of this method in the Turbulence scenario

[0100] Video Name The method of the present invention (%) Turbulence0 99.95 Turbulence1 99.92 Turbulence2 98.62 Turbulence3 98.34

[0101] The performance of this method in different categories of scenes has good performance in all scenes in the CDNet dataset. It can be seen that the method of the present invention can be applied to moving target detection in different scenes and extreme weather environments, and has strong stability and certain universality.

[0102] In order to verify the advantages of the method of the present invention over the motion target detection method, this embodiment selects five currently widely used motion target detection methods for comparison with the motion target detection method of the present invention based on data distribution difference and multi-scale feature fusion. The comparison results are shown in Table 12.

[0103] Table 12 F-measure comparison results of different methods on CDNet dataset

[0104]

Claims

1. A moving target detection method based on data distribution difference and multi-scale feature fusion, characterized by: The fast background subtraction algorithm is combined with the image segmentation algorithm for moving target detection. The frame difference method and background difference method are used for moving target detection. Then the multi-scale foreground extraction network is used for foreground extraction. Finally, the IOU calculation is performed on the two results to obtain the binary mask of the moving target. The multi-scale foreground extraction network is improved and the channel attention mechanism is added to its decoder to give different weights to different channels of the feature map, remove the integrated features and noise, and obtain a cleaner foreground. The specific steps are as follows: The first step is efficient background modeling based on sparse representation and iterative outlier removal: In the fast background extraction part based on machine learning, an efficient background modeling algorithm based on sparse representation and iterative outlier removal is adopted. Instead of using all frames to simulate the background, this algorithm simply uses the identification frame, using sparse representation to reduce the size of the video, exploring its identification frame, and effectively improving the speed of background modeling. Then, a cyclic iterative algorithm based on the PCP model is used to extract the background from the identification frame. The cyclic iterative process consists of a tensor model and pixel processing. The tensor model considers the overall information, while the pixel processing focuses on specific information. After the cyclic iterative process, an accurate background image is obtained. The second step is to combine the background difference method with the frame difference method to perform preliminary moving target detection: After obtaining the background image in the first step, that is, there is no person or other object of interest in the image, it is input into the network, and the current frame and the background image are used to perform background difference method to preliminarily estimate the moving target. The background difference method is to perform difference between the current image and the known background model and calculate the matching degree between the current image and the background model. For the Frame image, For the background image, is the coordinate point position, is the threshold value, and the binary image obtained after the background difference method is the moving target detection result. The formula is shown in (1): ； The frame difference method is used, that is, the current frame is differentiated from the previous frame to obtain the outer contour of the moving target. The frame difference method is to first convert the color image into a grayscale image, and then perform a difference operation on the grayscale image. The difference method between consecutive images in a video image sequence is to compare the pixel size differences of corresponding pixel positions in adjacent images to detect the moving target. The pixel size of the moving target position will change accordingly, while the non-target background area will remain unchanged or change slightly. Finally, the results obtained by the background difference method and the frame difference method are combined to obtain a complete moving target. The third step is to use the encoder network to extract low-level features: It contains a triplet CNN that encodes features at three different scales. The first four blocks of the encoder network are modified copies of the pre-trained VGG-16 network, that is, the 3rd and 4th maximum pooling layers are removed on the basis of the VGG-16 network, and dropout is inserted between each layer of the 4th convolutional block to prevent overfitting. The input of the encoder is a picture transformed into three pictures of different scales. These three pictures are sent to the triplet CNN at the same time. The architecture of the triplet CNN is exactly the same and shares weights. After feature extraction by the triplet CNN, three feature maps F1, F2, and F3 of different scales are obtained. F2 and F3 are scaled to match the scale of F1, and then connected along the depth axis to form a combined feature image, i.e., feature map F; The fourth step is to use the multi-scale feature extraction method of the fusion attention mechanism to extract the foreground of the current frame: A multi-scale feature extraction method integrating attention mechanism is used to extract the foreground of the current frame. After the multi-scale encoder network in the third step extracts the low-level features, a feature map F is obtained. The feature map F is input into the attention transposed convolutional neural network, and the weights are learned for decoding. The final output is a segmentation mask of the same size as the original input image. Step 5: Calculate the IOU of the moving targets obtained using machine learning methods and all foreground targets obtained using deep convolutional neural networks, including moving targets and stationary targets.

2. The moving target detection method based on data distribution difference and multi-scale feature fusion according to claim 1, characterized in that: Use sparse representation and outlier iteration point method to extract background, refine the frame sequence, and obtain a new information set, which consists of selected distinguishable frames. By solving the maximum linear independent group of video frames, sparse representation is used to explore the discriminant frame. The sparse representation process based on video content is highly robust to noise. Once a frame is represented by other frames, its content is no longer distinguishable. Therefore, some frames are used to represent all frames: ； in, is the Frobenius norm, which is equal to the square root of the sum of the squares of the tensor terms, is a coefficient matrix, To balance these two parts, in addition, yes norm, is The sum of all rows in , No. Line record Frames represent coefficients of other frames, No. The other frames in the column represent The coefficients of the frame are then obtained by observing The corresponding rows in are used to infer the role of each frame. Frames with zero coefficients are considered redundant. The non-zero rows in correspond to the discriminant frames, forming a new set To include all the discriminant frames, after sparsely representing the video, the original video Refine to form a new set of selected discriminant frames .

3. The moving target detection method based on data distribution difference and multi-scale feature fusion according to claim 1, characterized in that: Use IOU calculation to combine and filter the preliminary moving target detection results and foreground segmentation results to obtain the binary mask of the moving target. The IOU calculation formula is as follows: ； in is the target obtained by motion detection, It is the segmentation mask of the foreground detection part. When the IOU value is greater than the threshold, the segmentation mask of the foreground target is output.

Images