Moving object detection method based on multi-frame images

Abstract

The present invention relates to the field of image processing technology, and in particular to a method for detecting moving targets based on multi-frame images, comprising: S1: inputting an aerial remote sensing image sequence, and performing normalization processing to obtain an impact data set to be trained; S2: inputting the kth frame image and the k-nth frame image into the first and second basic semantic feature extraction networks respectively, and performing feature extraction and fusion through the first motion fusion network to obtain a motion fusion feature map containing motion information; S3: inputting the motion fusion feature map into the high-level semantic feature extraction network and the downsampling network respectively, and performing information fusion on the corresponding downsampling feature map in the second motion fusion network; S4: inputting the three enhanced feature maps obtained in step S3 into a decoupling detection network to obtain a final network output. The present invention accurately and quickly obtains the quantity and position information of moving targets and stationary targets, and effectively reduces the false alarm rate.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a moving target detection method based on multi-frame images. Background Art

[0002] Moving target detection involves detecting changing regions in an image sequence and extracting the moving target from the background image. This extracted moving target information provides a reference area for subsequent tasks such as target recognition, tracking, and behavior analysis. Accurately and quickly locating moving targets in remote sensing image sequences is crucial for both civilian and military applications. In civilian applications, it can be used for intelligent control, human-computer interaction, and visual navigation. In the military, it can be used for target surveillance, long-range early warning, and precision guidance. However, moving target detection based on manual interpretation suffers from low data utilization, poor intelligence timeliness, and is susceptible to the physical, mental, and subjective influences of the detector. Existing moving target detection methods for aerial image sequences generally divide the task into two steps: first, the background motion generated by the moving platform is compensated for to generate a static background image or an updateable background image. Then, moving target detection is performed on the background image using moving target detection techniques against a static background. However, such detection methods rely heavily on a realistic approximation of the motion model and robust solution of the motion model parameters. Therefore, accurate temporal modeling of the same pixel is difficult to achieve when the aerial remote sensing imaging platform is in motion. Furthermore, background motion compensation is extremely time-consuming, and detection results require the cooperation of post-processing steps to be achieved, which means they lack real-time performance. Existing network structures typically use an additional two-frame difference image or optical flow image input network as the motion detection branch, and then combine the results of the target detection network with the motion detection results to obtain information about the moving target. Summary of the Invention

[0003] In order to solve the problems that the existing feature extraction network backbone (including the basic semantic feature extraction network and the advanced semantic feature extraction network) extracts features of different convolution depths of two frames of images respectively, which is time-consuming, and the detection results require the cooperation of post-processing steps to be realized, and have the disadvantages of low detection efficiency, poor reliability and lack of real-time performance, the present invention provides a moving target detection method based on multi-frame images. The method has simple parameter setting and low computational complexity, can quickly extract moving targets and stationary targets in aerial image sequences, and accurately and quickly obtain the number and position information of moving targets and stationary targets, effectively reducing the false alarm rate and enabling effective real-time detection.

[0004] The present invention proposes a method for detecting moving objects based on multiple frames of images, which specifically includes the following steps:

[0005] S1: Input the aerial remote sensing image sequence and normalize the aerial remote sensing image sequence to obtain the impact data set to be trained;

[0006] S2: Input the k-th frame image into the first basic semantic feature extraction network to obtain a first shallow feature map, and input the kn-th frame image into the second basic semantic feature extraction network to obtain a second shallow feature map, and input the first shallow feature map and the second shallow feature map into the first motion fusion network for feature extraction and fusion to obtain a motion fusion feature map containing motion information;

[0007] S3: The motion fusion feature map is input into the high-level semantic feature extraction network and the downsampling network respectively, and three semantic feature maps and three downsampling feature maps are obtained accordingly. After the three semantic feature maps pass through the path aggregation network, the information of the three downsampling feature maps is fused in the second motion fusion network to obtain three enhanced feature maps containing motion information and semantic features;

[0008] S4: Input the three enhanced feature maps obtained in step S3 into the decoupling detection network to obtain the final network output.

[0009] Preferably, step S1 specifically includes the following steps:

[0010] S11: Crop the input size of all images in the aerial remote sensing image sequence to 608 608 pixels;

[0011] S12: Pair the k-th frame image with the kn-th frame image to obtain an impact data set to be trained.

[0012] Preferably, in step S12, when the moving distance of the moving target between two adjacent frame images is large, n in the kn-th frame image is set to n<5; when the moving distance of the moving target between two adjacent frame images is small, n in the kn-th frame image is set to n≥5.

[0013] Preferably, the first basic semantic feature extraction network and the second basic semantic feature extraction network both include a Conv-1 network layer and a Res-2 network layer, and the first motion fusion network includes a first MEF module, a second MEF module, a 1 1Ordinary convolution layer and downsampling module.

[0014] Preferably, step S2 specifically includes the following steps:

[0015] S21: Input the k-th frame image and the kn-th frame image into the Conv-1 network layer of the first basic semantic feature extraction network and the second basic semantic feature extraction network respectively, and obtain the feature map P11 and the feature map P21 respectively. Perform motion information feature extraction and fusion on the feature map P11 and the feature map P21 through the first MEF module to obtain the feature map P1;

[0016] S22: The feature map P11 and the feature map P21 are respectively transmitted to the Res-2 network layer in the first basic semantic feature extraction network and the second basic semantic feature extraction network to obtain a first shallow feature map and a second shallow feature map respectively. The second MEF module is used to extract and fuse the motion information features of the first shallow feature map and the second shallow feature map to obtain a feature map P2.

[0017] S23: The feature map P1 is downsampled by the downsampling module and then added to the feature map P2. 1. Ordinary convolution layer performs feature integration to obtain motion fusion feature map.

[0018] Preferably, the first MEF module and the second MEF module include a first 1 1 ordinary convolution layer, the second 1 1 ordinary convolution layer, the third 1 1 ordinary convolution layer, first 3 3 ordinary convolutional layers, FDE modules and Prewitt modules, wherein the feature map P11 or the first shallow feature map passes through the first 1 1 ordinary convolution layer and then input to FDE module and Prewitt module respectively, while the feature map P21 or the second shallow feature map passes through the second 1 1 normal convolution layer and then the first 3 3. Ordinary convolution layer performs background compensation to obtain compensation feature map. After inputting the compensation feature map into FDE module, the temporal and spatial difference compensation feature is obtained. After the temporal and spatial difference compensation feature and the spatial texture feature map output by Prewitt module are subjected to Cat operation, the third 1 1 Ordinary convolution layer completes the feature extraction and fusion of motion information.

[0019] Preferably, the spatial gradient splicing formula of the FDE module is:

[0020] ;

[0021] in, The first shallow feature map passes through the first 1 1The feature map obtained after ordinary convolution layer processing, is the compensation feature map, i is the current number of channels, n is the number of channels of the compensation feature map, It is the feature difference obtained by enhancing each channel of the compensation feature map.

[0022] Preferably, the high-level semantic feature extraction network includes a Res-3 network layer, a Res-4 network layer and a Res-5 network layer, which are connected in sequence; the downsampling network includes a Down-1 network layer, a Down-2 network layer and a Down-3 network layer, which are connected in sequence.

[0023] Preferably, step S3 specifically includes the following steps:

[0024] S31: Add the motion fusion feature map to the first shallow feature map and pass them through the Res-3 network layer, the Res-4 network layer and the Res-5 network layer in sequence to obtain feature maps P13, P14 and P15 respectively. At the same time, pass the motion fusion feature map through the Down-1 network layer, the Down-2 network layer and the Down-3 network layer in sequence to obtain the first motion information T1, the second motion information T2 and the third motion information T3 respectively.

[0025] S32: After the feature map P13, the feature map P14 and the feature map P15 pass through the path aggregation network respectively, they are added to the first motion information T1, the second motion information T2 and the third motion information T3 in the second motion fusion network to obtain three enhanced feature maps.

[0026] Preferably, the decoupling detection network includes a fourth 1 1 ordinary convolution layer, the first branch and the second branch, the first branch and the second branch are connected in parallel, wherein the first branch is connected in sequence to the second 3 3 ordinary convolutional layers, the third 3 3 ordinary convolutional layers, the fifth 1 1 ordinary convolution layer and the first Feature convolution layer, the second branch is the fourth 3 3 ordinary convolutional layers, the fifth 3 3 ordinary convolutional layers, sixth 1 1 ordinary convolution layer, seventh 1 1 ordinary convolution layer, the second Feature convolution layer and the third Feature convolution layer, and the sixth 1 1 The branch where the ordinary convolution layer and the second Feature convolution layer are located is the same as the seventh 1 1 The branch where the ordinary convolution layer and the third feature convolution layer are located is connected in parallel, and the three enhanced feature maps pass through the fourth 1 1 After the ordinary convolution layer, the network is input to the first branch and the second branch for decoupling.

[0027] Compared with the prior art, the present invention can achieve the following beneficial effects:

[0028] (1) The present invention proposes a method of using both early fusion and late fusion for motion information fusion. Information fusion is performed twice in the middle of the feature extraction network backbone (including the basic semantic feature extraction network and the advanced semantic feature extraction network) and before the decoupling detection network. Finally, the separation of moving targets and stationary targets is achieved through the decoupling detection head.

[0029] (2) The present invention proposes a motion information extraction module, which uses a classic edge detection operator to perform edge detection on each channel of the feature, and combines the difference features of the enhanced features to obtain enhanced motion information. The feature map containing motion information is downsampled to the same spatial dimension of the semantic feature map through a downsampling method, and the semantic information and motion information are fused through an element-by-element addition method to obtain an enhanced feature map, thereby enhancing the robustness of the motion target detection method based on multi-frame images to motion target detection.

[0030] (3) The present invention inputs two adjacent frames of images based on the acquired image sequence and directly outputs the target's motion state and position information end-to-end, which reduces the detection process of moving targets, improves the detection speed of moving targets, and enables the network to achieve real-time detection. In addition, the appropriate two-frame interval parameter can be set according to the actual speed of the moving target, so that the moving target detection method based on multi-frame images proposed by the present invention can simultaneously take into account both fast-moving and slow-moving targets.

[0031] (4) Based on the characteristic that the difference between two frames of images contains motion information, the present invention strengthens the motion information between the two frames of images by setting a first MFE module and a second MFE module. According to the characteristic that shallow features contain motion information, the high-level semantic feature extraction part of the feature extraction network backbone of the second frame of image is removed, thereby greatly reducing the amount of calculation. The motion information contained in the first shallow feature map is added to the high-level semantic feature extraction part of the feature extraction network backbone of the first frame, so as to obtain a better motion feature extraction effect and increase the accuracy of moving target detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 2. This is a diagram showing an application scenario of a method for detecting a moving target based on multiple frames of images according to an embodiment of the present invention;

[0033] Figure 2 is a flow chart of a method for detecting a moving target based on multiple frames of images according to an embodiment of the present invention;

[0034] Figure 3 2 is a schematic diagram of a network structure of a moving target detection method based on multiple frame images provided by an embodiment of the present invention;

[0035] Figure 4 2 is a schematic diagram of the network structure of the MFE module provided according to an embodiment of the present invention;

[0036] Figure 5 3 is a schematic diagram of the network structure of a decoupling detection network provided according to an embodiment of the present invention. DETAILED DESCRIPTION

[0037] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In the following description, identical modules are denoted by identical reference numerals. In the case of identical reference numerals, their names and functions are also identical. Therefore, their detailed description will not be repeated.

[0038] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and do not constitute a limitation of the present invention.

[0039] The embodiment of the present invention provides a method for detecting moving targets based on multiple frames of images. The method strengthens the motion information between the two frames of images by setting a first MFE module and a second MFE module, removes the high-level semantic feature extraction part in the feature extraction network backbone (including the basic semantic feature extraction network and the high-level semantic feature extraction network) of the second frame of image, and adds the motion information contained in the first shallow feature map to the high-level semantic feature extraction network of the first frame feature extraction network backbone to obtain a better motion feature extraction effect and increase the detection accuracy of moving targets. Finally, the semantic features extracted by the high-level semantic feature extraction network are passed through a path aggregation network (PAN network) and then fused one by one with the motion information output by the corresponding downsampling network in the second motion fusion network. The motion information is then passed through a decoupled detection network to achieve detection of moving targets.

[0040] Figure 1 A diagram showing an application scenario of a moving target detection method based on multiple frame images provided according to an embodiment of the present invention is shown.

[0041] like Figure 1 As shown in the figure, compared with the moving target detection task under a fixed base, the difficulties of the moving target detection task based on multi-frame images are mainly:

[0042] (1) Motion imaging: UAV mobile monitoring, airborne imaging detection, etc. are all moving platform imaging. Moving platform imaging will make the background no longer remain constant in space and time. The moving background will produce false motion. Reasonable distinction between background motion and target motion is a key issue that needs to be solved in moving target detection under moving platform imaging;

[0043] (2) Complex background: There are a large number of objects in aerial remote sensing images that have the same or similar features as moving targets, and the shooting of aerial images is easily affected by weather such as clouds and fog. Therefore, it is necessary to consider the impact of complex meteorological conditions on aerial images;

[0044] (3) Dynamic background: In addition to the main moving targets we are concerned about, some natural phenomena can also become interference backgrounds for moving targets, including trees blown by the wind, wavy water surfaces, moving white clouds, etc. Dynamic backgrounds need to be excluded from natural phenomena according to application requirements;

[0045] (4) Shadow motion: Shadows are usually treated as moving targets and detected by conventional moving target detection technology. Therefore, it is necessary to separate the shadow and the target during the detection process of moving targets.

[0046] (5) Real-time detection: Most of the detection of moving targets requires timely processing of image data. Most of the existing methods with good robustness and detection effect are difficult to achieve real-time image processing.

[0047] Figure 2 The following is a flow chart of a method for detecting a moving object based on multiple frames of images according to an embodiment of the present invention; Figure 3 The network structure of the moving target detection method based on multiple frame images provided according to an embodiment of the present invention is shown.

[0048] like Figure 2-Figure 3 As shown, the moving target detection method based on multi-frame images proposed in the present invention specifically includes the following steps:

[0049] S1: Input the aerial remote sensing image sequence and normalize the aerial remote sensing image sequence to obtain the impact data set to be trained.

[0050] Step S1 specifically includes the following steps:

[0051] S11: Crop the input size of all images in the aerial remote sensing image sequence to 608 608 pixels.

[0052] S12: Pair the k-th frame image with the kn-th frame image to obtain an impact data set to be trained.

[0053] In step S12, when the moving distance of the moving target between two adjacent frame images is large, n in the kn-th frame image is set to n<5; when the moving distance of the moving target between two adjacent frame images is small, n in the kn-th frame image is set to n≥5.

[0054] S2: Input the kth frame image into the first basic semantic feature extraction network to obtain a first shallow feature map, and input the knth frame image into the second basic semantic feature extraction network to obtain a second shallow feature map, input the first shallow feature map and the second shallow feature map into the first motion fusion network for feature extraction and fusion to obtain a motion fusion feature map containing motion information.

[0055] The first basic semantic feature extraction network and the second basic semantic feature extraction network both include a Conv-1 network layer and a Res-2 network layer, and the first motion fusion network includes a first MEF module, a second MEF module, a 1 1Ordinary convolution layer and downsampling module.

[0056] Step S2 specifically includes the following steps:

[0057] S21: Input the k-th frame image and the kn-th frame image into the Conv-1 network layer of the first basic semantic feature extraction network and the second basic semantic feature extraction network respectively, and obtain the feature map P11 and the feature map P21 respectively. Perform motion information feature extraction and fusion on the feature map P11 and the feature map P21 through the first MEF module to obtain the feature map P1.

[0058] S22: The feature map P11 and the feature map P21 are correspondingly transmitted to the Res-2 network layer in the first basic semantic feature extraction network and the second basic semantic feature extraction network to obtain the first shallow feature map and the second shallow feature map respectively. The motion information feature of the first shallow feature map and the second shallow feature map are extracted and fused by the second MEF module to obtain the feature map P2.

[0059] S23: The feature map P1 is downsampled by the downsampling module and then added to the feature map P2. 1. Ordinary convolution layer performs feature integration to obtain motion fusion feature map.

[0060] The weights of the k-th frame image and the kn-th frame image are shared.

[0061] Figure 4 The figure shows the network structure of the MFE module provided according to an embodiment of the present invention.

[0062] like Figure 4 As shown, the first MEF module and the second MEF module include a first 1 1 ordinary convolution layer, the second 1 1 ordinary convolution layer, the third 1 1 ordinary convolution layer, first 3 3 ordinary convolutional layers, FDE modules and Prewitt modules, wherein the feature map P11 or the first shallow feature map passes through the first 1 1 ordinary convolution layer and then input to FDE module and Prewitt module respectively, while the feature map P21 or the second shallow feature map passes through the second 1 1 normal convolution layer and then the first 3 3. Ordinary convolution layer performs background compensation to obtain compensation feature map. After inputting the compensation feature map into FDE module, the temporal and spatial difference compensation feature is obtained. After the temporal and spatial difference compensation feature and the spatial texture feature map output by Prewitt module are subjected to Cat operation, the third 1 1 Ordinary convolution layer completes the feature extraction and fusion of motion information.

[0063] Inspired by the brightness constant assumption in the classic optical flow method (i.e., the brightness of the same target does not change when it moves between different image frames), the calculation idea applied to image optical flow is applied to deep features relative to shallow features. Suppose the pixel point in the first frame is at Light intensity at all times , after a certain time , the pixel moves to , that is, the light intensity is , based on the constant brightness assumption, we can get:

[0064] (1);

[0065] Taylor expansion of the right side of equation (1) yields:

[0066] (2);

[0067] Substituting equation (1) into equation (2) and ignoring the high-order term ε, we can get:

[0068] (3);

[0069] Divide both sides of formula (3) by We can get:

[0070] (4);

[0071] in, At the pixel The light flow at 、 、 Represents pixel points The spatial gradient in the x direction, the spatial gradient in the y direction and the temporal gradient at . The traditional edge detection operator Prewitt operator is used as the convolution kernel to calculate the spatial gradient of the depth features in the x and y directions of each channel of its depth features.

[0072] The template of the Prewitt module operator is:

[0073] (5);

[0074] in, is the operator in the x direction in the prewitt operator, is the operator in the y direction in the prewitt operator.

[0075] Compared with the traditional moving target detection method, which directly uses the difference of image frames as an indicator of motion information between images, the present invention strengthens the difference between feature maps by the following formula, and squares each channel of the feature difference of the feature map and sums them to obtain the enhanced feature difference: , all feature differences are accumulated into one channel, and the spatial gradient splicing between feature maps is completed through formula (6).

[0076] The spatial gradient splicing formula of the FDE module is:

[0077] (6);

[0078] in, The first shallow feature map passes through the first 1 1The feature map obtained after ordinary convolution layer processing, is the compensation feature map, i is the current number of channels, n is the number of channels of the compensation feature map, It is the feature difference obtained by enhancing each channel of the compensation feature map.

[0079] S3: The motion fusion feature map is input into the high-level semantic feature extraction network and the downsampling network respectively, and three semantic feature maps and three downsampling feature maps are obtained accordingly. After the three semantic feature maps pass through the path aggregation network (PAN network), the information of the three downsampling feature maps is fused in the second motion fusion network to obtain three enhanced feature maps containing motion information and semantic features.

[0080] The high-level semantic feature extraction network includes Res-3 network layer, Res-4 network layer and Res-5 network layer, which are connected in sequence; the downsampling network includes Down-1 network layer, Down-2 network layer and Down-3 network layer, which are connected in sequence.

[0081] Step S3 specifically includes the following steps:

[0082] S31: Add the motion fusion feature map to the first shallow feature map and pass them through the Res-3 network layer, the Res-4 network layer and the Res-5 network layer in sequence to obtain feature maps P13, P14 and P15 respectively. At the same time, pass the motion fusion feature map through the Down-1 network layer, the Down-2 network layer and the Down-3 network layer in sequence to obtain the first motion information T1, the second motion information T2 and the third motion information T3 respectively.

[0083] S32: After the feature map P13, the feature map P14 and the feature map P15 pass through the path aggregation network respectively, they are added to the first motion information T1, the second motion information T2 and the third motion information T3 in the second motion fusion network to obtain three enhanced feature maps.

[0084] S4: Input the three enhanced feature maps obtained in step S3 into the decoupling detection network to obtain the final network output.

[0085] Figure 5 The network structure of the decoupling detection network provided according to an embodiment of the present invention is shown.

[0086] like Figure 5 As shown, the decoupling detection network includes a fourth 1 1 ordinary convolution layer, the first branch and the second branch, the first branch and the second branch are connected in parallel, wherein the first branch is connected in sequence to the second 3 3 ordinary convolutional layers, the third 3 3 ordinary convolutional layers, the fifth 1 1 ordinary convolution layer and the first Feature convolution layer, the second branch is the fourth 3 3 ordinary convolutional layers, the fifth 3 3 ordinary convolutional layers, sixth 1 1 ordinary convolution layer, seventh 1 1 ordinary convolution layer, the second Feature convolution layer and the third Feature convolution layer, and the sixth 1 1 The branch where the ordinary convolution layer and the second Feature convolution layer are located is the same as the seventh 1 1 The branch where the ordinary convolution layer and the third feature convolution layer are located is connected in parallel, and the three enhanced feature maps pass through the fourth 1 1 After the ordinary convolution layer, the network is input to the first branch and the second branch for decoupling.

[0087] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in the present disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in the present disclosure can be achieved. This is not limited herein.

[0088] The above specific embodiments do not limit the scope of protection of the present invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention are intended to be included within the scope of protection of the present invention.

Claims

1. A moving target detection method based on multi-frame images, characterized in that: The specific steps include: S1: inputting an aerial remote sensing image sequence, and normalizing the aerial remote sensing image sequence to obtain an impact data set to be trained; S2: inputting the k-th frame image into the first basic semantic feature extraction network to obtain a first shallow feature map, and inputting the kn-th frame image into the second basic semantic feature extraction network to obtain a second shallow feature map, and inputting the first shallow feature map and the second shallow feature map into the first motion fusion network for feature extraction and fusion to obtain a motion fusion feature map containing motion information; S3: inputting the motion fusion feature map into the high-level semantic feature extraction network and the downsampling network respectively, and obtaining three semantic feature maps and three downsampling feature maps accordingly; after the three semantic feature maps pass through the path aggregation network, information fusion is performed on the three downsampling feature maps in the second motion fusion network to obtain three enhanced feature maps containing motion information and semantic features; S4: Input the three enhanced feature maps obtained in step S3 into the decoupled detection network to obtain the final network output.

2. The moving target detection method based on multi-frame images according to claim 1 is characterized in that: The step S1 specifically includes the following steps: S11: cropping the input size of all images in the aerial remote sensing image sequence to 608 608 pixels; S12: Pair the k-th frame image with the kn-th frame image to obtain an impact data set to be trained.

3. The moving target detection method based on multi-frame images according to claim 2 is characterized in that: In step S12, when the moving distance of the moving target between two adjacent frame images is large, n in the kn-th frame image is set to n<5; when the moving distance of the moving target between two adjacent frame images is small, n in the kn-th frame image is set to n≥5.

4. The moving target detection method based on multi-frame images according to claim 1, characterized in that: The first basic semantic feature extraction network and the second basic semantic feature extraction network both include a Conv-1 network layer and a Res-2 network layer, and the first motion fusion network includes a first MEF module, a second MEF module, a 1 1 Ordinary convolution layer and downsampling module.

5. The moving target detection method based on multi-frame images according to claim 4 is characterized in that: The step S2 specifically includes the following steps: S21: input the k-th frame image and the kn-th frame image into the Conv-1 network layer of the first basic semantic feature extraction network and the second basic semantic feature extraction network respectively, and obtain feature maps P11 and P21 respectively, and extract and fuse the motion information features of the feature maps P11 and P21 through the first MEF module to obtain feature map P1; S22: The feature map P11 and the feature map P21 are transmitted to the Res-2 network layer in the first basic semantic feature extraction network and the second basic semantic feature extraction network respectively, and the first shallow feature map and the second shallow feature map are obtained respectively, and the motion information feature extraction and fusion are performed on the first shallow feature map and the second shallow feature map by the second MEF module to obtain the feature map P2; S23: downsampling the feature map P1 through the downsampling module and adding it to the feature map P2.

1. The ordinary convolution layer performs feature integration to obtain the motion fusion feature map.

6. The moving target detection method based on multi-frame images according to claim 5 is characterized in that: The first MEF module and the second MEF module include a first 1 1 normal convolution layer, the second 1 1 normal convolution layer, the third 1 1 normal convolution layer, first 3 3 ordinary convolutional layers, FDE modules and Prewitt modules, wherein the feature map P11 or the first shallow feature map is processed by the first 1 1 ordinary convolution layer and then input into the FDE module and the Prewitt module respectively, and at the same time, the feature map P21 or the second shallow feature map passes through the second 1 1 Ordinary convolution layer and then the first 3 3. The ordinary convolution layer performs background compensation to obtain a compensation feature map, and after the compensation feature map is input into the FDE module, a temporal and spatial difference compensation feature is obtained. After the temporal and spatial difference compensation feature and the spatial texture feature map output by the Prewitt module are subjected to Cat operation, the third 1 1 Ordinary convolution layer completes the feature extraction and fusion of motion information.

7. The moving target detection method based on multi-frame images according to claim 6 is characterized in that: The spatial gradient splicing formula of the FDE module is: ； in, The first shallow feature map is obtained by the first 1 1The feature map obtained after ordinary convolution layer processing, is the compensation feature map, i is the current number of channels, n is the number of channels of the compensation feature map, The characteristic difference is obtained by enhancing each channel of the compensation feature map.

8. The moving target detection method based on multi-frame images according to claim 1, characterized in that: The high-level semantic feature extraction network includes a Res-3 network layer, a Res-4 network layer and a Res-5 network layer, and the Res-3 network layer, the Res-4 network layer and the Res-5 network layer are connected in sequence; the downsampling network includes a Down-1 network layer, a Down-2 network layer and a Down-3 network layer, and the Down-1 network layer, the Down-2 network layer and the Down-3 network layer are connected in sequence.

9. The moving target detection method based on multi-frame images according to claim 8, characterized in that: The step S3 specifically includes the following steps: S31: adding the motion fusion feature map to the first shallow feature map and sequentially passing through the Res-3 network layer, the Res-4 network layer and the Res-5 network layer, and correspondingly obtaining feature maps P13, P14 and P15; and simultaneously sequentially passing the motion fusion feature map through the Down-1 network layer, the Down-2 network layer and the Down-3 network layer, and correspondingly obtaining first motion information T1, second motion information T2 and third motion information T3; S32: After the feature map P13, the feature map P14 and the feature map P15 pass through the path aggregation network respectively, they are added to the first motion information T1, the second motion information T2 and the third motion information T3 in the second motion fusion network to obtain three enhanced feature maps.

10. The moving target detection method based on multi-frame images according to claim 1, characterized in that: The decoupling detection network includes a fourth 1 1 ordinary convolution layer, a first branch and a second branch, the first branch and the second branch are connected in parallel, wherein the first branch is connected in sequence to the second 3 3 normal convolutional layers, the third 3 3 ordinary convolutional layers, fifth 1 1 ordinary convolution layer and the first Feature convolution layer, the second branch is composed of the fourth 3 3 ordinary convolutional layers, fifth 3 3 ordinary convolutional layers, sixth 1 1 ordinary convolution layer, seventh 1 1 ordinary convolution layer, the second Feature convolution layer and the third Feature convolution layer, and the sixth 1 1 The branch where the common convolution layer and the second Feature convolution layer are located is connected to the seventh 1 1 The ordinary convolution layer and the branch where the third Feature convolution layer is located are connected in parallel, and the three enhanced feature maps are sequentially passed through the fourth 1 1 After the common convolution layer, the first branch and the second branch are respectively inputted for decoupling.

Images