A moving target detection method based on multi-scale dilated convolutional coding-decoding

Abstract

This invention discloses a moving target detection method based on multi-scale dilated convolutional encoding-decoding, comprising: 1. constructing a multi-information video sequence set; 2. building an encoding-decoding model with multi-scale dilated convolution; 3. training the network model; and 4. testing the network model. This invention solves the problems of traditional encoder-decoder methods, such as the inability to transmit deeper detailed features and the limitation to detecting only simple scenes. It enables rapid and accurate extraction of foreground targets, especially small objects affected by background information, in complex real-world scenes, thereby improving the detection capability for foreground targets.

Description

Technical Field

[0001] This invention belongs to the field of computer vision technology, specifically a moving target detection method based on multi-scale dilated convolutional coding-decoding. Background Technology

[0002] Moving object detection technology is a hot topic in the field of computer vision, playing a vital role in many areas such as human motion analysis, anomaly detection, human-computer interaction, and robot navigation. Due to the influence of factors such as lighting changes, shadows, and dynamic backgrounds on videos, detecting foreground objects with localized changes in a video scene has always been a challenging task. One widely used method for extracting foreground objects is background subtraction.

[0003] Traditional algorithms are unsupervised and rely on background modeling to distinguish moving objects. Therefore, they are easily affected by scene factors, can only handle scenes with simple backgrounds, and show a high degree of scene dependence. They have not yet reached the standard for detecting unseen videos that are unrelated to the training video.

[0004] Recently, deep learning-based algorithms have demonstrated excellent scene learning capabilities, with detection accuracy far exceeding that of traditional moving target detection methods. Among them, the encoder-decoder structure is the most widely used, but this method is prone to information loss during the secondary sampling process and performance degradation for small objects due to the low receptive field. Summary of the Invention

[0005] This invention aims to address the shortcomings of existing technologies by proposing a moving target detection method based on multi-scale dilated convolutional encoding-decoding. This method aims to solve the problems of traditional encoder-decoder systems, such as the inability to transmit detailed features to deeper levels and the limitation of detecting only simple scenes. In this way, it can quickly and accurately extract foreground targets, especially small targets affected by background information, in complex real-world scenes, thereby improving the detection capability of foreground targets.

[0006] To achieve the above-mentioned objectives, the present invention adopts the following technical solution:

[0007] The moving target detection method based on multi-scale dilated convolutional coding-decoding of this invention is characterized by the following steps:

[0008] Step 1: Construct a multi-information video sequence set;

[0009] Step 1.1: From the video data with pixel-level labels, select T video sequences and normalize each frame in the video sequences to obtain a normalized set of video sequences S = {S1, S2, ..., S...} t ,...,S T}, St Let t be the video sequence of the t-th segment, and f i t Let S be the t-th video sequence. t The i-th frame in the sequence is given by M, where M is the number of image frames in each video sequence; the true pixel-level label of the t-th video sequence is defined as Y. t ∈{0,1};

[0010] Step 1.2: Use time median filtering to respectively... The images of the first X and first Y frames are processed to obtain the t-th "empty" video sequence S′. t And the most recent video sequence S″ of segment t t ;1 <X<Y<M;

[0011] Step 1.3: Using the pre-trained HRnet network to process S... t ,S′ t ,S″ t After processing, the original semantic information video sequence F of segment t is obtained. t The video sequence F containing the semantic information of the "empty" segment t. t The most recent semantic information video sequence F of segment t. t ";

[0012] Step 1.4: Construct the t-th multi-information video sequence C t ={S t ,S′ t ,S″ t ,F t ,F′ t ,F″ t};

[0013] Step 2: Construct a multi-scale dilated convolutional encoder-decoder network, including: a multi-scale dilated convolutional encoder module, a feature enhancement and detail processing module, and a feature decoding module;

[0014] Step 2.1: The multi-scale dilated convolutional coding module is constructed by concatenating N coding blocks. Each coding block consists of a dilated convolutional block and a convolutional block connected together. The dilated convolutional block includes: a parallel convolutional layers with different dilation rates; the outputs of the a convolutional layers are summed and processed by a Dropout layer before being input into subsequent convolutional blocks; the convolutional block is composed of a convolutional layer, a BN layer, and a ReLU layer connected in series.

[0015] The t-th segment of the multi-information video sequence C t The input is processed in a multi-scale dilated convolutional coding module to obtain the t-th segment of the multi-scale coded feature sequence O from the output of N coding blocks. t ={O t1 O t 2 ,...,O t n ,...,O t N}; where O t n This represents the multi-scale coded features output by the nth coded block;

[0016] Step 2.2: The feature enhancement and detail processing module is constructed by N feature processing blocks connected in parallel; each feature processing block is composed of a parallel attention module and a detail processing module connected in parallel; wherein, the parallel attention module is composed of a position attention unit and a channel attention unit connected in parallel.

[0017] Step 2.2.1: The parallel attention module in the nth feature processing block is positioned at the attention unit pair O. t n The process is performed to obtain the fusion feature E at the nth position. t n ;

[0018] Step 2.2.2: The nth channel attention unit pairs with O t n The process is performed to obtain the fusion feature E of the nth channel. t ′ n ;

[0019] Step 2.2.3, E t n With E t ′ n After linear addition, the input is processed by a convolutional layer to obtain the nth position-channel fusion feature H output by the nth parallel attention module. t n ;

[0020] Step 2.2.4: The detailed processing module processes O. t n The nth detail processing feature SC obtained by processing t n ;

[0021] Step 2.2.5, nth position-channel fusion feature H t n With the nth detail processing feature SC t n The summation yields the nth information enhancement feature (HSC) output by the nth feature processing block. t n Thus, the t-th information-enhanced feature sequence HSC output by the feature enhancement and detail processing module is obtained.t ={HSC t 1 HSC t 2 ,...,HSC t n ,...,HSC t N};

[0022] Step 2.3: The feature decoding module consists of N deconvolution modules and a Sigmoid layer. Each deconvolution module is composed of a deconvolution layer, a BN layer, a convolution layer, a BN layer, and a ReLU layer connected in series.

[0023] Enhance the Nth information feature HSC t N After being processed in the Nth deconvolution module, the result obtained is similar to that of HSC. t N-1 Linear addition is performed, and the result is then fed into the (N-1)th deconvolution module for processing. The resulting value is then compared with the HSC. t N-2 Linear addition is performed, and so on. The final sum is then fed into the first deconvolution module for processing. The result is then fed into the Sigmoid layer for further processing, thus obtaining the t-th multi-information video sequence C. t Predicted pixel-wise foreground probability

[0024] Step 3: Train the network:

[0025] Step 3.1: Use equation (1) to establish the predicted prospect probability. With pixel-level label Y t The t-th loss between

[0026]

[0027] In equation (1), e is the smoothing parameter, and m and n are the spatial pixel positions;

[0028] Step 3.2, based on T-segment multi-information video sequence {C t |t=1,2,…,T}, using the Adam optimizer to calculate T loss values. Backpropagation is performed to the multi-scale dilated convolutional encoder-decoder network, and the network parameters are continuously updated until the loss function tends to converge, resulting in the trained multi-scale dilated convolutional encoder-decoder model.

[0029] Step 4: Use the trained multi-scale dilated convolutional encoder-decoder model to process the intermittent object motion image to be predicted, and obtain the pixel-wise foreground probability corresponding to the intermittent object motion image.

[0030] A threshold P is set, and the foreground probability of each pixel is compared with the threshold P. Pixels with a probability greater than the threshold P are set as foreground pixels, and the rest are set as background pixels. This yields the segmentation result of the moving target in the image of intermittent object motion to be predicted.

[0031] The moving target detection method based on multi-scale dilated convolutional coding-decoding described in this invention is also characterized in that step 2.2.1 includes:

[0032] The nth multi-scale encoded feature O t n The input is fed into the nth parallel attention module corresponding to the nth feature processing block, and then the nth positional attention unit uses pooling with scales l1, l2, ..., l r ,…,l L The adaptive pooling layer and one convolutional layer respectively affect O t n After processing, L pooling features R are obtained. t 1 ,R t 2 ,...,R t r ,...,R t L and dimensionality reduction feature F0; where, l r R represents the pooling scale of the r-th adaptive pooling layer. t r O t n The pooling features obtained after the r-th adaptive pooling layer; then R t 1 ,R t 2 ,...,R t r ,...,R t L After cascading, the nth cluster center is obtained. Where C represents the number of channels at the nth aggregation center, and J represents the sum of the pooling scales of the L adaptive pooling layers, J = l1 + l2 + ... + l r +…+l L ;

[0033] The nth cluster center W t nAfter being processed by the fully connected layer in the nth position attention unit, the input is multiplied by the dimensionality-reduced feature F0 and then fed into the Softmax layer for further processing to obtain the nth position attention map S. t n ; will S t n With multi-scale coding features O t n After multiplication, we obtain the nth position fusion feature E output by the nth parallel attention module. t n .

[0034] Step 2.2.2 includes:

[0035] The nth multi-scale encoded feature O t n The input is fed into the nth parallel attention module corresponding to the nth feature processing block, and the nth channel attention unit uses a convolutional layer to process the O... t n After dimensionality reduction, then with O t n Multiplying the transposes of the products and then processing the product through a Softmax layer, we obtain the attention map U for the nth channel. t n ;Change U t n With multi-scale coding features O t n After multiplication, the fused feature E′ of the nth channel is obtained from the output of the nth channel attention unit. t n .

[0036] Step 2.2.4 includes:

[0037] The detail processing module consists of an edge detail branch and a context information branch connected in parallel. The nth edge detail branch in the detail processing module corresponding to the nth feature processing block uses the Sobel operator to process the nth multi-scale encoded feature O. t n After processing in both the horizontal and vertical directions, the two results are linearly added together and then fed into a convolutional layer for further processing to obtain the nth edge feature SB. t n ;

[0038] The context information branch in the detail processing module corresponding to the nth feature processing block will encode the nth multi-scale feature O. t nAfter being processed by the convolutional layer and the softmax layer in sequence, the result is then processed by another convolutional layer and the LeakyReLU activation function layer to obtain the nth context feature CB. t n ;

[0039] SB by channel dimension t n and CB t n After concatenation, the data is input into a convolutional layer for processing, yielding the nth detail feature (SC) output by the nth detail processing module. t n ;

[0040] The present invention provides an electronic device, comprising a memory and a processor, wherein the memory is used to store a program that supports the processor in executing the moving target detection method, and the processor is configured to execute the program stored in the memory.

[0041] The present invention discloses a computer-readable storage medium on which a computer program is stored, wherein the computer program is executed by a processor to perform the steps of the moving target detection method.

[0042] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0043] 1. This invention solves the problem of information loss during secondary sampling by proposing to replace traditional convolution with dilated convolution as the encoder convolutional layer, expands the receptive field of the feature map, and by fusing information from these different scales, the model can learn comprehensive features and improve detection performance.

[0044] 2. This invention designs a set of multi-branch hybrid dilated convolutional blocks with multiple different dilation values, which avoids the grid effect of dilated convolution, maintains the continuity and relevance of information, and improves the performance degradation of deep learning networks for small objects caused by low receptive field.

[0045] 3. The novel background subtraction model proposed in this invention can be easily deployed in unseen scenes, overcoming the problems that traditional algorithms can only handle simple scenes and existing deep learning models cannot be extended to unseen scenes. It solves the diversity and variability of real application scenarios and brings significant improvements to background subtraction methods. Attached Figure Description

[0046] Figure 1 This is a schematic diagram of the method flow of the present invention;

[0047] Figure 2 This is a diagram of the network structure based on multi-scale dilated convolutional encoding-decoding of the present invention;

[0048] Figure 3a This invention uses partial frame images from a portion of video sequences within the test set;

[0049] Figure 3b This invention uses ground truth images corresponding to some frames in a portion of a video sequence within the test set.

[0050] Figure 3c The image shown is a semantic information map corresponding to some frame images in some video sequences in the test set using the pre-trained HRnet network of this invention.

[0051] Figure 3d The image shown is a binarized foreground segmentation map obtained by using a multi-scale dilated convolutional encoder-decoder network according to the present invention. Detailed Implementation

[0052] In this embodiment, a moving target detection method based on multi-scale dilated convolutional coding-decoding first uses a pre-trained HRnet network to construct a multi-information video sequence set as input. Then, it fuses multi-scale contextual information through a multi-scale dilated convolutional coding module and a feature enhancement and detail module to capture foreground information. Finally, it obtains a foreground segmentation map of the moving target through a feature decoding module. Figure 1 As shown, the specific steps are as follows:

[0053] Step 1: Construct a multi-information video sequence set;

[0054] Step 1.1: From the video data with pixel-level labels, select T video sequences and normalize each frame in the video sequences to obtain a normalized set of video sequences S = {S1, S2, ..., S...} t ,...,S T}, S t Let t be the video sequence of the t-th segment, and f i t Let S be the t-th video sequence. t The i-th frame in the sequence is given by M, where M is the number of image frames in each video sequence; the true pixel-level label of the t-th video sequence is defined as Y. t ∈{0,1};

[0055] In this embodiment, the video sequence T is set to 49, and the number M of image frames in each video sequence is taken as 1000, but it is not limited to this value. The dataset CDnet-2014 is used as the training set and the test set, which contains 10 categories and 49 video scenes with various challenges, including: dynamic background, shadow, bad weather, low frame rate, intermittent object motion, turbulence, etc.; by extracting one video from each category in the dataset to create a test set containing only unseen videos, and the remaining videos are used for training.

[0056] Step 1.2: Use temporal median filtering to process the first X frames and the first Y frames of the image respectively to obtain the t-th "empty" video sequence S' t and the t-th nearest video sequence S'' t ; 1 < X < Y < M; in a specific implementation, X = 50 and Y = 100, but it is not limited to this value.

[0057] Step 1.3: Use the pre-trained HRnet network to process S t , S' t , S'' t respectively, and correspondingly obtain the t-th original semantic information video sequence F t , the t-th "empty" semantic information video sequence F t ', and the t-th nearest semantic information video sequence F t '';

[0058] In this embodiment, the HRnet network is pre-trained on the ADE 20K dataset, where the categories C ∈ {c0, c1,..., c 149} in the dataset. Twelve categories such as people, cars, and trucks in the dataset are used as foregrounds, and the rest are used as backgrounds. Since the HRnet network provides the per-pixel probability p j ∈ C, let I[m, n] be the input frame at the spatial position m, n, c be the predicted probability distribution of I[m, n], and the respective semantic information video sequences can be obtained by calculating ; where Q represents the set of 12 foreground categories.

[0059] Step 1.4: Construct the t-th multi-information video sequence C t = {S t , S' t , S'' t , F t , F' t , F'' t};

[0060] Step 2: Construct a multi-scale dilated convolutional encoder-decoder network, including: a multi-scale dilated convolutional encoder module, a feature enhancement and detail processing module, and a feature decoding module;

[0061] Step 2.1, as follows Figure 2 As shown on the left, the multi-scale dilated convolutional coding module is constructed by concatenating N coding blocks. Each coding block consists of a dilated convolutional block and a convolutional block connected together. The dilated convolutional block includes: a parallel convolutional layers with different dilation rates; the outputs of the a convolutional layers are added together, processed by the Dropout layer, and then input into the subsequent convolutional blocks; the convolutional block is composed of convolutional layers, BN layers, and ReLU layers connected in sequence.

[0062] The t-th segment of the multi-information video sequence C t The input is processed in a multi-scale dilated convolutional coding module to obtain the t-th segment of the multi-scale coded feature sequence O from the output of N coding blocks. t ={O t 1 O t 2 ,...,O t n ,...,O t N}; where O t n This represents the multi-scale coded features output by the nth coded block;

[0063] In this embodiment, N=3, where the kernel size of the four parallel convolutional layers in the dilated convolution block is 3×3, with dilation rates of 1, 3, 5, and 7, respectively, resulting in actual receptive field sizes of 3×3, 7×7, 11×11, and 15×15. This solves the information loss problem during the secondary sampling process. Dilated convolution inserts zeros between standard convolutional kernel parameters to expand the kernel size and increase the receptive field without increasing the number of network parameters. The convolutional block uses convolutional layers with a kernel size of 1×1 and a stride of s=2.

[0064] Step 2.2, as follows Figure 2 As shown in the middle section, the feature enhancement and detail processing module is constructed by N feature processing blocks connected in parallel; each feature processing block is constructed by a parallel attention module and a detail processing module connected in parallel; the parallel attention module is constructed by a position attention unit and a channel attention unit connected in parallel, and the detail processing module is constructed by an edge detail branch and a context information branch connected in parallel.

[0065] Step 2.2.1, Processing of the position attention unit:

[0066] The nth multi-scale encoded feature O t nThe input is fed into the nth parallel attention module corresponding to the nth feature processing block, and then the nth positional attention unit uses pooling with scales l1, l2, ..., l r ,…,l L The adaptive pooling layer and one convolutional layer respectively affect O t n After processing, L pooling features R are obtained. t 1 ,R t 2 ,...,R t r ,...,R t L and dimensionality reduction feature F0; where, l r R represents the pooling scale of the r-th adaptive pooling layer. t r O t n The pooling features obtained after the r-th adaptive pooling layer; then R t 1 ,R t 2 ,...,R t r ,...,R t L After cascading, the nth cluster center is obtained. Where C represents the number of channels at the nth aggregation center, and J represents the sum of the pooling scales of the L adaptive pooling layers, J = l1 + l2 + ... + l r +…+l L ;

[0067] The nth cluster center W t n After being processed by the fully connected layer in the nth position attention unit, the input is multiplied by the dimensionality-reduced feature F0 and then fed into the Softmax layer for further processing to obtain the nth position attention map S. t n ; will S t n With multi-scale coding features O t n After multiplication, we obtain the nth position fusion feature E output by the nth parallel attention module. t n ;

[0068] In this embodiment, L is set to 4, l1, l2, l3, and l4 are set to 1, 2, 3, and 6 respectively, the kernel size of the convolutional layer is 1×1, and the stride is 1. Multi-scale clustering centers with different contexts are captured through multi-pooling operations, and the spatial perception capability of each spatial pixel and multi-scale clustering center is enhanced by using relational awareness of each center in the multi-scale clustering centers for weighting.

[0069] Step 2.2.2, Processing of Channel Attention Units:

[0070] The nth multi-scale encoded feature O t n The input is fed into the nth parallel attention module corresponding to the nth feature processing block, and the nth channel attention unit uses a convolutional layer to process the O... t n After dimensionality reduction, then with O t n Multiplying the transposes of the products and then processing the product through a Softmax layer, we obtain the attention map U for the nth channel. t n ;Change U t n With multi-scale coding features O t n After multiplication, the fused feature E of the nth channel is obtained from the output of the nth channel attention unit. t ′ n ;

[0071] In this embodiment, the convolutional layer kernel size is 1×1, and the stride is 1. By utilizing the spatial information of all corresponding locations to model channel correlations, a relationship is established between each input channel and the channel aggregation center, thereby enhancing the network's spatial information understanding capability.

[0072] Step 2.2.3, E t n With E t ′ n After linear addition, the input is processed by a convolutional layer to obtain the nth position-channel fusion feature H output by the nth parallel attention module. t n ;

[0073] In this embodiment, the kernel size of the convolutional layer is 3×3, and the stride is 1. An attention mechanism is used to strengthen the connection between channel and spatial feature information, reducing the weight of background information. This achieves the goal of suppressing background information and focusing on foreground information, thereby improving the feature information extraction capability.

[0074] Step 2.2.4, Processing of details module:

[0075] The nth edge detail branch in the detail processing module corresponding to the nth feature processing block uses the Sobel operator to encode the nth multi-scale feature O. t n After processing in both the horizontal and vertical directions, the two results are linearly added together and then fed into a convolutional layer for further processing to obtain the nth edge feature SB. t n ;

[0076] The context information branch in the detail processing module corresponding to the nth feature processing block will encode the nth multi-scale feature O. t n After being processed by the convolutional layer and the softmax layer in sequence, the result is then processed by another convolutional layer and the LeakyReLU activation function layer to obtain the nth context feature CB. t n In practice, the convolutional layers used for the edge detail branch and the context information branch are both 1×1 kernels with a stride of 1.

[0077] Step 2.2.5: Sort SB by channel dimension t n and CB t n After concatenation, the data is input into a convolutional layer for processing, yielding the nth detail feature (SC) output by the nth detail processing module. t n The texture of the components is enhanced by using edge detail branches, and the context information branches enhance the global component information by capturing long-range dependencies. In the specific implementation, the kernel size of the convolutional layer is 1×1 and the stride is 1.

[0078] Step 2.2.6, nth position-channel fusion feature H t n With the nth detail processing feature SC t n The summation yields the nth information enhancement feature (HSC) output by the nth feature processing block. t n Thus, the t-th information-enhanced feature sequence HSC output by the feature enhancement and detail processing module is obtained. t ={HSC t 1 HSC t 2 ,...,HSC t n ,...,HSC t NBy using location-channel fusion features and detail-processing features, long-range spatial dependencies are captured while preserving accurate location information to enhance the representation of features of interest.

[0079] Step 2.3, as follows Figure 2 As shown on the right, the feature decoding module consists of N deconvolution modules and a Sigmoid layer. Each deconvolution module is composed of a deconvolution layer, a BN layer, a convolution layer, a BN layer, and a ReLU layer connected in series.

[0080] Enhance the Nth information feature HSC t N After being processed in the Nth deconvolution module, the result obtained is similar to that of HSC. t N-1 Linear addition is performed, and the result is fed into the (N-1)th deconvolution module for processing. The resulting value is then compared with the HSC value. t N-2 Linear addition is performed, and so on. The final sum is then fed into the first deconvolution module for processing. The result is then fed into the Sigmoid layer for further processing, resulting in the t-th multi-information video sequence C. t Predicted pixel-wise foreground probability In the specific implementation, N=3 is set. The kernel size in the deconvolution layer is 3×3 with a stride of 2, and the kernel size in the convolutional layer is 3×3 with a stride of 1. The input image resolution is restored step by step by using a feature decoding module.

[0081] Step 3: Train the network:

[0082] Step 3.1: To address the imbalance between the number of background and foreground pixels, the Jaccard exponent is used as the t-th loss function, and the predicted foreground probability is established using equation (1). With pixel-level label Y t Losses between

[0083]

[0084] In equation (1), e is the smoothing parameter, and m and n are the spatial pixel positions;

[0085] Step 3.2, based on T-segment multi-information video sequence {C t |t=1,2,...,T}, using the Adam optimizer to calculate T loss values. Backpropagation is performed to the multi-scale dilated convolutional encoder-decoder network, and the network parameters are continuously updated until the loss function is reached. The training process converges, resulting in a multi-scale dilated convolutional encoder-decoder model. Specifically, 16 samples are used in each training iteration. The learning rate of the Adam optimizer is set to 0.0001, and the betas are set to 0.9 and 0.999, respectively. When the training epochs reach 150, the loss function converges, at which point the optimal multi-scale dilated convolutional encoder-decoder model is saved.

[0086] Step 4: Use the trained multi-scale dilated convolutional encoder-decoder model to process the intermittent object motion image to be predicted, and obtain the pixel-wise foreground probability corresponding to the intermittent object motion image.

[0087] A threshold P is set, and the foreground probability of each pixel is compared with the threshold P. Pixels with a probability greater than the threshold P are set as foreground pixels, and the rest are set as background pixels. This yields the segmentation result of the moving target in the image of intermittent object motion to be predicted.

[0088] In this example, the threshold P is set to 0.5. Pixels with values ​​greater than 0.5 are designated as foreground pixels, and the remaining pixels are designated as background pixels. The resulting binarized foreground segmentation image of the test set is shown below. Figure 3a As shown in Table 1, the overall quantitative results of the test set are presented.

[0089] In this embodiment, an electronic device includes a memory and a processor. The memory stores a program that supports the processor in executing the above-described method, and the processor is configured to execute the program stored in the memory.

[0090] In this embodiment, a computer-readable storage medium stores a computer program, which is executed by a processor to perform the steps of the above method.

[0091] Table 1 shows the overall average Precision, Recall, and F-Measure results for different categories in the dataset according to this invention.

[0092] Epoch (Optimal Cycle) Precision Recall F-Measure 150 0.924 0.919 0.920

[0093] like Figure 3c As shown, using pre-trained HRnet networks to obtain semantic information from the dataset, along with video sequences as input, can enhance the network's focus on moving objects and reduce false detections caused by shadows, etc. Figure 3d The binarized foreground segmentation image shown is... Figure 3b By comparing the pixel-level labels shown, it can be seen that the multi-scale dilated convolutional encoder-decoder network proposed in this invention has good robustness and can adapt well to various challenges in the real world, especially for small target motion detection.

Claims

1. A method for moving object detection based on multi-scale dilated convolutional encoding-decoding, characterized in that, is performed according to the following steps: Step 1, constructing a multi-information video sequence set; Step 1.

1. Select T video sequences from the video data with pixel-level labels, and normalize each frame of image in the video sequence, thereby obtaining a normalized video sequence set S = {S1, S2,..., St,..., ST}, St represents the t-th video sequence, and t ,...,S T} represents the i-th frame of image in the t-th video sequence St, and M is the number of image frames in each video sequence; define the true pixel-level label of the t-th video sequence as Yt∈{0,1}. t f i t t t ​​​​ Step 1.2, respectively, the median filtering of time is used to process the middle X frames and the front Y frames of images to obtain the t-th segment of "empty" video sequence S' and the t-th segment of the nearest video sequence S" t and the t-th segment of the nearest video sequence S" t ; 1 < X < Y < M; Step 1.3, respectively, using the pre-trained HRnet network to process S t ,S′ t ,S″ t , respectively, to obtain the t-th original semantic information video sequence F t , the t-th "empty" semantic information video sequence F t ', and the t-th nearest semantic information video sequence F t ". Step 1.

4. Constructing the t-segment multi-information video sequence C t = {S t , S' t , S" t , F t , F t , F t "}; Step 2, building a multi-scale dilated convolutional encoding-decoding network, including: a multi-scale dilated convolutional encoding module, a feature enhancement and detail processing module, and a feature decoding module; Step 2.1, the multi-scale dilated convolutional encoding module is constructed by connecting N encoding blocks in series, and each encoding block is connected by an expanded convolution block and a convolution block; the expanded convolution block includes: a layer of parallel convolution layers with different expansion rates; the outputs of the a layer of convolution layers are added, then processed by a Dropout layer, and then input into the subsequent convolution block; the convolution block is connected in series by a convolution layer, a BN layer, and a ReLU layer; The t-th segment multi-information video sequence C t The t-th segment multi-information video sequence C t The t-th segment multi-information video sequence C t 1 The t-th segment multi-information video sequence C t 2 The t-th segment multi-information video sequence C t n The t-th segment multi-information video sequence C t N The t-th segment multi-information video sequence C t n The t-th segment multi-information video sequence C Step 2.2, the feature enhancement and detail processing module is constructed by connecting N feature processing blocks in parallel; each feature processing block is connected by a parallel attention module and a detail processing module; wherein the parallel attention module is connected by a position attention unit and a channel attention unit; Step 2.2.1, parallel attention module in the nth feature processing block Position attention unit O t n Processing to obtain the nth position fusion feature E t n ; Step 2.2.2, the n-th channel attention unit pair O t n processing to obtain the n-th channel fusion feature Step 2.2.3, E t n With The n-th position-channel fusion feature H of the n-th parallel attention module output is obtained by linearly adding and then inputting convolutional layer processing t n ; Step 2.2.4, the detail processing module processes O t n the n-th detail processing feature SC t n ; Step 2.2.5, n-th position-channel fused feature H t n with the n-th detail processing feature SC t n After addition, the n-th information enhancement feature HSC of the n-th feature processing block output is obtained t n ; thereby obtaining the t-th information enhancement feature sequence HSC output by the feature enhancement and detail processing module t = {HSC t 1 ,HSC t 2 ,...,HSC t n ,...,HSC t N} Step 2.3, the feature decoding module is composed of N deconvolution modules and a Sigmoid layer, and each deconvolution module is connected in series by a deconvolution layer, a BN layer, a convolution layer, a BN layer, and a ReLU layer; Send the Nth information enhancement feature HSC t N into the Nth deconvolution module for processing, and the result obtained is linearly added to HSC t N-1 Then, the result of the addition is sent into the (N - 1)th deconvolution module for processing, and the result obtained is linearly added to HSC t N-2 And so on. The result of the final addition is sent into the 1st deconvolution module for processing, and the result obtained is sent to the Sigmoid layer for processing, thereby obtaining the tth multi-information video sequence C t Predicted foreground probability per pixel Step 3, training the network: Step 3.1, establishing a prediction foreground probability with formula (1) between the pixel-level label Y t the tth loss between the pixel-level label Y In formula (1), e is a smoothing parameter, and m and n are spatial pixel positions; Step 3.2, based on T-segment multi-information video sequence {C t |t=1,2,...,T}, using the Adam optimizer to calculate T loss values. Backpropagation is performed to the multi-scale dilated convolutional encoder-decoder network, and the network parameters are continuously updated until the loss function tends to converge, resulting in the trained multi-scale dilated convolutional encoder-decoder model. Step 4, using the trained multi-scale dilated convolutional encoding-decoding model to process the intermittent object motion image to be predicted to obtain the pixel-by-pixel foreground probability corresponding to the intermittent object motion image; Set a threshold P, compare the pixel-by-pixel foreground probability of each pixel with the threshold P, so as to set the pixels greater than the threshold P as foreground pixels, and the remaining pixels as background pixels, and obtain the motion target segmentation result in the intermittent object motion image to be predicted. 2.The motion target detection method based on multi-scale dilated convolutional encoding-decoding according to claim 1, characterized in that, The step 2.2.1 includes: The nth multi-scale coding feature O t n is input into the nth parallel attention module corresponding to the nth feature processing block, and is processed by the nth position attention unit using adaptive pooling layers with pooling scales l1, l2, …, l r ,…,l L and a convolutional layer to obtain L pooled features R t n t 1 ,R t 2 ,...,R t r ,...,R t L and a reduced dimension feature F0; wherein l r represents the pooling scale of the rth adaptive pooling layer, R t r represents the pooled feature obtained after O t n passes through the rth adaptive pooling layer; and after R t 1 ,R t 2 ,...,R t r ,...,R t L are concatenated, the nth aggregation center Cn is obtained wherein C represents the number of channels of the nth aggregation center, J represents the sum of the pooling scales of the L adaptive pooling layers, J = l1 + l2 + … + l r +…+l L ;​ The nth aggregation center W t n After processing by the full connection layer in the nth position attention unit, multiplied by the dimension reduction feature F0, processed in the Softmax layer, and the nth position attention map S is obtained t n ; S t n After multiplication with the multi-scale coding feature O t n , the nth position fusion feature E output by the nth parallel attention module is obtained t n .

3. The method of claim 2, wherein the method is based on multi-scale dilated convolutional encoding-decoding for moving object detection. The step 2.2.2 includes: The nth multi-scale encoded feature O t n The input is fed into the nth parallel attention module corresponding to the nth feature processing block, and the nth channel attention unit uses a convolutional layer to process the O... t n After dimensionality reduction, then with O t n Multiplying the transposes of the products and then processing the product through a Softmax layer, we obtain the attention map U for the nth channel. t n ;Change U t n With multi-scale coding features O t n After multiplication, the fused feature of the nth channel is obtained from the output of the nth channel attention unit.

4. The method of claim 3, wherein the method is based on multi-scale dilated convolutional encoding-decoding for moving object detection. The step 2.2.4 includes: The detail processing module is formed by an edge detail branch and a context information branch in parallel, an nth edge detail branch in the detail processing module corresponding to an nth feature processing block uses a Sobel operator to calculate an nth multi-scale edge feature SB t n After processing in the horizontal and vertical directions, linear addition is performed on the two results, and the result is input into a convolution layer for processing, so as to obtain an nth edge feature SB t n ; The context information branch in the detail processing module corresponding to the nth feature processing block will the nth multi-scale coding feature O t n After the processing of the convolution layer and the Softmax layer in sequence, the obtained result is further processed by the convolution layer and the activation function LeakyReLU layer, thereby obtaining the nth context feature CB t n ; SB is input into the convolutional layer in the channel dimension t n and CB t n After splicing, the first n detail processing features SC are obtained by processing in the convolutional layer t n .

5. An electronic device comprising a memory and a processor, characterized in that The memory is used to store a program supporting the processor to execute the motion target detection method in any one of claims 1-4, and the processor is configured to execute the program stored in the memory.

6. A computer-readable storage medium having stored thereon a computer program, characterized in that The computer program is executed by the processor to perform the steps of the motion target detection method in any one of claims 1-4.

Images