An Unsupervised Moving Object Detection Method Based on Information Reduction Rate

Abstract

The present invention discloses an unsupervised moving object detection method based on the information reduction rate. The method is as follows: collect a video sequence through a camera and perform preprocessing to construct a database; calculate the optical flow image corresponding to the video sequence through the trained PWCNet and normalize it; use the video sequence and its corresponding optical flow image as inputs to train a generative adversarial network model; perform the same processing on the video sequence to be detected; extract the generator module of the trained generative adversarial network model to detect moving objects in the video sequence to be detected. Based on the property that the background image region does not contain the information of the foreground image region, a generative adversarial network model is constructed according to the relationship of optical flow to realize the discrimination between the background and the moving object. The generative adversarial network includes a generator and a restorer; the feature channels of the moving object are fused through an attention mechanism, reducing the interference of the background and improving the detection performance of the moving object.

Description

Technical Field

[0001] The present invention belongs to the field of deep learning in computer vision, and particularly relates to an unsupervised moving object detection method based on the information reduction rate. Background Art

[0002] Object detection is an important branch of computer vision. Its main purpose is to detect moving objects as foregrounds and extract them from video sequences, while separating the surrounding environment of the moving objects as the background from the moving objects. As an interdisciplinary comprehensive discipline, object detection integrates theories and algorithms in multiple fields such as image processing, machine learning, and optimization, and is the premise and foundation for completing higher-level image understanding tasks (such as object behavior recognition). Object detection technology has great research and application value and has been widely used in many fields such as intelligent video surveillance, intelligent human-computer interaction, intelligent transportation, visual navigation, driverless, unmanned autonomous flight, and battlefield situation reconnaissance. In recent years, with the continuous development of computer technology and deep learning technology, object detection models have also evolved continuously, and various detection models have emerged.

[0003] In the field of object detection, the average overlap rate IoU between the target object and the prediction result is often used as the core evaluation criterion. In recent years, the research on object detection can be divided into two categories: one is the supervised learning method; the other is the unsupervised learning method. The PDB algorithm is a typical supervised algorithm that uses a pyramid dilation convolution module to extract spatial features simultaneously at multiple scales and connects these spatial features to input into an extended DB-ConvLSTM structure to learn temporal information, obtaining good detection results. For unsupervised object detection algorithms, their greatest feature is that they do not require a large number of labeled samples and have great development space. The SAGE algorithm generates a spatio-temporal saliency map to estimate background and foreground information by calculating the geodesic distance between superpixels and edge pixels. However, this method mainly relies on the edge features and motion gradient features of images and is prone to generating noise regions in complex texture scenes. The CIS algorithm draws on the idea of the generative adversarial network and distinguishes the background from the moving object according to the defined information reduction rate based on the optical flow information, and can better detect the moving object. However, for existing unsupervised object tracking algorithms, in the presence of factors such as the angle, illumination, occlusion, background interference, and noise brought by the device during the shooting of the target image, the performance of many algorithms will decline. Summary of the Invention

[0004] The purpose of the present invention is to provide an unsupervised moving object detection method based on the information reduction rate, which fully utilizes the optical flow information of the target and the background, fuses the feature channels of the moving object through the attention mechanism, reduces the interference of the background, and improves the detection performance of the moving object.

[0005] The technical solution for achieving the object of the present invention is as follows: An unsupervised moving object detection method based on the information reduction rate, comprising the following steps:

[0006] Step 1: Collect a video sequence through a camera, perform preprocessing, and construct a database;

[0007] Step 2: Calculate the optical flow image corresponding to the video sequence through the trained PWCNet, and perform normalization;

[0008] Step 3: Use the video sequence and its corresponding optical flow image as inputs to train a generative adversarial network model;

[0009] Step 4: Perform the processing of Steps 1 to 2 on the video sequence to be detected as well;

[0010] Step 5: Extract the generator module of the trained generative adversarial network model to detect moving objects in the video sequence to be detected.

[0011] Compared with the prior art, the present invention has the following remarkable advantages: (1) Based on the property that the background image region does not contain the information of the foreground image region, a generative adversarial network model is constructed according to the relationship of optical flow to realize the discrimination between the background and the moving object. The generative adversarial network includes a generator and a restorer; introducing an attention mechanism effectively improves the robustness of the tracking algorithm and reduces the interference of background noise, etc. on the tracking of the object; (2) Make full use of the optical flow information of the object and the background, fuse the feature channels of the moving object through the attention mechanism, reduce the interference of the background, and improve the detection performance of the moving object. Description of the Drawings

[0012] Figure 1 It is a specific implementation flowchart of the present invention.

[0013] Figure 2 It is a basic network structure diagram of the present invention.

[0014] Figure 3 It is a detection output result diagram of the moving object in a partial video sequence by the generator module of the network model.

[0015] Figure 4 It is a network structure diagram of the generator module of the network model. Specific Embodiments

[0016] An unsupervised moving object detection method based on the information reduction rate of the present invention comprises the following steps:

[0017] Step 1: Collect a video sequence through a camera, perform preprocessing, and construct a database;

[0018] Step 2: Calculate the optical flow image corresponding to the video sequence through the trained PWCNet and normalize it;

[0019] Step 3: Use the video sequence and its corresponding optical flow image as inputs to train the generative adversarial network model;

[0020] Step 4: Perform the processing of Steps 1 to 2 on the video sequence to be detected as well;

[0021] Step 5: Extract the generator module of the trained generative adversarial network model to detect moving targets in the video sequence to be detected.

[0022] Furthermore, for the training of the generative adversarial network model in Step 3, the specific steps are as follows:

[0023] Step 3.1: Distinguishing between moving targets and the background:

[0024] Based on the principle that the background image region should not contain information of the foreground image region of moving targets, the region outside the region of interest can be learned to interpret the image in the region of interest as poorly as possible. Specifically, for a certain frame image I of the video sequence, assuming the image region is D, the image region of the moving target is Ω, and the background is Ω c = D / Ω, and its optical flow to the adjacent frame (the previous frame or the next frame) is u. Among them, the optical flow represents the apparent motion of the image brightness pattern and contains important information about the surface structure and dynamic behavior of the object. Use to represent the mutual information of two random variables. Given the optical flow u i , u j at positions i and j in the image I, formalize the concept of the foreground Ω as the region with 0 mutual information with the background:

[0025]

[0026] Among them, the mutual information represents the amount of information that the optical flow u j at position j in the given image I can provide about the optical flow u i at position i. The larger the value of this mutual information, the more information is provided; the Shannon information entropy H(u i |I) represents the uncertainty of u i . The greater the uncertainty of the variable, the greater the information entropy, and the value is always greater than 0; H(u i |u j ,I) represents the uncertainty of u j under the condition that u i is known;

[0027] Step 3.2: Loss function based on the information reduction rate:

[0028] Based on the above-defined foreground and background, combined with the Shannon information entropy theory, define the information reduction rate to construct the optimization objective; using two subsets in D, namely region x and region y, as inputs, and the optical flows of region x and region y are u x 、u y , define the information reduction rate γ as follows:

[0029]

[0030] where, represents the amount of information that the optical flow u y of region y in the given image I can provide about the optical flow u x of region x; the Shannon information entropy H(u x |I) represents the uncertainty of u x ; H(u x |u y ,I) represents the uncertainty of u y under the condition that u x is known;

[0031] γ(x|y;I) represents the amount by which the uncertainty of u y decreases under the condition that u x is known, and its value ranges from 0 to 1; when u x and u y are independent, that is, one belongs to the foreground and the other belongs to the background image region, γ = 0; the optical flow in the target image region Ω is represented by u in ={u i ,i∈Ω}, and the optical flow in the background region Ω c is represented by u out ={u j ,j∈Ω c}, so there is:

[0032]

[0033] where, P(u in |I) represents the probability that the optical flow is the foreground optical flow, and P(u in |u out ,I) represents the probability of u out under the condition that u in is known; according to the information reduction rate γ, define the loss function as When is the smallest, the optical flow of the background is sufficient to predict the foreground;

[0034] Make strict assumptions about the model as follows:

[0035]

[0036] Among them, φ(Ω, y, I) = ∫u in dP(u in |u out , I); |||| 2 denotes taking the modulus of a vector, and σ denotes variance;

[0037] Meanwhile, introduce the function χ to represent D, Ω, Ω c :

[0038]

[0039] Therefore, the optical flow flowing into Ω is expressed as u i in = χu i and the one flowing out is u i out = (1 - χ)u i ;

[0040] Finally, select χ and φ as the parameter function classes in the convolutional neural network. Denote the parameter by w, and the corresponding functions are and

[0041] To simplify the representation, omit the constant term of the loss function and convert it to the opposite of the original loss function, and the final loss function can be obtained

[0042]

[0043] Among them, is the restorer i, which minimizes the above formula, and w1 is the parameter of the restorer i; is the generator g, select such that u i out does not provide information for u i in and maximizes the above formula, and w2 is the parameter of the generator g; I is the image;

[0044] Finally, the optimization objective of is expressed in the following form:

[0045]

[0046] Step 3.3: Construct the generator g and the restorer i. The generator g and the restorer i together form a generative adversarial network to solve the optimization problem in Step 3.2; the generator g is used to generate the optical flow mask image mask of the moving target; the restorer i takes the CPN as the basic network architecture and restores the optical flow information inside the mask image according to the mask image generated by the generator g and the corresponding optical flow image;

[0047] Step 3.4: Use the DAVIS2016 dataset to train the constructed generative adversarial network to obtain the final generative adversarial network model.

[0048] Furthermore, the generator g and the restorer i described in step 3.3 together form a generative adversarial network, and the specific model is as follows:

[0049] 1) The input of the generator g is the RGB image I t and its corresponding optical flow u t:t+δT , and the output is the mask image mask of the moving target, where δT is randomly sampled from the uniform distribution U = [-5, 5] and δT ≠ 0, so as to introduce more information about the change of the optical flow of the image I t ; The generator g consists of an encoder and a decoder; The encoder part consists of 5 convolutional layers, and there is a BN layer after each convolutional layer. Each convolutional layer reduces the original image to 1 / 4 of the input image; After the encoder, there are 4 dilated convolutional layers with gradually increasing radii, and the radii are 2, 4, 8, and 16 in sequence; The decoder part consists of 5 convolutional layers, and generates a mask image with the same size as the input image through upsampling;

[0050] 2) The input of the restorer i is the RGB image I t and the mask image mask generated by the generator g, and the output is the optical flow image outside the predicted mask image, that is, the optical flow image of the background; The encoder part of the restorer i includes two branches, and the structures and parameters of the two branches are exactly the same, and each consists of 9 convolutional layers. LeakyReLu is used as the activation function after each convolutional layer. One network branch takes the normalized frame image as the input, and the other branch takes the optical flow image and the mask image generated by the generator as the input. The features encoded by the two network branches are connected by a concatenation operation (concat) and then passed to the decoder. The decoder is mainly composed of a transposed convolutional layer and a LeakyReLu activation function, and at the same time uses a skip structure to upsample the deep features and fuse them with the shallow features. Finally, an optical flow image with the same size as the input image is output.

[0051] Furthermore, a lightweight attention mechanism is introduced into the encoder parts of the generator g and the restorer i of the generative adversarial network model described in step 3; The attention module includes channel attention, spatial attention, and global attention.

[0052] 1) Channel attention mainly includes three operations: squeezing, excitation, and recalibration. First, for the input feature map F with size h×w×c, through the squeezing operation, the input features are compressed in the spatial dimension to obtain a feature vector s of size 1×1×c, representing the global features of the channel. Each element in this feature vector corresponds to each channel in the feature map. In fact, it is a process of performing global pooling on each feature map; then, through the excitation operation, the correlation between channels is established, and the weights w are used to learn the correlation between c channels, obtaining a channel weight e of size 1×1×c, which is generally implemented through a 1×1 convolution operation; finally, through the recalibration operation, the channel weights are multiplied by the original input feature map to obtain the weighted output feature map F′ C 。

[0053] 2) For the feature map F′ C Max pooling and average pooling operations are respectively performed to generate two feature matrices F MAX and F AVG ; then, a fusion operation is performed on the two feature matrices to obtain the fused feature map F MA , and the spatial attention weight W is obtained through the Sigmoid activation function. The fusion operation generally involves simply concatenating the feature matrices by channel and then performing a convolution operation; finally, the spatial attention weight matrix W is multiplied by the original input feature map F matrix to obtain the weighted output feature map F′ S 。

[0054] 3) The squeezing operation of global attention is the same as that of channel attention. For the excitation operation, it is replaced by the implementation of the following 4 consecutive operations: fc(2C / 16)→ReLU→fc(1)→Sigmoid. The excitation operation will generate a size selection factor where fc(·) represents the fully connected operation, C is the number of channels, and ReLU and Sigmoid are both activation functions. According to the output F′ S of the spatial attention mechanism and the size selection factor μ, the size-sensitive feature F′ Scale is calculated as follows:

[0055] F′ G =F+(μ*F′ S )

[0056] where, in order to avoid losing some important information in the region where the attention value is close to 0, an identity mapping term F is added.

[0057] Furthermore, in steps 4 to 5, the generator module of the trained generative adversarial network model is used to detect moving targets in the video sequence to be detected. The specific steps are as follows:

[0058] First, perform the preprocessing operation described in Step 1 on the video sequence to be detected;

[0059] Then, calculate the corresponding optical flow image according to the method described in Step 2;

[0060] Finally, input the preprocessed video sequence image and the corresponding optical flow image into the generator g obtained in Step 3, and the obtained output image is the prediction result of the moving target.

[0061] The present invention will be further clarified below with reference to the accompanying drawings of the specification. It should be understood that these embodiments are only used to illustrate the present invention and not to limit the scope of the present invention. After reading the present invention, those skilled in the art's various equivalent modifications of the present invention all fall within the scope defined by the appended claims of this application.

[0062] Embodiment

[0063] The present invention proposes an unsupervised moving target detection method with an attention mechanism. Based on the property that the background image region does not contain the information of the foreground image region, a generative adversarial network model is constructed according to the relationship of optical flow to realize the discrimination between the background and the moving target. The generative adversarial network includes a generator and a restorer; introducing the attention mechanism effectively improves the robustness of the tracking algorithm and reduces the interference of background noise, etc. on the tracking of the target. The basic idea is: first, construct a video database and preprocess the video; then use PWCNet to calculate the optical flow information for adjacent frames of each video; then use the preprocessed video and its corresponding optical flow information as the input of the generative adversarial network based on the attention mechanism to train the network model; finally, for the video sequence to be detected, the generator module of the network model can be used to obtain the detection result of the moving target.

[0064] As Figures 1 to 4 shown, the implementation of the present invention mainly includes four steps: (1) Preprocessing of the video sequence; (2) Obtaining the optical flow image of the video sequence through PWCNet; (3) Training the generative adversarial network with the video sequence and its corresponding optical flow image as the input; (4) Using the generator module of the trained network model to detect the moving target in the video sequence and output the detection result.

[0065] Step 1: Collect a video sequence through a camera and perform preprocessing to construct a database;

[0066] Since the video sequence collected in the natural scene may be interfered by factors such as uneven illumination, it is preprocessed, mainly including histogram equalization, normalization, etc. of the video sequence;

[0067] Step 2: Obtain the optical flow image of the video sequence through PWCNet;

[0068] Given the image I to be measured, the optical flow u from it to the next (previous) frame: D1 → R 2 is I1: to I2: mapping. PWCNet is a high-performance optical flow learning network that can efficiently obtain the optical flow information of a video sequence. The present invention uses PWCNet to calculate the optical flow information and perform normalization. The normalization operation mainly includes adjusting the optical flow image to the same size as the video sequence and then dividing by a constant, that is, reducing the values of the optical flow image proportionally to accelerate the training of the network.

[0069] Step 3: Use the video sequence and its corresponding optical flow image as inputs to train a generative adversarial network;

[0070] Step 3.1. Distinguishing moving objects from the background. Based on the principle that the background image region should not contain the information of the foreground image region of the moving object, the image of the region of interest can be interpreted as poorly as possible by learning the region outside the region of interest. Specifically, for a certain frame image I of the video sequence, assuming the image region is D, the image region of the moving object is Ω, and the background is Ω c = D / Ω, and the optical flow from it to the adjacent frame (the previous frame or the next frame) is u. Among them, the optical flow represents the apparent motion of the image brightness pattern and contains important information about the surface structure and dynamic behavior of the object. Use to represent the mutual information of two random variables. Given the optical flows u i 、u j at two positions in the image I, the concept of the foreground Ω can be formalized as the region with zero mutual information with the background::

[0071]

[0072] Among them, the mutual information represents the amount of information that the optical flow u j can provide about the predicted optical flow u i . The larger the value, the more information is provided; represents the information entropy, which is used to quantify the amount of information. The greater the uncertainty of the variable, the greater the information entropy, and the value is always greater than 0.

[0073] Step 3.2. Loss function based on the information reduction rate. According to the foreground and background defined above, combined with the Shannon information entropy theory, define the information reduction rate to construct the optimization objective; using two subsets (regions) x, y in D as inputs, define the information reduction rate γ as follows:

[0074]

[0075] Among them, represents the optical flow uy The amount of information that can be provided about predicting the optical flow u x ; the Shannon information entropy H(u x |I)u x represents the uncertainty of u x ; H(u x |u y ,I) represents the uncertainty of u y under the condition that u x is known; γ(x|y;I) represents the amount by which the uncertainty of u y decreases under the condition that u x is known, and its value ranges from 0 to 1. In particular, when u x and u y are independent, that is, one belongs to the foreground and the other belongs to the background image region, γ = 0; the optical flow in the target image region Ω is represented by u in ={u i ,i∈Ω}, and the one in the background region Ω c is represented by u out ={u j ,j∈Ω c}, so there is:

[0076]

[0077] where P(u in |I) represents the probability that the optical flow is the foreground optical flow, and P(u in |u out ,I) represents the probability of u out under the condition that u in is known. According to the information reduction rate γ, the loss function is defined as When is minimized, the optical flow of the background is sufficient to predict the foreground. Strict assumptions are made for the model as follows:

[0078]

[0079] where φ(Ω,y,I)=∫u in dP(u in |u out ,I); |||| 2 represents taking the modulus of the vector, and σ represents the variance. At the same time, the function χ is introduced to represent D, Ω, Ω c :

[0080]

[0081] Therefore, the optical flow flowing into Ω is represented by u i in =χu i , and the outflowing one is ui out =(1 - χ)u i ;

[0082] Finally, χ and φ are selected as the parameter function classes in the convolutional neural network. Let w represent the parameters, and the corresponding functions are and To simplify the representation, the constant term of the loss function is omitted, and it is converted to the negative of the original loss function, and the final loss function can be obtained

[0083]

[0084] where is the restorer i, which minimizes the above formula, and w1 is its parameter; is the generator g, and by choosing an appropriate such that u i out is not u i in provides information to maximize the above formula, and w2 is its parameter; I is the image.

[0085] Finally, the optimization objective of is expressed in the following form:

[0086]

[0087] Step 3.3: Construct the generator g and the restorer i. The generator g and the restorer i together form a generative adversarial network, which can effectively solve the optimization problem in Step 3.2. The generator g includes an encoder and a decoder part, which is used to generate the optical flow mask image of the moving target, and its network structure and parameters are shown in Table 1. The restorer i includes an encoder and a decoder part, which can restore the optical flow information outside the mask image through the mask image generated by the generator g, and its network structure and parameters are shown in Table 2.

[0088] 1) The input of the generator g is the RGB image I t and its corresponding optical flow u t:t+δT , and the output is the mask image mask of the moving target, where δT is randomly sampled from the uniform distribution U = [-5, 5] and δT ≠ 0, so as to introduce more information about the image I tThe change information of optical flow; The generator g consists of two parts: an encoder and a decoder; The encoder part consists of 5 convolutional layers, and each convolutional layer is followed by a BatchNormalization layer. Each convolutional layer reduces the original image to 1 / 4 of the input image; After the encoder, there are 4 dilated convolutional layers with gradually increasing radii, and the radii are 2, 4, 8, and 16 in sequence; The decoder part consists of 5 convolutional layers, and generates a mask image with the same size as the input image through upsampling;

[0089] 2) The restorer i takes the RGB image I t and the mask image mask generated by the generator g as inputs, and outputs the optical flow image outside the predicted mask image, that is, the optical flow image of the background; The encoder part of the restorer i includes two branches, and the structures and parameters of the two branches are exactly the same. Each is composed of 9 convolutional layers, and LeakyReLu is used as the activation function after each convolutional layer. One network branch takes the normalized frame image as the input, and the other branch takes the optical flow image and the mask image generated by the generator as the inputs. The features encoded by the two network branches are connected using the concatenation operation (concat) and then passed to the decoder. The decoder is mainly composed of transposed convolutional layers and LeakyReLu activation functions, and at the same time uses a skip structure to upsample the deep features and perform feature fusion with the shallow features. Finally, an optical flow image with the same size as the input image is output.

[0090] Table 1 Generator network parameters

[0091]

[0092] Note 1: There is a Batch Normalization after each convolutional layer, which is not shown in the figure.

[0093] Note 2: Dilated convolution fills rate - 1 zeros in the middle of the convolution kernel, which can expand the receptive field and capture multi - scale context information.

[0094] Note 3: Transposed convolutional layers can restore signals and perform upsampling.

[0095] Note 4: Attention modules are added in convolutional layers 2 - 3, 4 - 5, 7 - 10, and 11 - 12 to reduce the interference of background noise.

[0096] Table 2 Restorer network parameters

[0097]

[0098] Step 3.4: Use the training dataset to train the constructed generative adversarial network to obtain the final network model.

[0099] Step 4: Use the generator g of the trained network model to detect moving objects in the video sequence;

[0100] First, perform the preprocessing operation described in Step 1 on the video sequence to be detected; then, calculate the corresponding optical flow image according to the method described in Step 2; finally, input the preprocessed video sequence image and the corresponding optical flow image into the generator g obtained in Step 3, and the obtained output image is the mask image of the moving object.

[0101] An unsupervised moving object detection method based on the information reduction rate in the present invention discriminates the background and moving objects by constructing a generative adversarial network model according to the relationship of optical flow based on the property that the background image region does not contain the information of the foreground image region. The generative adversarial network includes a generator and a restorer; introducing an attention mechanism effectively improves the robustness of the tracking algorithm and reduces the interference of background noise, etc. on the tracking of the target. The basic idea is: first construct a video database and preprocess the video; then use PWCNet to calculate the optical flow information for adjacent frames of each video; then use the preprocessed video and its corresponding optical flow information as the input of the generative adversarial network based on the attention mechanism to train the network model; finally, for the video sequence to be detected, the detection result of the moving object can be obtained by using the generator module of the network model. Compared with the existing unsupervised moving object detection algorithms, the present invention makes full use of the optical flow information of the target and the background, fuses the feature channels of the moving object through the attention mechanism, reduces the interference of the background, and improves the detection performance of the moving object.

Claims

1. An unsupervised moving object detection method based on the information reduction rate, characterized in that, It includes the following steps: Step 1: Collect a video sequence through a camera and perform preprocessing to construct a database; Step 2: Calculate the optical flow image corresponding to the video sequence through the trained PWCNet and perform normalization; Step 3: Use the video sequence and its corresponding optical flow image as inputs to train a generative adversarial network model; Step 4: Perform the processing of Step 1 to Step 2 on the video sequence to be detected as well; Step 5: Extract the generator module of the trained generative adversarial network model to detect moving targets in the video sequence to be detected; For the training of the generative adversarial network model in Step 3, the specific steps are as follows: Step 3.1: Distinguishing moving targets from the background; For a frame image I of a video sequence, assume the image region is D, the image region of the moving object is Ω, and the background is Ω c = D / Ω, the optical flow from the current frame to the adjacent frame is u, and the adjacent frame is the previous frame or the next frame; where the optical flow represents the apparent motion of the image brightness pattern and contains information about the object surface structure and dynamic behavior; use to represent the mutual information of two random variables. Given the optical flow u at position i and position j in the image I i 、u j , the concept of the foreground, i.e., the image region Ω of the moving object, is formalized as the region with zero mutual information with the background: Among them, the mutual information represents the optical flow u at position j in the given image I j and the amount of information it can provide about the optical flow u at position i i . The larger the value of this mutual information, the greater the amount of information provided; The Shannon information entropy H(u i |I) represents the uncertainty of u i . The greater the uncertainty of the variable, the greater the information entropy, and the value is always greater than 0; H(u i |u j ,I) represents the uncertainty of u j under the condition that u i is known; Step 3.2: Loss function based on the information reduction rate; According to the above-defined foreground and background, combined with the Shannon information entropy theory, define the information reduction rate to construct the optimization objective; take two subsets in D, namely region x and region y, as inputs, and the optical flows of region x and region y are u x and u y , and define the information reduction rate γ as follows: Among them, represents the optical flow u of region y in the given image I y that can provide about the optical flow u of region x x ; the Shannon information entropy H(u x |I) represents the uncertainty of u x ; H(u x |u y ,I) represents the uncertainty of u y under the condition that u x is known; γ(x|y; I) indicates that u is known y Under the condition of x The uncertainty reduction of is between 0 and 1. x and u y When one belongs to the foreground image area and the other belongs to the background image area, γ = 0; the optical flow in the image area Ω of the moving target is u in = {u i ,u∈Ω} represents the background area Ω c The use of u out = {u j ,j∈Ω c } indicates that: where P(u in |I) represents the probability that the optical flow is the foreground optical flow, and P(u in |u out ,I) represents the probability of u out given that u in is known; according to the information reduction rate γ, the loss function is defined as When is minimized, the optical flow of the background is sufficient to predict the foreground; Make strict assumptions about the model as follows: where, φ(Ω,y,I) = ∫y in dP(u in |u out ,I); |||| 2 denotes taking the modulus of a vector, and σ denotes variance; Meanwhile, introduce the function χ to represent D, Ω, Ω c : Therefore, the optical flow flowing into Ω is expressed as u i in = χu i , and the outflow is u i out = (1 - χ)u i ; Finally, χ and φ are selected as the parameter function classes in the convolutional neural network. Let w represent the parameters, and the corresponding functions are and Omit the constant term of the loss function and convert it to the opposite of the original loss function to obtain the final loss function Among them, is the restorer i that minimizes the above formula, and w1 is the parameter of the restorer i; is the generator g that selects such that u i out is not u i in provides information to maximize the above formula, and w2 is the parameter of the generator g; I is the image; Final optimization objective Is expressed in the following form: Step 3.3: Construct a generator g and a restorer i. The generator g and the restorer i together form a generative adversarial network to solve the optimization problem in Step 3.2; the generator g is used to generate an optical flow mask image mask of the moving target; the restorer i uses the CPN as the basic network architecture to restore the optical flow information inside the mask image according to the mask image generated by the generator g and the corresponding optical flow image; Step 3.4: Use the DAVIS2016 dataset to train the constructed generative adversarial network to obtain the final generative adversarial network model.

2. The unsupervised moving object detection method based on the information reduction rate according to claim 1, wherein The generator g and the restorer i in Step 3.3 together form a generative adversarial network, and the specific model is as follows: 1) The input of the generator g is the RGB image I t and its corresponding optical flow u t:t+δT , and the output is the mask image mask of the moving target, where δT is randomly sampled from the uniform distribution U = [-5, 5] and δT ≠ 0, thus introducing more information about the change of the optical flow of the image I t ; The generator g consists of an encoder and a decoder; the encoder part consists of 5 convolutional layers, and there is a BN layer after each convolutional layer. Each convolutional layer reduces the original image to 1 / 4 of the input image; after the encoder, there are 4 dilated convolutional layers with gradually increasing radii, and the radii are 2, 4, 8, and 16 in sequence; the decoder part consists of 5 convolutional layers, and generates a mask image with the same size as the input image through upsampling; 2) The input of the restorer i is the RGB image I t and the mask image mask generated by the generator g, and the output is the optical flow image outside the predicted mask image, that is, the optical flow image of the background; the encoder part of the restorer u includes two branches, and the structures and parameters of the two branches are exactly the same, each consisting of 9 convolutional layers, and LeakyReLu is used as the activation function after each convolutional layer; one network branch takes the normalized frame image as the input, and the other branch takes the optical flow image and the mask image generated by the generator as the input; the features encoded by the two network branches are connected by the concatenation operation concat and then passed to the decoder, which is mainly composed of deconvolutional layers and LeakyReLu activation functions, and at the same time, the skip structure is used to upsample the deep features and fuse them with the shallow features; finally, an optical flow image with the same size as the input image is output.

3. The unsupervised moving object detection method based on the information reduction rate according to claim 2, wherein In the encoder parts of the generator g and the restorer i of the generative adversarial network model in Step 3, a lightweight attention mechanism is introduced. The attention module includes channel attention, spatial attention, and global attention: 1) Channel attention consists of three operations: squeezing, excitation, and recalibration. First, for the input feature map F with size h×w×c, the squeezing operation compresses the input features in the spatial dimension to obtain a feature vector s of size 1×1×c, representing the global features of the channel. Each element in this feature vector corresponds to each channel in the feature map, which is actually a process of performing global pooling on each feature map. Then, the excitation operation establishes the correlation between channels, uses the weight w to learn the correlation between c channels, and obtains a channel weight e of size 1×1×c, which is achieved through a 1×1 convolution operation. Finally, through the recalibration operation, the channel weight is multiplied by the original input feature map to obtain the weighted output feature map F'. C ; 2) For the feature map F' C Max pooling and average pooling operations are respectively performed to generate two feature matrices F MAX and F AVG ; Then, a fusion operation is performed on the two feature matrices to obtain a fused feature map F MA , and a spatial attention weight matrix W is obtained through processing by the Sigmoid activation function, where the fusion operation includes concatenating the feature matrices by channel and then performing a convolution operation; Finally, the spatial attention weight matrix W is multiplied by the original input feature map F matrix to obtain a weighted output feature map F' S ; 3) The squeezing operation of global attention is the same as that of channel attention. For the excitation operation, we replace it with the implementation of the following four consecutive operations: fc(2C / 16) → ReLU → fc(1) → Sigmoid. The excitation operation generates a size selection factor where fc(·) represents the fully connected operation, C is the number of channels, and both ReLU and Sigmoid are activation functions; according to the output F′ of the spatial attention mechanism S and the size selection factor μ, the size-sensitive feature F′ is obtained Scale , as shown in the following formula: F′ G = F + (μ * F′ s ) Among them, in order to avoid losing important information in the area where the attention value is close to 0, an identity mapping term F is added.

4. The unsupervised moving object detection method based on the information reduction rate according to claim 3, characterized in that In Step 4 to Step 5, extract the generator module of the trained generative adversarial network model to detect moving targets in the video sequence to be detected. The specific steps are as follows: First, perform the preprocessing operation described in Step 1 on the video sequence to be detected; Then, calculate the corresponding optical flow image according to the method described in Step 2; Finally, input the preprocessed video sequence image and the corresponding optical flow image into the generator g obtained in Step 3, and the obtained output image is the prediction result of the moving target.

Images