A scene-adaptive object detection method based on moving foreground

By acquiring moving foreground target boxes and using feature extraction and similarity measurement networks for feature alignment, the problem of poor detection performance during scene changes in existing technologies is solved, achieving better detection accuracy and generalization ability.

CN114399697BActive Publication Date: 2024-11-19HANGZHOU INNOVATION RES INST OF BEIJING UNIV OF AERONAUTICS & ASTRONAUTICS
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN202111416174.2
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-11-25
Publication Date
2024-11-19
Estimated Expiration
2041-11-25

AI Technical Summary

Technical Problem

Existing domain-adaptive object detection methods do not perform well when scene changes, especially since the domain differences in regression are not fully considered, leading to inaccurate candidate region extraction and poor detection performance in images with large domain differences.

Method used

By acquiring the moving foreground bounding boxes of the source and target domains, feature alignment is performed using a feature extraction module and a generative similarity measurement network. Combined with a global feature alignment network, the network is optimized to reduce domain differences and improve detection performance.

Benefits of technology

It effectively improves the model's generalization performance in new scenarios, enhances the accuracy of candidate box extraction and detection precision, and reduces the risk of model overfitting.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114399697B_ABST
    Figure CN114399697B_ABST
Patent Text Reader

Abstract

With the increasing development of deep learning technology, the requirements for model generalization performance in real environments are increasing, and the impact of differences in lighting, background, etc. on the generalization performance of models has attracted widespread attention. The present invention discloses a scene adaptive target detection method based on motion foreground. The method effectively utilizes the motion foreground target frame by utilizing the prior of the consistency of the motion foreground and global target data distribution, and at the same time calculates the instance feature similarity through the decoder, which greatly improves the effect of the model in the target domain. Experimental results show that the method provided by the present invention has greatly improved the target detection effect in a real environment.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to a scene adaptive target detection method based on motion foreground. Background Art

[0002] In the field of computer vision, object detection is an important topic. Its task is to find the area of ​​interest in the image video and determine its category and location. At present, many deep learning-based methods can achieve good results on benchmark datasets. However, due to the existence of domain differences, that is, when the target size, camera angle, lighting, and background environment change, the effect of the model decreases to varying degrees. The simplest and most effective way to solve this problem when training models on the same domain is data-driven training. However, on the one hand, manually annotating datasets takes a lot of manpower and material resources, and on the other hand, many practical fields cannot be manually annotated. Therefore, in order to solve the problem of decreased model generalization performance due to different data distributions, domain adaptive object detection methods have emerged.

[0003] At present, the domain adaptation-based object detection methods include feature-based and model-based methods. The most classic method (DA-FasterRcnn) is to minimize the domain differences of feature maps by nesting adversarial training modules, so that the classifier cannot distinguish the candidate box features. Subsequent related algorithms are improved based on this algorithm. Another type of algorithm achieves pixel-level domain alignment through adversarial generation.

[0004] However, the above algorithm only considers the domain difference in classification, but not the domain difference in regression, which leads to unsatisfactory results after scene transformation. In addition, for images with large domain differences, due to unknown data distribution, it is impossible to effectively extract suitable candidate box targets in the first stage of the two-stage target detection, i.e., extracting candidate regions RPN, and it is also impossible to determine which regions’ features need to be aligned during feature alignment. Summary of the invention

[0005] The present invention proposes a scene adaptive target detection method based on motion foreground to solve the above technical problems existing in the prior art.

[0006] According to one aspect of the present invention, a scene adaptive target detection method based on motion foreground is provided, comprising the following steps:

[0007] A) obtaining a source domain dataset and a target domain dataset, wherein the source domain dataset includes a source domain RGB image, a target detection manual label, and a motion foreground target frame label, and the target domain dataset includes a target domain RGB image and a motion foreground target frame label;

[0008] B) Inputting the source domain dataset and the target domain dataset into the feature extraction module to obtain source domain features and target domain features respectively;

[0009] C) inputting the source domain features and instance domain features obtained in step B) into the first candidate frame foreground frame feature aggregation module to obtain source domain instance features and target domain instance features respectively;

[0010] D) inputting the source domain features obtained in step B) into the second candidate frame foreground frame feature aggregation module to obtain source domain classification regression features;

[0011] E) inputting the source domain instance features and the target domain instance features obtained in step C) into the generative similarity measurement network module, calculating the loss, optimizing the network, and reducing the domain difference;

[0012] F) inputting the source domain classification regression features obtained in step D) into the classification regression module, calculating the loss, and optimizing the network;

[0013] G) inputting the source domain features and target domain features obtained in step B) into the global feature alignment network module, calculating the loss, and optimizing the network;

[0014] Wherein, the method for obtaining the moving foreground target frame in step A) includes but is not limited to vibe, mixed Gaussian background elimination, frame difference, and optical flow;

[0015] The first candidate frame foreground frame feature aggregation module in step C) includes, during the training process, combining the RPN candidate frame with a higher confidence with the motion foreground target frame, and extracting source domain instance features and target domain instance features after sample equalization;

[0016] The second candidate frame foreground frame feature aggregation module in step D) includes, during the training process, combining the motion foreground target frame with the RPN candidate frame to extract source domain classification regression features;

[0017] The generative similarity measurement network module in step E) includes, during the training process, reconstructing the instance features of the extracted source domain instance features and the target domain instance features using a decoder to obtain decoded features, calculating the similarity loss of the decoded features, and achieving instance feature alignment;

[0018] The classification regression module in step F) includes, during the training process, calculating the classification regression loss of the source domain data set through the source domain target frame true value label, so as to ensure the accuracy of source domain target detection;

[0019] Among them, the global feature alignment network module in step G) includes a gradient reversal layer and a classifier, thereby realizing image-level feature alignment.

[0020] Compared with the prior art, the present invention has the following beneficial effects:

[0021] The present invention proposes a scene-adaptive target detection method based on motion foreground. The method effectively utilizes the prior knowledge of motion foreground and uses a decoder for feature alignment to achieve good detection effect and effectively improve the generalization performance of the model in new scenes. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 is a flow chart of a scene adaptive target detection method based on motion foreground according to an embodiment of the present invention;

[0023] Figure 2 is a schematic diagram of the construction of a first candidate frame foreground frame aggregation module according to an embodiment of the present invention;

[0024] Figure 3 is a schematic diagram of the construction of a second candidate frame foreground frame aggregation module according to an embodiment of the present invention;

[0025] Figure 4 is a schematic diagram of the structure of a generative similarity measurement module according to an embodiment of the present invention;

[0026] Figure 5 is a schematic diagram of the construction of a global feature alignment module according to an embodiment of the present invention; DETAILED DESCRIPTION

[0027] Figure 1 In the embodiment of ns represents the number of real samples in the source domain, represents sample i in the source domain, represents the set of target box coordinate values ​​of sample i in the source domain, represents the target category of the i-th sample in the source domain. In this specific embodiment, there is only a single category of pedestrians. represents the set of motion foreground target frame coordinate values ​​of the i-th sample in the source domain, The number of target boxes in the set and The number of target boxes in the set is inconsistent; the target domain dataset uses Represents, where nT represents the number of real samples in the target domain, represents sample i in the target domain, Represents a dataset of moving foreground target box coordinate values ​​contained in sample i in the target domain.

[0028] The scene adaptive target detection method based on motion foreground according to the present embodiment trains the model based on the source domain data and the motion foreground data of the target domain, so that the model has a good detection effect even without the target domain (T) labeled data set, which includes the following steps:

[0029] A) The source domain continuous frame sample set And the target domain continuous frame sample set Input the Vibe moving target detection algorithm to obtain the source domain moving foreground target frame and the target domain moving foreground target frame (where S represents the source domain and T represents the target domain), so the source domain dataset D S With the target domain dataset D T is acquired.

[0030] B) Source domain dataset D S With the target domain dataset D T Input feature extraction module (S101), obtain source domain feature f1 and target domain feature f2, in this specific embodiment, the backbone network of the feature extraction module is ResNet-101;

[0031] C) The source domain feature f1 and the source domain motion foreground target frame Input the first candidate frame foreground frame aggregation module PFA1 (S112) to obtain the source domain instance feature pfs, and combine the target domain feature f2 and the target domain motion foreground target frame Input the candidate frame foreground frame aggregation module PFA1 (S113) to obtain the target domain instance feature pft;

[0032] D) Combine the source domain feature f1 with the moving foreground target box Input the second candidate frame foreground frame aggregation module PFA2 (S111) to obtain the source domain classification regression feature crs;

[0033] E) Input the source domain classification regression feature crs into the classification regression module (S121). Calculate the classification regression loss of the source domain data set through the source domain target box true value label, and optimize the network weights of the feature extraction module and the classification regression module by training the source domain data set;

[0034] F) inputting the source domain instance feature pfs and the target domain instance feature pft into the generative similarity measurement module GSM (S122), training the source domain data set and the target domain data set so that the source domain instance feature and the target domain instance feature are as similar as possible, optimizing the network weights of the feature extraction module and the generative similarity measurement module GSM, and thus improving the generalization performance of the model;

[0035] G) The source domain feature f1 and the target domain feature f2 are passed through a global feature alignment module GFA (S123); by training the source domain dataset and the target domain dataset, the source domain and target domain features are made as similar as possible, and the network weights of the feature extraction module and the global feature alignment module GFA (gradient reversal layer (GRL) and classifier) ​​are optimized so that the domains to which the source domain feature f1 and the target domain feature f2 belong cannot be distinguished.

[0036] According to a further aspect of the present invention, Figure 2 As shown, the first candidate frame foreground frame aggregation module PFA1 includes submodules that respectively perform the following steps:

[0037] Step S201: Take the source domain continuous frame samples and target domain consecutive frame samples Input the RPN network (region proposal network) to generate a set of positive and negative candidate boxes in represents the jth candidate box of the i-th image sample in the source domain and the target domain, C represents the number of candidate boxes generated by the RPN network, and in this embodiment, the value is 64. represents the i-th image sample in the source domain, represents the i-th image sample in the target domain;

[0038] Step S202: In this embodiment, the source domain continuous frame samples and target domain consecutive frame samples Input the Vibe moving target detection algorithm to obtain the source domain moving foreground target frame and the target domain moving foreground target frame respectively fb i is the set of motion foreground target frames of the i-th image sample;

[0039] Step S211: Selecting a set of positive and negative candidate boxes The candidate boxes whose confidence is greater than the preset threshold TH, in this embodiment, TH is 0.7;

[0040] Step S212: Merge the candidate frame obtained in step S211 with the moving foreground target frame obtained in step S202

[0041] Step S213: Input the output of step S212 into the sample equalization filter to obtain the source domain PFA1 candidate frame set and the target domain PFA1 candidate frame set Where {b if} jrepresents the jth PFA1 candidate box set in the i-th sample in the dataset, f is the identifier representing the candidate box set generated by the PFA1 module, S represents the source domain, T represents the target domain, C Sf , C Tf represents the number of frames in the set of candidate frames and moving foreground target frames in the source domain and the number of frames in the set of candidate frames and moving foreground target frames in the target domain, respectively, where C Sf =C Tf ;

[0042] Among them, the sample equalization filter sets a fixed sample number f_num, which is set to 8 in this embodiment, so that the number of PFA1 candidate boxes in the i-th sample in the source domain (S) and the i-th sample in the target domain (T) are consistent, thereby eliminating sample imbalance.

[0043] According to a further aspect of the present invention, Figure 3 As shown, the second candidate frame foreground frame aggregation module PFA2 includes submodules that respectively perform the following operations:

[0044] Step S301: source domain continuous frame samples Generate a set of positive and negative candidate regions in the source domain through the RPN network Where C is a constant representing the number of candidate boxes generated by the RPN network. In this embodiment, the value is 64. is the jth candidate box of the i-th sample in the source domain;

[0045] Step S302: In this embodiment, the source domain continuous frame samples Input into the Vibe moving target detection algorithm to obtain the source domain moving foreground target box fb i is the set of motion foreground target frames of the i-th image sample;

[0046] Step S311: Overlay the source domain positive and negative candidate regions generated in step S301 The source domain motion foreground target box generated in step S302 Generate source domain PFA2 candidate box set Where {b ia} j represents the jth PFA2 candidate box set in the i-th sample, a is the identifier representing the candidate box set generated by the PFA2 module, S represents the source domain, C Sa The number of boxes in the set of candidate boxes and motion foreground target boxes in the source domain. The addition of motion foreground target boxes solves the problem of not being able to generate accurate candidate boxes when the target sizes of the two domains are too different.

[0047] According to a further aspect of the present invention, in the classification and regression module S121, the source domain feature f1 is combined with the PFA2 candidate frame set Input the classifier and regressor to regress and classify the samples. The loss function of this part is as follows:

[0048] L det =L RPN +L T

[0049] In the above formula, L det Represents the source domain detection loss function, represented by L RPN , L T It consists of two parts, L RPN is the RPN loss function, L T is a two-stage classification and regression loss function; wherein the subscript det refers to the name of the total loss function of the classification and regression module, RPN refers to the name of the loss function of the first stage RPN stage of the two-stage target detection framework, and T refers to the name of the loss function of the second stage classification and regression stage of the two-stage target detection framework. In this specific embodiment, the classification loss uses the cross entropy loss, and the regression loss uses the mean square error (MSE) loss.

[0050] According to a further aspect of the present invention, Figure 4 As shown, the generative similarity measurement module GSM includes submodules that respectively perform the following operations:

[0051] Step S401: The PFA1 candidate box set generated by the PFA1 module Input the source domain feature f1 and target domain feature f2 extracted by the feature extraction module to generate the source domain instance feature f S and the target domain instance feature f T ;

[0052] Step S402: The source domain instance feature f S and the target domain instance feature f T Input adaptive average pooling layer output pooling feature f Ss402 ,f Ts402 , its output size is 8*8, and the number of channels is equal to the source domain instance feature f S Number of feature channels;

[0053] Step S403: Input the output in S402 into the first 1*1 convolutional layer. In this embodiment, the number of channels is 1024, and output the first convolutional layer features of the source domain and the first convolutional layer features of the target domain f Ss403 ,f Ts403 ;

[0054] Step S404: Input the output in S403 into the first upsampling module, which consists of an interpolation upsampling layer, a convolution layer, a batch normalization layer, etc. The feature size after upsampling is 16*16, and the number of channels in this embodiment is 256. The first upsampling feature of the source domain and the first upsampling layer feature of the target domain f are output. Ss404 ,f Ts404 ;

[0055] Step S405: input the output of S404 into the second upsampling module. The feature size after upsampling is 32*32. In this embodiment, the number of channels is 256. The second upsampling feature of the source domain and the second upsampling layer feature of the target domain are output. Ss405 ,f Ts405 ;

[0056] Step S406: input the output of S405 into upsampling module 3. After upsampling, the feature size is 64*64. In this embodiment, the number of channels is 256. The second upsampling feature of the source domain and the second upsampling layer feature of the target domain are output. Ss406 ,f Ts406 ;

[0057] Step S407: Input the features output in S406 into the second 1*1 convolutional layer, and generate features with 3 channels, generating source domain decoding features and target domain decoding features f SG ,f TG .

[0058] Calculate the source domain and target domain decoding features f SG ,f TG The perceptual loss is obtained by ins ;

[0059] L ins =E(G(S),G(T))

[0060] Where E is the perceptual loss, which is a loss function used to measure the similarity between images;

[0061] L ins Decode features for the source domain and target domain f SG ,f TG Perceptual loss value, E is the perceptual loss calculation function (this perceptual loss function is an existing technology), G(S) and G(T) respectively represent the source domain instance features and the target domain instance features f S ,f T The source domain decoding features and target domain decoding features f generated by steps S402-S407 (shared decoder G) SG ,f TG. This scheme can effectively measure the similarity between instance features of two domains (source domain and target domain). By training the feature extraction module and the generative similarity measurement module GSM, the instance features of the source domain and the instance features of the target domain can be made as similar as possible, ensuring the accuracy of the classification and regression module in the target domain; at the same time, the use of the decoder enhances the generalization performance of the model, reduces the risk of model overfitting, and reduces the failure rate of model training.

[0062] According to a further aspect of the present invention, Figure 5 As shown, the global feature alignment module GFA includes submodules that perform the following operations respectively:

[0063] Step S501: Obtain the source domain feature f1 and the target domain feature f2 generated in the above steps;

[0064] Step S502: Input the source domain feature f1 and the target domain feature f2 into the gradient reversal layer GRL. In the conventional back propagation, the loss (the difference between the predicted value and the true value) is passed forward layer by layer. Each layer of the network calculates the gradient according to the transferred loss, and then updates the parameters of the network in this layer. The GRL layer reverses the error transmitted to this layer, so that the network training objectives before and after GRL are opposite, so as to achieve the adversarial effect;

[0065] Step S503: input the source domain feature f1 and the target domain feature f2 into the classifier to distinguish the source domain feature from the target domain feature. The classifier includes a convolution layer and an activation layer, and specifically performs the operations of steps S511-S513 respectively.

[0066] Among them, the loss function of the global feature alignment module GFA is the loss function L of the classifier img In this embodiment, L img is the cross entropy loss function:

[0067]

[0068] Among them, N is the sum of all samples in the source domain and the target domain, i is the sample number, and y i is the actual label of the sample, i.e., whether it belongs to the source domain or the target domain, p i is the probability of belonging to different categories after passing through the classifier.

[0069] In one embodiment of the present invention, the final global loss function is:

[0070] L=L det +λ 1 L ins +λ 2 L img

[0071] where λ 1 ,λ2 It is an empirical value used to measure the contribution of the three losses to the final loss, and is all taken as 1.

[0072] Advantages of the present invention include:

[0073] (1) The present invention makes full use of the prior of motion foreground and integrates it well into the training framework. By using the FPA1 candidate frame foreground frame aggregation module and the FPA2 two major candidate frame foreground frame aggregation modules, the candidate frames extracted by the RPN network are effectively fused with the motion foreground target frames, so that the two types of candidate frames complement and influence each other, thereby optimizing the model effect.

[0074] (2) When performing instance feature alignment, in order to reduce the risk of model overfitting and improve the accuracy of target box regression, the present invention abandons the existing method of feature alignment through a classifier and instead uses a decoder to reduce overfitting and calculates the loss through a perceptual loss function, thereby greatly improving the effect of the model in the target domain.

[0075] (3) In the fusion of the candidate box extracted by the RPN network and the moving foreground target box, sample balance is effectively achieved through the sample balance filter.

[0076] In order to verify the effectiveness and improvement of the method of the present invention, the inventors conducted the following experiment, in which the test process only needs to be tested according to the two-stage detection algorithm test process, so the speed is consistent with the conventional two-stage algorithm. By adding some components during model training, the trained model can obtain good results in both the source domain and the target domain.

[0077] The source domain dataset and target domain dataset used in the test instances of this experiment are both from real scenes, and are named DML dataset and ZN dataset respectively, where DML dataset is the source domain dataset and ZN dataset is the target domain dataset.

[0078] Experimental details: The parameters used in this experiment are consistent with the original DA-FasterRcnn algorithm (classic domain adaptive detection algorithm). The backbone network uses ResNet-50, and the backbone network is initialized with ImageNet pre-trained weights. After training 70,000 images, the average precision map of the target domain is calculated. The experiment is based on the PyTorch framework and the hardware platform used is: NVIDIAGTX-2080Ti.

[0079] Table 1 is a comparison chart of experimental results, where the method Da-FasterRcnn is a classic domain adaptive detection algorithm, and the method PFA1 is to add the first candidate frame foreground frame aggregation module PFA1 to the classic algorithm, that is, to fuse the motion foreground target frame on the basis of the RPN candidate frame. It can be seen that the method of the present invention significantly improves the detection effect of the target domain.

[0080] Table 1: Domain Adaptive Detection Results

[0081] method map(%) DA-FasterRcnn 27.45 PFA1 33.97 Method of the present invention 63.81

Claims

1. A scene adaptive object detection method based on motion foreground, which trains the model based on source domain data and target domain foreground data, so that the model also has good detection effect in the target domain (T), characterized by The steps include: A) Input the source domain continuous frame sample set and the target domain continuous frame sample set into the moving target detection algorithm, output the moving foreground target frame of the source domain continuous frame sample and the moving foreground target frame of the target domain continuous frame sample, and together with the source domain annotation label, form the source domain dataset and the target domain dataset; B) inputting the source domain dataset and the target domain dataset into a feature extraction module to obtain source domain features and target domain features; C) inputting the source domain features, the target domain features and the moving foreground target frame into the first candidate frame foreground frame aggregation module (PFA1) respectively to obtain the source domain instance features and the target domain instance features; D) inputting the source domain features and the source domain motion foreground target frame into the second candidate frame foreground frame aggregation module (PFA2) to obtain the source domain classification regression features; E) Input the source domain classification regression features into the classification regression module and calculate the loss with the source domain target box true value label to obtain the optimized detection effect on the source domain; F) Input the source domain instance features and the target domain instance features into the Generative Similarity Measurement Module (GSM) to make the source domain instance features and the target domain instance features as similar as possible and improve the generalization performance and reduce overfitting; G) Input the source domain features and the target domain features into the global feature alignment module (GFA) to align the image features, so that the domains to which the source domain features and the target domain features belong cannot be distinguished. The first candidate frame foreground frame aggregation module (PFA1) includes submodules that respectively perform the following operations: Step S201: inputting source domain continuous frame samples and target domain continuous frame samples into the RPN network to generate a source domain positive and negative candidate box set and a target domain positive and negative candidate box set; Step S211: selecting source domain positive and negative candidate boxes and target domain positive and negative candidate boxes whose confidences are greater than a preset threshold TH from the source domain positive and negative candidate box set and the target domain positive and negative candidate box set generated in step S201; Step S202: obtaining a source domain moving foreground target frame and a target domain moving foreground target frame through a moving target detection algorithm; Step S212: Merge the source domain positive and negative candidate frames and the target domain positive and negative candidate frames whose confidences are greater than a preset threshold TH obtained in step S211 with the source domain motion foreground target frame and the target domain motion foreground target frame obtained in step S202 to obtain a source domain merged target frame and a target domain merged target frame; Step S213: obtaining the source domain candidate box and the target domain candidate box of the first candidate box foreground box aggregation module (PFA1) through the sample equalization filter; The sample equalization filter copies or deletes the source domain merged target box and the target domain merged target box generated in step S212, so that the number of source domain merged target boxes included in the i-th sample in the source domain (S) and the number of target domain merged target boxes included in the i-th sample in the target domain (T) are consistent, thereby effectively utilizing the motion foreground prior and eliminating sample imbalance; The second candidate frame foreground frame aggregation module (PFA2) includes submodules that respectively perform the following operations: Step S301: Pass source domain continuous frame samples through the RPN network to generate a source domain positive and negative candidate box set; Step S302: using a moving object detection algorithm to obtain a moving foreground object frame in the source domain; Step S311: superimposing the source domain positive and negative candidate frame sets and the source domain motion foreground target frame to generate a source domain candidate frame set of the second candidate frame foreground frame aggregation module (PFA2), and adding the source domain motion foreground target frame to the source domain positive and negative candidate frame sets to solve the problem that the model cannot generate a good candidate target frame when the target size difference between the source domain and the target domain is too large; The Generative Similarity Measurement Module (GSM) includes submodules that perform the following operations: Step S401: using the source domain candidate frame of the first candidate frame foreground frame aggregation module (PFA1) generated in step S213, extracting the source domain instance features in the source domain features; Using the target domain candidate frame of the first candidate frame foreground frame aggregation module (PFA1) generated in step S213, intercepting the target domain instance features in the target domain features; Step S402: input the source domain instance features and the target domain instance features into the adaptive average pooling layer, change the size of the source domain instance features and the target domain instance features to 8*8, the number of channels is equal to the number of channels of the source domain instance features, and output the source domain pooling layer features and the target domain pooling layer features; Step S403: inputting the source domain pooling layer features and the target domain pooling layer features obtained in S402 into the first 1*1 convolutional layer, where the first 1*1 convolutional layer is a 1*1 convolutional layer and outputs the source domain first convolutional layer features and the target domain first convolutional layer features; Step S404: inputting the first convolutional layer features of the source domain and the first convolutional layer features of the target domain obtained in S403 into a first upsampling module, which performs interpolation upsampling, convolution and / or batch normalization layer operations, and outputs the first upsampling layer features of the source domain and the first upsampling layer features of the target domain; Step S405, inputting the output of the first upsampling module in step S404 into a second upsampling module, the second upsampling module performs interpolation upsampling, convolution and / or batch normalization layer operations, and outputs source domain second upsampling layer features and target domain second upsampling layer features; Step S406, inputting the output of the second upsampling module in step S405 into a third upsampling module, the third upsampling module performs interpolation upsampling, convolution and / or batch normalization layer operations, and outputs source domain third upsampling layer features and target domain third upsampling layer features; Step S407, input the output of the third upsampling module in step S406 into the second 1*1 convolutional layer, the second 1*1 convolutional layer is a 1*1 convolutional layer, generate source domain decoding features and target domain decoding features, the number of channels of the source domain decoding features and the target domain decoding features is 3, calculate the perceptual loss of the source domain decoding features and the target domain decoding features, and obtain the loss L ins : L ins =E(G(S),G(T)) in: L ins is the perceptual loss value of the source domain decoding feature and the target domain decoding feature, E is the perceptual loss calculation function, G(S) refers to the source domain decoding feature generated by the source domain instance feature through steps S402-S407, G(T) refers to the target domain instance feature generated through steps S402-S407 and the target domain decoding feature, The global feature alignment module includes submodules that perform the following operations: Step S501: Acquire source domain features and target domain features; Step S502: Input the source domain features and the target domain features into the gradient reversal layer to output the classification features. The gradient reversal layer reverses the error transmitted to this layer, so that the network training targets before and after the gradient reversal layer are opposite to each other, so as to achieve the adversarial effect and output the classification features. Step S503: input the above classification features into a classifier to distinguish source domain features from target domain features, the classifier comprising a first classifier convolution layer, a first classifier activation layer, and a second classifier convolution layer; in: The gradient reversal layer achieves a certain degree of feature alignment at the image level, and the loss function of the global feature alignment module (GFA) is the loss function of the classifier.

2. The scene-adaptive target detection method based on moving foreground according to claim 1, characterized in that: The step B) includes inputting the source domain continuous frame samples and the target domain continuous frame samples into ResNet-101 as the feature extraction network, and using the obtained last layer features as the source domain features and the target domain features.

3. The scene adaptive target detection method based on moving foreground according to claim 1, characterized in that: The classification regression module uses the source domain candidate frame set of the second candidate frame foreground frame aggregation module (PFA2) generated in step S311 to intercept the source domain instance features to generate classification regression features, and accesses the first convolutional layer of the classification regression module to regress and classify the samples. The loss function involved is Includes: classification regression loss function L T And RPN loss function L RPN , the loss function L of the source domain target detection algorithm det for: L det =L RPN +L T In the above formula, L RPN , L T They are the RPN loss function and the classification regression loss function respectively; the subscript det refers to the total loss function of the classification regression module, RPN refers to the loss function of the first stage RPN of the two-stage target detection framework, and T refers to the name of the loss function of the second stage classification regression stage of the two-stage target detection framework.

4. The scene-adaptive target detection method based on motion foreground according to claim 1, Features: The loss function L of the global feature alignment module img is the cross entropy loss function: Among them, N is the total number of samples in the source domain and the target domain, i is the sample number, and y i is the actual label of the sample, i.e., whether it belongs to the source domain or the target domain, p i is the probability of belonging to different categories after passing through the classifier.

5. The scene adaptive target detection method based on moving foreground according to claim 4 is characterized in that: The global loss function is: L=L det +λ1L ins +λ2L img Among them, λ1 and λ2 are empirical values, which are used to measure the contribution of each of the three losses to the final loss.

6. The scene adaptive target detection method based on moving foreground according to claim 1, characterized in that: The moving target detection algorithm includes a frame difference method and / or a background elimination method.

7. A computer-readable storage medium storing a computer program, wherein the computer program enables a processor to execute the scene adaptive target detection method according to any one of claims 1 to 6.

Citation Information

Patent Citations

  • Target detection method of image data and related device

    CN111339891A

  • Target detection method based on two-stage local feature alignment

    CN113052184A

  • Adaptive target detection method and system based on foreground selection domain

    CN113343989A

  • Image salient object segmentation method and apparatus based on reciprocal attention between foreground and background

    US20200372660A1

  • Target detection and training for target detection network

    US20210056708A1