A method for object detection in complex background based on domain adaptation mechanism

By applying the ForegroundNet-DA model based on the domain adaptive mechanism in complex background object detection tasks, the problems of low detection performance and poor generalization ability caused by small training data are solved, and efficient object detection in complex backgrounds are achieved.

CN115147639BActive Publication Date: 2025-04-29SOUTHEAST UNIV
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202210558590.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-05-20
Publication Date
2025-04-29
Estimated Expiration
2042-05-20

AI Technical Summary

Technical Problem

The training data scale of the object detection task in complex context is small, resulting in limited performance of the object detection algorithm and poor generalization ability, making it difficult to achieve good results in practical applications.

Method used

The ForegroundNet-DA model based on the domain adaptation mechanism is adopted, and the general object detection knowledge is migrated to complex background object detection tasks by introducing large-scale general object detection data sets to assist in training.

Benefits of technology

In a complex context, the detection performance of the model is improved, the average absolute error is reduced, and the enhancement-alignment index, structural index and weighted F index are improved, which significantly enhances the practical application value of the model.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115147639B_ABST
    Figure CN115147639B_ABST
Patent Text Reader

Abstract

The present invention discloses a method for target detection in a complex background based on a domain adaptation mechanism, including: obtaining a public general target detection dataset and a complex background target detection dataset, constructing a source domain training set, a target domain training set and a test set; constructing an artificial neural network model ForegroundNet-DA model based on a domain adaptation mechanism; pre-training some parameters of the ForegroundNet-DA model in the domain adaptation stage on the source domain training set and the target domain training set; training the ForegroundNet-DA model in the fine-tuning stage on the target domain training set; and evaluating the detection performance of the converged ForegroundNet-DA model on the constructed test set. Compared with all current complex background target detection algorithms, the present invention can achieve more accurate detection performance under the support of a smaller-scale dataset. The detection results of the present invention on the test set have a lower mean absolute error, and the enhanced alignment index, structural index and weighted F index are higher, which is a more accurate target detection algorithm in a complex background.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to a method for detecting an object under a complex background based on a domain adaptation mechanism, and is applicable to the technical field of detecting an object under a complex background in computer vision. Background Art

[0002] Object detection under complex background is a challenging task in the field of computer vision. Its purpose is to detect the foreground target of interest when the background of the image is relatively complex, and output the segmentation map of the foreground target as the detection result. Object detection under complex background has very high research significance and practical application value, and has a wide range of applications in the fields of medicine, agriculture, ocean, and military. Thanks to the development of deep learning technology, target detection methods based on deep neural networks have achieved great success in recent years, and a large number of efficient detection algorithms have been proposed. However, under complex backgrounds, images usually have characteristics such as cluttered colors, variable lighting conditions, and foreground targets with camouflage colors, which makes the foreground target of interest in the image easily blend into the background, bringing great difficulties to the detection work.

[0003] When general target detection algorithms are directly applied to target detection tasks under complex backgrounds, their performance usually decays significantly and the detection accuracy is greatly reduced. Therefore, it is necessary to design a special algorithm for target detection under complex backgrounds based on the characteristics of complex backgrounds and the physical meaning of detection methods. Currently, most of the existing target detection algorithms under complex backgrounds are optimized from aspects such as enhancing image features, multi-scale feature fusion, and introducing attention mechanisms. Although good detection performance has been achieved, due to the small size of the public data set for target detection tasks under complex backgrounds and the limitation of the size of training data, the detection performance of these algorithms is often not fully utilized, and the generalization ability is also greatly affected. However, in practical applications, it is very costly to label the collected data on a large scale in some professional fields, and the data size used for training is often smaller. Therefore, the problem of training data size not only limits the performance of target detection algorithms under complex backgrounds, but also affects the practical application value of these algorithms. Summary of the invention

[0004] In view of the problem that the performance of the target detection algorithm is limited due to the small scale of training data in the complex background target detection task, the present invention proposes a target detection method in a complex background based on a domain adaptation mechanism. The method constructs a ForegroundNet-DA model, introduces a large-scale general target detection data set to assist in training during the training process of the network model, and uses the domain adaptation mechanism to transfer the knowledge learned by the network model on the large-scale general target detection data set to the complex background target detection task. The present invention can improve the detection performance of the model when the scale of the complex background target detection data set is limited. The detection performance of the trained ForegroundNet-DA network model is better than that of other target detection algorithms in the complex background. While reducing the mean absolute error (MAE) index, it improves the enhanced-alignment measure (EM), structure measure (SM) and weighted F measure (WFM) of the detection result, indicating that the present invention can improve the target detection accuracy of the model when the scale of the training set is limited.

[0005] In order to achieve the above object, the present invention provides the following technical solutions:

[0006] A method for object detection in a complex background based on a domain adaptation mechanism, characterized by comprising the following steps:

[0007] Step S1: construct a source domain training set, a target domain training set, and a test set based on a public general object detection dataset and an object detection dataset under complex backgrounds;

[0008] Step S2: construct a ForegroundNet-DA network model based on domain adaptation mechanism;

[0009] Step S3: Performing supervised pre-training of the constructed ForegroundNet-DA model in the domain adaptation phase on the constructed source domain training set and target domain training set;

[0010] Step S4: Load some model parameter weights obtained in the domain adaptation training phase and solidify them, and perform fine-tuning training on the ForegroundNet-DA model on the constructed target domain training set to obtain a ForegroundNet-DA model that converges to the optimal performance;

[0011] Step S5: Evaluate the detection performance of the converged ForegroundNet-DA model on the constructed test set.

[0012] Furthermore, the step S1 specifically includes:

[0013] Step S101: Obtain a public general object detection dataset and a complex background object detection dataset. The general object detection dataset is the COCO 2017 dataset, and the complex background object detection dataset includes the datasets COD10K, CAMO, and CHAMELEON.

[0014] Step S102: the constructed source domain training set contains 118287 pairs of image label pairs, the constructed target domain training set contains 4040 pairs of image label pairs, the constructed validation set contains 101 pairs of image label pairs, and the constructed test set contains 2352 pairs of image label pairs.

[0015] Furthermore, the network model ForegroundNet model constructed in step S2 mainly includes: a feature extraction module, a foreground background proposal module, a feature generation module, a convolutional attention-based feature fusion decoding module and a domain classification module.

[0016] Furthermore, the step S3 specifically includes:

[0017] Step S301: Randomly extract training images from the source domain training set and the target domain training set for preprocessing. First, use the interpolation algorithm to resize the input image and the corresponding true label to H×W, then perform image data enhancement processing, and finally normalize the image and input it into the ForegroundNet-DA network model for supervised pre-training in the domain adaptation stage;

[0018] Step S302: The ForegroundNet-DA model extracts multi-scale abstract features through the feature extraction module, inputs the image features into the foreground background proposal module and the domain classification module, and outputs N layers of foreground background proposal results and N layers of domain classification results, respectively. and To represent, where the foreground background proposal results include the foreground target proposal results and the background target proposal results, that is

[0019] Step S303: Based on the proposal results and domain classification results, calculate the loss function L of the model in the domain adaptation stage DA , and its calculation method is shown in formula (1).

[0020]

[0021] in, represents the loss corresponding to the foreground background proposal result of the kth layer, represents the loss function corresponding to the classification result of the k-th domain. Their calculation expressions are shown in formula (2) and formula (3) respectively.

[0022]

[0023]

[0024] In the above formula, GT d represents the domain classification true label of the input data, and Respectively represent the k-th layer foreground target proposal results And background target proposal results The corresponding cross entropy loss weighted by structural information is, and Respectively represent the k-th layer foreground target proposal results And background target proposal results The corresponding intersection-over-union loss is weighted by structural information. Taking the foreground object detection result as an example, the calculation methods of cross entropy loss and intersection-over-union loss are shown in equations (4) and (5) respectively.

[0025]

[0026]

[0027] Among them, H represents the image height, W represents the image width, and GT fg (x,y) and They represent the true label of the foreground target and the value of the position coordinate (x, y) in the kth foreground target proposal result, respectively. γ is a parameter related to the structural weight. w(x, y) represents the structural weight corresponding to the position with coordinate (x, y). Its calculation method is shown in formula (6).

[0028]

[0029] Among them, A xy Represents the set of pixels around the pixel with coordinates (x, y);

[0030] Step S304: Loss function L based on the model in the domain adaptation stage DA , the stochastic gradient descent algorithm is used to optimize and update the network parameters of the ForegroundNet-DA model.

[0031] Furthermore, the step S4 specifically includes:

[0032] Step S401: Load the network parameters of the feature extraction module and the foreground background proposal module in the ForegroundNet-DA model after the supervised pre-training in the domain adaptation stage, and solidify the network parameter weights related to feature extraction so that the network parameters are not iterated during the training process of the fine-tuning stage;

[0033] Step S402: Only training data is randomly extracted from the target domain training set for preprocessing, and the input image and the corresponding true label are resized to H×W using an interpolation algorithm, followed by image data enhancement processing, and finally the image is normalized and input into the supervised training of the ForegroundNet-DA network model fine-tuning stage;

[0034] Step S403: The ForegroundNet-DA model extracts multi-scale abstract features through the feature extraction module. The image features are sequentially passed through the foreground background proposal module, the feature generation module, and the feature fusion decoding module based on convolutional attention, and the N-layer foreground background proposal results and the M-layer foreground target detection results are output. and To represent, where the foreground background proposal result Contains foreground object detection results And background target detection results

[0035] Step S404: Based on the N-layer proposal results and the M-layer foreground object detection results, calculate the loss function L of the model in the fine-tuning stage FT , which is calculated as follows:

[0036]

[0037] Step S405: Based on the loss function L in the fine-tuning stage FT , the stochastic gradient descent algorithm is used to iteratively update the network parameters of the ForegroundNet-DA model.

[0038] Step S406: The model gradually converges to the optimal performance during the fine-tuning phase. Then, the network parameters of the model are solidified, and the image to be detected is input for forward calculation. Among the output N+M layer detection results, the foreground target detection result with the largest scale is selected. As the final detection result of the model, pred M .

[0039] Furthermore, the step S5 specifically includes:

[0040] Step S501: read the images to be detected in the test set one by one, resize them to H×W using interpolation, then normalize the images, input them into the ForegroundNet-DA model that converges to the optimal performance for forward calculation, and output the corresponding detection result pred M ;

[0041] Step S502: Detect the foreground target based on the model M and the foreground target true label GT fg , calculate the objective evaluation indicators of the model on the test set, including: SM indicator, EM indicator, WFM indicator and MAE indicator.

[0042] Finally, the converged ForegroundNet-DA network model can be used to detect targets in complex backgrounds, input the image to be detected for forward calculation, and output the predicted foreground target segmentation map as the detection result.

[0043] The beneficial effect of the present invention is that the ForegroundNet-DA model constructed by the present invention can transfer the knowledge learned by the model on a large-scale general target detection dataset to a complex background target detection dataset through a domain adaptation mechanism, thereby improving the detection performance of the model on the test set. Compared with all current target detection models in complex backgrounds, the ForegroundNet-DA network model based on the domain adaptation mechanism proposed in the present invention can achieve more accurate detection results and improve the practical application value of the model when the size of the dataset is limited, which is specifically reflected in: the detection indicators of the model on the test set are overall better than all advanced models in the current field, and all detection performance evaluation indicators on the test set are significantly improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 This is a flowchart of the target detection method in a complex background based on the domain adaptation mechanism in Example 1.

[0045] Figure 2 This is the ForegroundNet-DA network model structure diagram in Example 1.

[0046] Figure 3 This is a diagram of the internal structure of the domain classifier in Example 1.

[0047] Figure 4 This is a flowchart of the training of the ForegroundNet-DA model in the domain adaptation stage in Example 1.

[0048] Figure 5 This is a flowchart of the training of the ForegroundNet-DA model in the fine-tuning stage in Example 1.

[0049] Figure 6 The following is a comparison of the detection performance of the method of the present invention in Example 1 with several currently most advanced methods in terms of evaluation indicators.

[0050] Figure 7 The visual comparison of the detection performance of the method of the present invention in Example 1 and several currently most advanced methods is shown. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0052] Example 1

[0053] See also Figure 1-Figure 7 ,This implementation provides a method for object detection in complex ,backgrounds based on domain adaptation mechanism.

[0054] Specifically, see Figure 1 , this method specifically includes:

[0055] Step S1: construct source domain training set, target domain training set and test set based on the public general object detection dataset COCO2017 dataset and object detection datasets under complex backgrounds, including: COD10K dataset, CAMO dataset and CHAMELEON dataset;

[0056] More specifically, the constructed source domain training set contains 118,287 pairs of image label pairs in the COCO2017 dataset; the constructed target domain training set contains 3,040 pairs of image label pairs in COD10K and 1,000 pairs of image label pairs in the CAMO dataset, totaling 4,040 data points; the constructed test set contains 2,026 pairs of image label pairs in COD10K, 250 pairs of image label pairs in the CAMO dataset, and 76 pairs of image label pairs in the CHAMELEON dataset, totaling 2,352 data points.

[0057] Step S2: construct a ForegroundNet-DA network model based on domain adaptation mechanism;

[0058] More specifically, the overall structure of the constructed ForegroundNet-DA model is as follows Figure 2As shown in the figure, the network model mainly consists of a feature extractor, a foreground-background proposal module, a feature generation module, a feature fusion decoding module based on convolutional attention, and a domain classification module. The feature extractor uses the Res2Net-50 model. The feature extractor extracts 4 layers of features of different scales based on the input image and inputs them into the foreground-background proposal module. The foreground-background proposal module mainly includes a receptive field enhancement module and a proposal module, which is responsible for predicting foreground-background targets based on 4 layers of features and outputting the proposal results. The feature generation module generates foreground features, background features, and edge features based on the proposed results, and inputs them into the feature fusion decoding module based on convolutional attention for feature fusion decoding. Each layer of decoded features is subjected to convolution operation for foreground target detection, and a total of 5 layers of foreground target detection results are output. The domain classification module is responsible for judging whether the image belongs to the source domain or the target domain based on the receptive field enhancement features output by the foreground and background proposal module, and outputs the 4-layer domain classification results.

[0059] More specifically, the domain classification module consists of a gradient flip layer and a domain classifier, which assists in implementing the domain adaptation mechanism during the training process of the domain adaptation stage. When the gradient flip layer calculates data forward, the input data and output data remain consistent, but when the gradient is back-propagated, the gradient of the domain classification module is reversed, and the output gradient sign is opposite to the input. The domain classifier is responsible for judging the domain of the image based on the input features. Its structure is as follows: Figure 3 As shown in the figure, the input features first pass through multiple layers of convolution, and then the features are flattened and the domain classification results are output through the fully connected layer.

[0060] Step S3: Perform supervised training of the ForegroundNet-DA model in the domain adaptation phase on the constructed source domain training set and target domain training set;

[0061] More specifically, the training process of the ForegroundNet-DA model in the domain adaptation stage is as follows Figure 4 As shown, including:

[0062] Step S301: Randomly extract training images from the source domain training set and the target domain training set for preprocessing. First, use the interpolation algorithm to resize the input image and the corresponding true label to 384×384, then perform image data enhancement processing such as randomly adjusting image attributes, random radiation transformation, and randomly erasing images. Finally, the data-enhanced image is normalized and input into the ForegroundNet-DA network model for supervised training in the domain adaptation stage.

[0063] Step S302: The ForegroundNet-DA model extracts multi-scale abstract features through the feature extraction module, inputs the image features into the foreground background proposal module and the domain classification module, proposes foreground objects and background objects and predicts domain categories based on the input features, and outputs 4 layers of foreground background proposal results and domain classification results, which are used respectively. and The foreground background proposal results include the foreground target proposal results and the background target proposal results, namely

[0064] Step S303: Based on the proposal results and domain classification results, calculate the loss function L of the model in the domain adaptation stage DA , and its calculation method is shown in formula (8).

[0065]

[0066] in, represents the loss corresponding to the foreground background proposal result of the kth layer, represents the loss function corresponding to the classification result of the k-th domain. Their calculation expressions are shown in equations (9) and (10) respectively.

[0067]

[0068]

[0069] In the above formula, GT d represents the domain classification true label of the input data, and Respectively represent the k-th layer foreground target proposal results And background target proposal results The corresponding cross entropy loss weighted by structural information is, and Respectively represent the k-th layer foreground target proposal results And background target proposal results The corresponding intersection-over-union loss is weighted by structural information. Taking the foreground object detection result as an example, the calculation methods of cross entropy loss and intersection-over-union loss are shown in equations (11) and (12).

[0070]

[0071]

[0072] Among them, H represents the image height, W represents the image width, and GT fg (x,y) and They represent the true label of the foreground target and the value of the position coordinate (x, y) in the kth foreground target proposal result, γ is a parameter related to the structural weight, and w(x, y) represents the structural weight corresponding to the position with coordinates (x, y). Its expression is as follows:

[0073]

[0074] Among them, A xy Represents the set of pixels around the pixel with coordinates (x, y);

[0075] Step S304: Loss function L based on the model in the domain adaptation stage DA , the stochastic gradient descent optimization algorithm is used to iteratively update the network parameters of the ForegroundNet-DA model.

[0076] Step S4: Load some model parameter weights obtained in the domain adaptation training phase and solidify them, and perform fine-tuning training on the ForegroundNet-DA model on the constructed target domain training set to obtain a ForegroundNet-DA model that converges to the optimal performance;

[0077] More specifically, the training process of the ForegroundNet-DA model is as follows Figure 5 As shown, including:

[0078] Step S401: Load the network parameters of the feature extraction module and the foreground background proposal module of the ForegroundNet-DA model after the supervised training in the domain adaptation stage, and solidify the network parameters related to feature extraction so that the network parameters are not iterated during the training process in the fine-tuning stage;

[0079] Step S402: Only training data is randomly extracted from the target domain training set for preprocessing, and the input image and the corresponding true label are resized to 384×384 using an interpolation algorithm, followed by image data enhancement processing, and finally the image is normalized and input into the supervised training of the ForegroundNet-DA network model fine-tuning stage;

[0080] Step S403: The ForegroundNet-DA model extracts multi-scale abstract features through the feature extraction module, inputs the image features into the foreground background proposal module to propose foreground objects and background objects, and outputs 4 layers of foreground background proposal results. To represent, the feature generation module proposes the result of the foreground target according to the kth layer And background target proposal results The edge confusing area is calculated, and the foreground feature, background feature and edge feature are generated. The feature fusion decoding module based on convolutional attention fuses and decodes these features, and outputs 5 layers of foreground target detection results. To express;

[0081] Step S404: Based on the 4-layer proposal results and the 5-layer foreground object detection results, calculate the loss function L of the model in the fine-tuning stage FT , which is calculated as follows:

[0082]

[0083] Step S405: Based on the loss function L in the fine-tuning stage FT , the stochastic gradient descent method is used to iteratively update the network parameters of the ForegroundNet-DA model.

[0084] Step S406: After the model gradually converges to the optimal performance, the network parameters of the model are solidified, and the image to be detected is input for forward calculation. Among the output 9-layer detection results, the foreground target detection result with the largest scale is selected. As the final detection result of the model, pred M .

[0085] Step S5: Perform detection on the constructed test set to evaluate the detection performance of the converged ForegroundNet-DA model.

[0086] More specifically, when testing the detection performance of the model, each image to be detected in the test set needs to be preprocessed, resized to 384×384, normalized, and then input into the converged ForegroundNet-DA model for forward calculation, and the detection result pred is output. M The detection results are resized to the original image size through the interpolation algorithm, compared with the true label, and the SM index, EM index, WFM index and MAE index corresponding to the image are calculated. The indexes of all images in the same dataset are averaged as the evaluation index of the ForegroundNet-DA model's detection performance on the dataset.

[0087] It should be noted that the smaller the mean absolute error MAE index is, the larger the structure index SM, the enhancement-alignment index EM and the weighted F index WFM are, and the more accurate the output detection result is. In addition, there are more than the above four evaluation indicators to measure the effect of target detection, as long as they can show the similarity or distinction between the predicted foreground target segmentation map and the true label.

[0088] Figure 6The comparison of the detection performance evaluation indicators of the proposed method and the advanced model in the field of complex background target detection on the test set is shown. Figure 6 From the numerical results shown, all evaluation indicators of the proposed method on the COD10K dataset and the CHAMELEON dataset exceed those of all current advanced models. On the CAMO dataset, the proposed method achieves optimal values ​​in the EM, WFM, and MAE indicators, with the EM indicator second only to the SINet-V2 model. Therefore, the overall detection performance of the proposed method on the test set exceeds that of all previous advanced models in the field, achieving the best detection performance.

[0089] Figure 7 The detection performance of the proposed method in this paper is compared with that of the advanced model in the field of complex background target detection in terms of image effects. Figure 7 By comparing the first and fourth rows of images, we can see that the method of the present invention can effectively overcome the interference of complex backgrounds and accurately and completely detect foreground targets. Figure 7 By comparing the second and third rows of the image, it can be seen that the method of the present invention can accurately distinguish the area where the foreground target is located from the background when the color and texture of the foreground target are similar to those of the background. Figure 7 From the 5th and 6th rows of images, we can see that compared with other methods, the method of the present invention can accurately distinguish between occluders and foreground targets, and accurately detect the edge of the target in the presence of occlusion. Figure 7 As can be seen from the 7th row of images, the invented method can detect the foreground target in detail and completely preserve the details of the area where the foreground target is located. Figure 7 It can be seen from the 8th row of images that the method of the present invention can accurately detect small targets. In summary, the method of the present invention can accurately detect foreground targets under various interferences caused by complex backgrounds, and significantly improves the detection performance when the data set size is limited.

[0090] The matters not described in detail in the present invention are all known technologies to those skilled in the art.

[0091] The preferred specific embodiments of the present invention are described in detail above. It should be understood that a person skilled in the art can make many modifications and changes based on the concept of the present invention without creative work. Therefore, any technical solution that can be obtained by a person skilled in the art through logical analysis, reasoning or limited experiments based on the concept of the present invention on the basis of the prior art should be within the scope of protection determined by the claims.

Claims

1. A method for object detection in complex background based on domain adaptation mechanism, characterized in that: The method comprises the following steps: Step S1: construct a source domain training set, a target domain training set, and a test set based on a public general object detection dataset and an object detection dataset under complex backgrounds; Step S2: construct a ForegroundNet-DA network model based on domain adaptation mechanism; Step S3: Performing supervised pre-training of the constructed ForegroundNet-DA network model in the domain adaptation phase on the constructed source domain training set and target domain training set; Step S4: Load some model parameter weights obtained in the domain adaptation training phase and solidify them, and perform fine-tuning training on the ForegroundNet-DA network model on the constructed target domain training set to obtain a ForegroundNet-DA network model that converges to the optimal performance; Step S5: Evaluate the detection performance of the converged ForegroundNet-DA network model on the constructed test set; In step S1, the general object detection dataset includes the COCO 2017 dataset, and the complex background object detection dataset includes the COD10K, CAMO and CHAMELEON datasets; The ForegroundNet-DA network model based on the domain adaptation mechanism constructed in step S2 includes: a feature extraction module, a foreground background proposal module, a feature generation module, a feature fusion decoding module based on convolutional attention, and a domain classification module; The step S3 specifically includes: Step S301: training images are randomly selected from the source domain training set and the target domain training set for preprocessing. First, the input image and the corresponding true label are resized to H×W using an interpolation algorithm, where H represents the image height and W represents the image width. Then, image data enhancement is performed. Finally, the image is normalized and input into the ForegroundNet-DA network model for supervised pre-training in the domain adaptation stage. Step S302: The ForegroundNet-DA network model extracts multi-scale abstract features through the feature extraction module, inputs the image features into the foreground background proposal module and the domain classification module, and outputs N layers of foreground background proposal results and N layers of domain classification results, respectively. and To represent, k = 1, 2, ..., N, where the foreground background proposal results include the foreground target proposal results and the background target proposal results, that is, Step S303: Based on the proposal results and domain classification results, calculate the loss function L of the model in the domain adaptation stage DA , the calculation method is shown in formula (1), in, represents the loss corresponding to the foreground background proposal result of the kth layer, represents the loss corresponding to the classification result of the k-th domain; their calculation expressions are shown in formula (2) and formula (3) respectively. In the above formula, GT d represents the domain classification true label of the input data, and Respectively represent the k-th layer foreground target proposal results And background target proposal results The corresponding cross entropy loss weighted by structural information is, and Respectively represent the k-th layer foreground target proposal results And background target proposal results The corresponding intersection-over-union loss weighted by structural information; Step S304: Loss function L based on the model in the domain adaptation stage DA , the stochastic gradient descent optimization algorithm is used to iteratively update the network parameters of the ForegroundNet-DA model.

2. According to claim 1, a method for object detection in a complex background based on a domain adaptation mechanism, characterized in that: Taking the foreground target detection result as an example, the calculation methods of cross entropy loss and intersection-over-union loss are shown in formula (2) and formula (3) respectively. Among them, GT fg (x,y) and They represent the true label of the foreground target and the value of the position coordinate (x, y) in the kth foreground target proposal result, γ is a parameter related to the structural weight, and w(x, y) represents the structural weight corresponding to the position with coordinates (x, y). Its expression is as follows: Among them, A xy Represents the set of pixels surrounding the pixel with coordinates (x, y).

3. According to claim 1, a method for object detection in a complex background based on a domain adaptation mechanism is characterized in that: The step S4 specifically includes: Step S401: Load the network parameters of the feature extraction module and the foreground and background proposal module of the ForegroundNet-DA model after supervised training in the domain adaptation phase, and solidify them so that the network parameters are not iterated during the training process of the fine-tuning phase; Step S402: Only training data is randomly extracted from the target domain training set for preprocessing, and the input image and the corresponding true label are resized to H×W using an interpolation algorithm, where H represents the image height and W represents the image width. Image data enhancement is then performed, and finally the image is normalized and input into the supervised training of the ForegroundNet-DA network model fine-tuning stage; Step S403: The ForegroundNet-DA network model extracts multi-scale abstract features through the feature extraction module. The image features are sequentially passed through the foreground background proposal module, the feature generation module, and the feature fusion decoding module based on convolutional attention, and the N-layer foreground background proposal results are output. To represent, where k = 1, 2..., N; the M-layer foreground target detection results are expressed as To represent, where k = N+1, N+2, ..., N+M, foreground background proposal results Contains foreground object detection results And background target detection results Step S404: Based on the N-layer proposal results and the M-layer foreground object detection results, calculate the loss function L of the model in the fine-tuning stage FT , which is calculated as follows: in, or wbce-bg The calculation method of is shown in formula (4). and The calculation method of is shown in formula (5); Step S405: Based on the loss function L in the fine-tuning stage FT , the stochastic gradient descent method is used to iteratively update the network parameters of the ForegroundNet-DA network model; Step S406: After the model gradually converges to the optimal performance during the fine-tuning phase, the network parameters of the model are solidified, and the image to be detected is input for forward calculation. Among the output N+M layer detection results, the foreground target detection result with the largest scale is selected. As the final detection result of the model, pred M .

4. According to claim 1, a method for object detection in a complex background based on a domain adaptation mechanism is characterized in that: The step S5 specifically includes: Step S501: read the images to be detected in the test set one by one, resize them to H×W using interpolation, then normalize the images, input them into the ForegroundNet-DA model that converges to the optimal performance for forward calculation, and output the corresponding detection result pred M ; Step S502: Detect the foreground target based on the model's pred M and the foreground target true label GT fg , calculate the objective evaluation index of the model on the test set.

5. According to claim 4, a method for object detection in a complex background based on a domain adaptation mechanism is characterized in that: The objective evaluation indicators include: SM indicator, EM indicator, WFM indicator and MAE indicator.

Citation Information

Patent Citations

  • Domain-adaptive foggy day image target detection method and device

    CN112633149A

  • Unsupervised domain adaptive target detection method based on uncertainty perception

    CN114139616A