Domain-adaptive object detection method based on large and small model feature collaboration

Abstract

The present invention relates to the field of target detection technology, and in particular, provides a domain-adaptive target detection method based on large- and small-model feature collaboration. The method comprises building a global-local enhancement network; building a large- and small-model collaborative feature interaction enhancement network based on the global-local enhancement network; connecting the global-local enhancement network and the feature interaction enhancement network to a detection head to construct a domain-adaptive target detection model; training the domain-adaptive target detection model, and encapsulating and deploying the trained domain-adaptive target detection model to achieve target recognition. This method improves the domain-adaptive capability of the model in different scenarios and achieves more efficient target recognition and positioning by effectively reducing the distribution differences between the source domain and the target domain.

Description

Technical Field

[0001] The present invention relates to the field of target detection technology, and in particular to a domain adaptive target detection method based on large and small model feature collaboration. Background Art

[0002] Domain adaptation is an important transfer learning strategy designed to address the problem of inconsistent distributions between training and test data. With the rapid development of deep learning technology, domain adaptation is increasingly being used in computer vision, particularly in object detection tasks. Object detection typically relies on large amounts of labeled data for model training, but in many practical scenarios, obtaining high-quality labeled data is often difficult and costly.

[0003] Domain adaptation techniques leverage knowledge from the source domain (labeled data) to improve detection performance in the target domain (unlabeled data). The core of this approach is to reduce the distribution discrepancy between the source and target domains to achieve better transfer results. Traditional domain adaptation methods primarily focus on feature-level alignment, minimizing the gap between the source and target domain feature distributions to improve the model's generalization ability in new domains.

[0004] However, domain adaptation still faces several challenges in practical applications. First, object morphology, background complexity, and environmental conditions can vary significantly across domains, making simple feature alignment methods ineffective. Second, object size and scale variations also affect the performance of detection models. Objects of different sizes may appear differently in the same image, necessitating the integration and utilization of multi-scale features. Summary of the Invention

[0005] In view of this, the present invention provides a domain adaptive target detection method based on large and small model feature collaboration to improve the domain adaptation capability of the model in different scenarios, and achieve more efficient target recognition and positioning by effectively reducing the distribution difference between the source domain and the target domain.

[0006] In a first aspect, the present invention provides a domain adaptive object detection method based on large and small model feature collaboration, the method comprising:

[0007] Step S1: Building a global and local enhancement network;

[0008] Step S2: Based on the global-local enhancement network in step S1, a feature interaction enhancement network for collaboration between large and small models is constructed;

[0009] Step S3: Connect the global-local enhancement network in step S1 and the feature interaction enhancement network in step S2 to the detection head to build a domain-adaptive object detection model;

[0010] Step S4: training the domain adaptive target detection model constructed in step S3, and encapsulating and deploying the trained domain adaptive target detection model to achieve target recognition.

[0011] Optionally, step S1 includes:

[0012] The global-local enhancement network consists of a global enhancement module and a local enhancement module; the global enhancement module and the local enhancement module enhance the image in parallel;

[0013] The global enhancement module includes color space conversion, initial exposure estimation, fractional differential mask and color space inverse conversion; the local enhancement module includes random target border extraction and multi-scale retinal enhancement with color restoration.

[0014] Optionally, the global enhancement module comprises:

[0015] a. Convert the source domain image to be enhanced from RGB color space to YCbCr color space. The converted luminance component is extracted as the initial image I′(x,y). The formula for generating the initial image is:

[0016] I'(x,y)=0.299R(x,y)+0.587G(x,y)+0.114B(x,y);

[0017] b. Use the Retinex algorithm based on fast guided filtering to process the initial image I′(x,y) and estimate the initial illumination component L′(x,y). According to the Retinex theory, the reflection component R′(x,y) can be calculated using the following formula:

[0018]

[0019] Where GF represents the fast guided filter function, r is the control parameter of the filter radius, is a parameter that controls the filter strength, τ represents a parameter for adjusting the edge preservation characteristics, ω represents a parameter for controlling the filter response, and δ k Represents the weight coefficient of each calculation, d x and d y Represents the displacement in the x and y directions, ∈ k represents the disturbance term that changes with k, which is used to avoid the situation where the denominator is zero;

[0020] c. Use adaptive fractional differential mask to process the reflection component R′(x,y) to obtain the enhanced reflection component R″(x,y). The generation formula is:

[0021] R″(x,y)=R′(x,y)×M(x,y,α);

[0022]

[0023] Where M(x,y,α) represents the fractional differential mask with fractional order α; R″(xk,yk) represents the pixel value of the local area of ​​the image, which is used to perform the convolution operation;

[0024] d. Combine the enhanced reflection component R″(x,y) with the original color difference components Cb(x,y) and Cr(x,y), and convert the YCbCr color space back to RGB color space to obtain the final global enhanced image I g (x,y).

[0025] Optionally, the local enhancement module comprises:

[0026] e. Randomly set a pixel coordinate as the center point within the provided real border and generate a horizontal or vertical border to be enhanced around it;

[0027] f. Image I in the border area to be enhanced b (x,y) is enhanced by the multi-scale retinal algorithm to obtain the final local enhanced image I l (x,y), the generating formula is:

[0028]

[0029] Among them, ω n is the weight of small-scale, medium-scale, and large-scale brightness estimation; G n (x,y) is the Gaussian filter function;

[0030] g. The image finally obtained by the global local enhancement network is represented as:

[0031] I a =βI g +(1-β)I l ;

[0032] where β represents the fusion factor.

[0033] Optionally, step S2 includes:

[0034] The feature interaction enhancement network consists of a feature interaction module and a feature causal attention alignment module.

[0035] Optionally, the feature interaction module comprises:

[0036] h. Extract features from the encoder of the segmentation model and the encoder of Faster RCNN respectively, and extract the features F of the segmentation model. S Through MLP and basic features FR To perform dimension alignment, the expression is:

[0037]

[0038] Among them, F′ S Represents the auxiliary features after MLP alignment, ensuring that they are consistent with the basic features F R Consistent in the number of channels;

[0039] i. Align the auxiliary features F' S and basic features F R Apply the self-attention mechanism respectively, and its expression is:

[0040] F″ S =SA(F′ S ),F″ R =SA(F R );

[0041] j. Introduce the cross attention mechanism to add auxiliary features F″ S and basic feature F″ R To interact, the expression is:

[0042]

[0043] Where Q, K, and V represent query, key, and value matrices respectively, and d k Indicates the dimension of the key, used to scale the result of the dot product;

[0044] k, the auxiliary feature F″ after self-attention processing S , basic feature F″ R and cross-attention fusion feature F C Aggregate and generate the final interaction feature, which is expressed as:

[0045]

[0046] where ω i 、v i , γ i represents the non-negative weight of each feature.

[0047] Optionally, the feature causal attention alignment module comprises:

[0048] m. Extract feature maps through the Faster RCNN feature extraction network and the segmentation model feature extraction network respectively; Based on these feature maps, calculate the corresponding attention map A R (I a ) and A S (I a ), whose expression is:

[0049] A R (I a )=σ(f R (I a )),A S (I a )=σ(f S (I a ));

[0050] where f R ,f S They represent the faster RCNN and segmentation model feature extraction networks respectively, and σ represents the Sigmoid function, which is used to normalize the feature map to the range of [0,1].

[0051] n. The attention map is processed by the binarization function B(·), and the similarity between them is measured using the Dice loss, which is expressed as:

[0052]

[0053] Where B(·) represents the pixel-level binarization operation.

[0054] Optionally, step S3 includes:

[0055] The detection head is sequentially connected to a global enhancement module, a local enhancement module, a feature interaction module, and a feature causal attention alignment module to construct a domain adaptive object detection model.

[0056] Optionally, step S4 includes:

[0057] v. The domain adaptive object detection model adopts a teacher-student network and updates weights through exponential moving average, and its expression is:

[0058] θ t (t) = αθ t (t-1)+(1-α)θ s (t);

[0059] w. During training, the supervised loss supLoss of the student network comes from the region proposal network and the ROI pooling detection head; when pre-training is completed, the weights of the student network are transferred to the teacher network; then, the unsupervised loss usupLoss is guided by the matching pseudo-labels generated by the teacher network, and its expression is:

[0060]

[0061] where f s (·) represents the detector of the student network, f s(·) denotes the detector of the teacher network;

[0062] s, the total loss function TotalLoss during training, its expression is:

[0063] TotalLoss=supLoss+μ1usupLoss+μ2CLoss;

[0064] t. Input the training set, validation set, and test set into the constructed target detection model and train it. Select the target detection model that has been trained for 90K rounds for TensorRT acceleration and encapsulate it into a DLL file.

[0065] In the technical solution provided by the present invention, the method includes building a global-local enhancement network; building a feature interaction enhancement network for collaborative large and small models based on the global-local enhancement network; connecting the global-local enhancement network and the feature interaction enhancement network with the detection head to construct a domain-adaptive target detection model; training the domain-adaptive target detection model, and encapsulating and deploying the trained domain-adaptive target detection model to achieve target recognition. This method improves the domain adaptation capability of the model in different scenarios, and achieves more efficient target recognition and positioning by effectively reducing the distribution differences between the source domain and the target domain. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0067] Figure 1 A flowchart of a domain-adaptive object detection method provided by an embodiment of the present invention;

[0068] Figure 2 A schematic diagram of a global-local enhancement network provided by an embodiment of the present invention;

[0069] Figure 3 A schematic diagram of a domain-adaptive object detection model provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0070] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0071] It should be understood that the embodiments described are only a portion of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without creative work are within the scope of protection of the present invention.

[0072] The terms used in the embodiments of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The singular forms "a", "the" and "the" used in the embodiments of the present invention are also intended to include plural forms, unless the context clearly indicates other meanings.

[0073] It should be understood that the term "and / or" as used herein simply describes a relationship between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A alone, A and B together, or B alone. Furthermore, the character " / " in this document generally indicates an "or" relationship between the associated objects.

[0074] The word "if," as used herein, may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to the determination" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)," depending on the context.

[0075] Figure 1 A flowchart of a domain adaptive object detection method provided by an embodiment of the present invention is shown in FIG. Figure 1 As shown, the method includes:

[0076] Step S1: Build a global and local enhancement network.

[0077] In the embodiment of the present invention, Figure 2 As shown, step S1 includes:

[0078] The global-local enhancement network consists of a global enhancement module and a local enhancement module; the global enhancement module and the local enhancement module enhance the image in parallel;

[0079] The global enhancement module includes color space conversion, initial exposure estimation, fractional differential mask and color space inverse conversion; the local enhancement module includes random target border extraction and multi-scale retinal enhancement with color restoration.

[0080] In the embodiment of the present invention, the global enhancement module comprises:

[0081] a. Convert the source domain image to be enhanced from RGB color space to YCbCr color space. The converted luminance component is extracted as the initial image I′(x,y). The formula for generating the initial image is:

[0082] I'(x,y)=0.299R(x,y)+0.587G(x,y)+0.114B(x,y);

[0083] b. Use the Retinex algorithm based on fast guided filtering to process the initial image I′(x,y) and estimate the initial illumination component L′(x,y). According to the Retinex theory, the reflection component R′(x,y) can be calculated using the following formula:

[0084]

[0085] Where GF represents the fast guided filter function, r is the control parameter of the filter radius, is a parameter that controls the filter strength, τ represents a parameter for adjusting the edge preservation characteristics, ω represents a parameter for controlling the filter response, and δ k Represents the weight coefficient of each calculation, d x and d y Represents the displacement in the x and y directions, ∈ k represents the disturbance term that changes with k, which is used to avoid the situation where the denominator is zero;

[0086] c. Use adaptive fractional differential mask to process the reflection component R′(x,y) to obtain the enhanced reflection component R″(x,y). The generation formula is:

[0087] R″(x,y)=R′(x,y)×M(x,y,α);

[0088]

[0089] Where M(x,y,α) represents the fractional differential mask with fractional order α; R″(xk,yk) represents the pixel value of the local area of ​​the image, which is used to perform the convolution operation;

[0090] d. Combine the enhanced reflection component R″(x,y) with the original color difference components Cb(x,y) and Cr(x,y), and convert the YCbCr color space back to the RGB color space to obtain the final global enhanced image I g (x,y).

[0091] In the embodiment of the present invention, the local enhancement module comprises:

[0092] e. Randomly set a pixel coordinate as the center point within the provided real border and generate a horizontal or vertical border to be enhanced around it;

[0093] f. Image I in the border area to be enhanced b (x,y) is enhanced by the multi-scale retinal algorithm to obtain the final local enhanced image I l (x,y), the generating formula is:

[0094]

[0095] Among them, ω n is the weight of small-scale, medium-scale, and large-scale brightness estimation; G n (x,y) is the Gaussian filter function;

[0096] g. The image finally obtained by the global local enhancement network is represented as:

[0097] I a =βI g +(1-β)I l ;

[0098] where β represents the fusion factor.

[0099] Step S2: Based on the global-local enhancement network in step S1, a feature interaction enhancement network for collaborative large and small models is constructed.

[0100] In the embodiment of the present invention, Figure 3 As shown, step S2 includes:

[0101] The feature interaction enhancement network consists of a feature interaction module and a feature causal attention alignment module.

[0102] In the embodiment of the present invention, the feature interaction module comprises:

[0103] h. Extract features from the encoder of the segmentation model and the encoder of the Faster RCNN, i.e., the 50th layer of the residual network, and extract the features F of the segmentation model. S Through the multi-layer perceptron MLP and basic features F R To perform dimension alignment, the expression is:

[0104]

[0105] Among them, F′ S Represents the auxiliary features after MLP alignment, ensuring that they are consistent with the basic features F R Consistent in the number of channels;

[0106] i. Align the auxiliary features F' Sand basic features F R Apply the self-attention mechanism respectively, and its expression is:

[0107] F″ S =SA(F′ S ),F″ R =SA(F R );

[0108] j. Introduce the cross attention mechanism to add auxiliary features F″ S and basic feature F″ R To interact, the expression is:

[0109]

[0110] Where Q, K, and V represent query, key, and value matrices respectively, and d k Indicates the dimension of the key, used to scale the result of the dot product;

[0111] k, the auxiliary feature F″ after self-attention processing S , basic feature F″ R and cross-attention fusion feature F C Aggregate and generate the final interaction feature, which is expressed as:

[0112]

[0113] where ω i 、v i , γ i represents the non-negative weight of each feature.

[0114] In the embodiment of the present invention, the feature causal attention alignment module comprises:

[0115] m. Extract feature maps through the Faster RCNN feature extraction network and the segmentation model feature extraction network respectively; Based on these feature maps, calculate the corresponding attention map A R (I a ) and A S (I a ), whose expression is:

[0116] A R (I a )=σ(f R (I a )0,A S (I a )=σ(f S (I a ));

[0117] where f R ,fS They represent the faster RCNN and segmentation model feature extraction networks respectively, and σ represents the Sigmoid function, which is used to normalize the feature map to the range of [0,1].

[0118] n. The attention map is processed by the binarization function B(·), and the similarity between them is measured using the Dice loss, which is expressed as:

[0119]

[0120] Where B(·) represents the pixel-level binarization operation.

[0121] Step S3: Connect the global-local enhancement network in step S1 and the feature interaction enhancement network in step S2 to the detection head to build a domain-adaptive object detection model.

[0122] In the embodiment of the present invention, step S3 includes:

[0123] The detection head is sequentially connected to a global enhancement module, a local enhancement module, a feature interaction module, and a feature causal attention alignment module to construct a domain adaptive object detection model.

[0124] Step S4: training the domain adaptive target detection model constructed in step S3, and encapsulating and deploying the trained domain adaptive target detection model to achieve target recognition.

[0125] In the embodiment of the present invention, step S4 includes:

[0126] v. The domain adaptive object detection model adopts a teacher-student network and updates weights through exponential moving average, and its expression is:

[0127] θ t (t) = αθ t (t-1)+(1-α)θ s (t);

[0128] w. During training, the supervised loss supLoss of the student network comes from the region proposal network and the ROI pooling detection head; when pre-training is completed, the weights of the student network are transferred to the teacher network; then, the unsupervised loss usupLoss is guided by the matching pseudo-labels generated by the teacher network, and its expression is:

[0129]

[0130] where f s (·) represents the detector of the student network, f s (·) denotes the detector of the teacher network;

[0131] s, the total loss function TotalLoss during training, its expression is:

[0132] TotalLoss=supLoss+μ1usupLoss+μ2CLoss;

[0133] t. Input the training set, validation set, and test set into the constructed target detection model and train it. Select the target detection model that has been trained for 90K rounds for TensorRT acceleration and encapsulate it into a DLL file.

[0134] By combining the features of different models, this paper enhances the adaptability of the base model to various targets while simultaneously improving detection performance in the target domain. This approach not only effectively reduces the feature differences between the source and target domains but also achieves better detection results in multiple scenarios, providing new insights and paths for the development of domain-adaptive object detection technology.

[0135] In the technical solution provided by the present invention, the method includes building a global-local enhancement network; building a feature interaction enhancement network for collaborative large and small models based on the global-local enhancement network; connecting the global-local enhancement network and the feature interaction enhancement network with the detection head to construct a domain-adaptive target detection model; training the domain-adaptive target detection model, and encapsulating and deploying the trained domain-adaptive target detection model to achieve target recognition. This method improves the domain adaptation capability of the model in different scenarios, and achieves more efficient target recognition and positioning by effectively reducing the distribution differences between the source domain and the target domain.

[0136] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A domain-adaptive object detection method based on large and small model feature collaboration, characterized in that: The method comprises: Step S1: Building a global and local enhancement network; Step S2: Based on the global-local enhancement network in step S1, a feature interaction enhancement network for collaboration between large and small models is constructed; Step S3: Connect the global-local enhancement network in step S1 and the feature interaction enhancement network in step S2 to the detection head to build a domain-adaptive object detection model; Step S4: training the domain adaptive object detection model constructed in step S3, and encapsulating and deploying the trained domain adaptive object detection model to achieve object recognition; The step S1 comprises: The global-local enhancement network consists of a global enhancement module and a local enhancement module; the global enhancement module and the local enhancement module enhance the image in parallel; The global enhancement module includes color space conversion, initial exposure estimation, fractional differential masking, and color space inverse conversion; the local enhancement module includes random target border extraction and multi-scale retinal enhancement with color restoration; The step S2 comprises: The feature interaction enhancement network consists of a feature interaction module and a feature causal attention alignment module; The feature interaction module comprises: h. Extract features from the encoder of the segmentation model and the encoder of Faster RCNN respectively, and extract the features of the segmentation model. Through MLP and basic features To perform dimension alignment, the expression is: ； in, Represents the auxiliary features after MLP alignment to ensure that they are consistent with the basic features Consistent in the number of channels; i. Align the auxiliary features and basic features Apply the self-attention mechanism respectively, and its expression is: ； j. Introduce the cross attention mechanism to take auxiliary features and basic features To interact, the expression is: ； Where Q, K, and V represent query, key, and value matrices, respectively. Indicates the dimension of the key, used to scale the result of the dot product; k, auxiliary features after self-attention processing , basic features and cross-attention fusion features Aggregate and generate the final interaction feature, which is expressed as: ； in 、 、 Represents the non-negative weight of each feature; The feature causal attention alignment module consists of: m. Extract feature maps through the Faster RCNN feature extraction network and the segmentation model feature extraction network respectively; based on these feature maps, calculate the corresponding attention map and , whose expression is: ； in Respectively represent faster RCNN and segmentation of all large model feature extraction networks, Represents the Sigmoid function, which is used to normalize the feature map to the range of [0, 1]; n, through the binarization function The attention maps are processed and the Dice loss is used to measure the similarity between them, which is expressed as: ； in, Represents pixel-level binarization operation.

2. The method according to claim 1, characterized in that The global enhancement module comprises: a. Convert the source domain image to be enhanced from RGB color space to YCbCr color space, and extract the brightness component after conversion as the initial image ; The formula for generating the initial image is: ； b. Use the Retinex algorithm based on fast guided filtering to resize the initial image Processing to estimate the initial exposure component ; According to Retinex theory, the reflection component It can be calculated by the following formula: ； ； Where GF represents the fast guided filter function, r is the control parameter of the filter radius, is a parameter that controls the filter strength, represents the parameters used to adjust the edge preservation characteristics, is used to control the filter response, represents the weight coefficient of each calculation, and represents the displacement in the x and y directions, represents the disturbance term that changes with k, which is used to avoid the situation where the denominator is zero; c. Adopting adaptive fractional differential mask to reflect the component Processing is performed to obtain the enhanced reflection component , the generating formula is: ； ； ； in, Indicates the application of fractional order Fractional differential mask of ; Represents the pixel value of the local area of ​​the image, used to perform convolution operations; d. The enhanced reflection component With the original color difference component and Combine and convert the YCbCr color space back to RGB color space to obtain the final global enhanced image .

3. The method according to claim 1, characterized in that The local enhancement module comprises: e. Randomly set a pixel coordinate as the center point within the provided real border and generate a horizontal or vertical border to be enhanced around it; f. Image of enhanced border area Enhanced by multi-scale retinal algorithm, the final local enhanced image is obtained , the generating formula is: ； in, are the weights of small-scale, medium-scale, and large-scale brightness estimates; is the Gaussian filter function; g. The image finally obtained by the global local enhancement network is represented as: ； in Represents the fusion factor.

4. The method according to claim 1, wherein The step S3 comprises: The detection head is sequentially connected to a global enhancement module, a local enhancement module, a feature interaction module, and a feature causal attention alignment module to construct a domain adaptive object detection model.

5. The method according to claim 1, characterized in that The step S4 comprises: v. The domain adaptive object detection model adopts a teacher-student network and updates weights through exponential moving average, and its expression is: ； w. During training, the supervised loss supLoss of the student network comes from the region proposal network and the ROI pooling detection head; when pre-training is completed, the weights of the student network are transferred to the teacher network; then, the unsupervised loss usupLoss is guided by the matching pseudo-labels generated by the teacher network, and its expression is: ； ； in represents the detector of the student network, represents the detector of the teacher network; s, total loss function during training , whose expression is: ； t. Input the training set, validation set, and test set into the constructed target detection model and train it. Select the target detection model that has been trained for 90K rounds for TensorRT acceleration and encapsulate it into a DLL file.

Images