Target detection model optimization method and device, electronic equipment and readable storage medium

Abstract

The application discloses a target detection model optimization method and device, electronic equipment and a readable storage medium. The target detection model comprises a feature extraction unit, a domain feature decoupling unit and a target detection unit. The method comprises the following steps: acquiring training image samples of different data distribution domains and domain information features of data distribution domains to which the training image samples belong; performing feature extraction on the training image samples by using the feature extraction unit sharing network parameters to obtain domain-shared image features; performing domain decoupling on the domain-shared image features by inputting the domain information features and the domain-shared image features into the domain feature decoupling unit together to obtain domain-decoupled features; performing target detection on the domain-decoupled features by using the target detection unit to obtain a target detection result; and optimizing the target detection model according to a target detection loss corresponding to the target detection result. The application solves the technical problem that the target detection model is seriously redundant for training image samples of different data distribution domains.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, and in particular to a method, apparatus, electronic device and readable storage medium for optimizing an object detection model. Background Technology

[0002] With the continuous development of artificial intelligence technology, the application fields of object detection are becoming more and more extensive. Due to the different environments, heights, and angles of the current large number of monitoring devices deployed, a massive amount of cross-domain data is introduced. Currently, for this cross-domain data, targeted single-domain object detection models are usually trained, and then these single-domain object detection models are used to detect objects in the image data of each domain separately, resulting in serious model redundancy. Summary of the Invention

[0003] The main objective of this application is to provide a method, apparatus, electronic device, and readable storage medium for optimizing object detection models, aiming to solve the technical problem of severe redundancy in object detection models for training image samples with different data distribution domains in the prior art.

[0004] To achieve the above objectives, this application provides a method for optimizing a target detection model. The target detection model includes a feature extraction unit, a domain feature decoupling unit, and a target detection unit. The optimization method for the target detection model includes:

[0005] Obtain training image samples from different data distribution domains and the domain information features of the data distribution domain to which the training image samples belong;

[0006] The feature extraction unit extracts features from the training image samples to obtain domain-shared image features, wherein training image samples from different data distribution domains share the network parameters of the feature extraction unit.

[0007] By inputting the domain information features and the domain shared image features into the domain feature decoupling unit, the domain shared image features are decoupled to obtain domain decoupling features.

[0008] By inputting the domain decoupling features into the target detection unit, target detection is performed on the training image samples to obtain the target detection result;

[0009] The target detection model is optimized based on the target detection loss obtained from the target detection results.

[0010] Optionally, the step of performing domain decoupling on the domain-shared image features to obtain domain-decoupled features includes:

[0011] Based on the domain information features and the domain-shared image features, construct channel attention weights based on domain information;

[0012] Based on the channel attention weights, the domain-shared image features are decoupled to obtain the domain-decoupled features.

[0013] Optionally, the domain feature decoupling unit includes a first encoder and a second encoder.

[0014] The step of constructing channel attention weights based on domain information according to the domain information features and the domain-shared image features includes:

[0015] By inputting the domain information features into the first encoder, the domain information features are encoded to obtain the domain information encoded features;

[0016] The domain information encoding features are aggregated with the domain shared image features to obtain aggregated domain information features;

[0017] The channel attention weights are obtained by inputting the aggregated domain information features into the second encoder and encoding the aggregated domain information features.

[0018] Optionally, the step of performing domain decoupling on the domain-shared image features based on the channel attention weights to obtain domain-decoupled features includes:

[0019] According to the preset constraint function, the channel attention weights are converted into reweighted parameters;

[0020] According to the reweighting parameters, the output channels of the domain-shared image features are reweighted to obtain the domain-decoupled features. The reweighting parameters are used to enhance the output feature values ​​of output channels that are strongly correlated with domain information and to suppress the output feature values ​​of output channels that are weakly correlated with domain information.

[0021] Optionally, the step of optimizing the target detection model based on the target detection loss obtained from the target detection result includes:

[0022] If the target detection loss converges, the target detection model is determined to be optimized.

[0023] If the target detection loss does not converge, then the feature extraction unit, the domain feature decoupling unit, the target detection unit, and the domain information features are updated by backpropagation based on the gradient calculated by the target detection loss.

[0024] Return to the execution steps: Obtain training image samples from different data distribution domains and the domain information features of the data distribution domain to which the training image samples belong, until the convergence of the target detection loss is detected.

[0025] Optionally, the step of optimizing the target detection model based on the target detection loss obtained from the target detection result includes:

[0026] If the target detection loss converges, the target detection model is determined to be optimized.

[0027] If the target detection loss does not converge, then the domain feature decoupling unit and the domain information feature are updated by backpropagation based on the gradient calculated by the target detection loss.

[0028] Return to the execution steps: Obtain training image samples from different data distribution domains and the domain information features of the data distribution domain to which the training image samples belong, until the convergence of the target detection loss is detected.

[0029] Optionally, obtaining the domain information features of the data distribution domain to which the training image samples belong includes:

[0030] Obtain the domain information of the data distribution domain to which the training image samples belong;

[0031] Domain information features are extracted from the domain information to obtain the domain information features corresponding to the domain.

[0032] This application also provides a target detection model optimization device, wherein the target detection model includes a feature extraction unit, a domain feature decoupling unit, and a target detection unit, and the target detection model optimization device includes:

[0033] The acquisition module is used to acquire training image samples from different data distribution domains and the domain information features of the data distribution domain to which the training image samples belong;

[0034] A domain-shared feature extraction module is used to extract features from the training image samples through the feature extraction unit to obtain domain-shared image features, wherein training image samples from different data distribution domains share the network parameters of the feature extraction unit;

[0035] The domain feature decoupling module is used to decouple the domain shared image features by inputting the domain information features and the domain shared image features into the domain feature decoupling unit, thereby obtaining domain decoupling features.

[0036] The target detection module is used to perform target detection on the training image samples by inputting the domain decoupling features into the target detection unit, and obtain the target detection result;

[0037] The model optimization module is used to optimize the target detection model based on the target detection loss obtained from the target detection results.

[0038] Optionally, the domain feature decoupling module is further configured to:

[0039] Based on the domain information features and the domain-shared image features, construct channel attention weights based on domain information;

[0040] Based on the channel attention weights, the domain-shared image features are decoupled to obtain the domain-decoupled features.

[0041] Optionally, the domain feature decoupling unit includes a first encoder and a second encoder, and the domain feature decoupling module is further configured to:

[0042] By inputting the domain information features into the first encoder, the domain information features are encoded to obtain the domain information encoded features;

[0043] The domain information encoding features are aggregated with the domain shared image features to obtain aggregated domain information features;

[0044] The channel attention weights are obtained by inputting the aggregated domain information features into the second encoder and encoding the aggregated domain information features.

[0045] Optionally, the domain feature decoupling module is further configured to:

[0046] According to the preset constraint function, the channel attention weights are converted into reweighted parameters;

[0047] According to the reweighting parameters, the output channels of the domain-shared image features are reweighted to obtain the domain-decoupled features. The reweighting parameters are used to enhance the output feature values ​​of output channels that are strongly correlated with domain information and to suppress the output feature values ​​of output channels that are weakly correlated with domain information.

[0048] Optionally, the model optimization module is further configured to:

[0049] If the target detection loss converges, the target detection model is determined to be optimized.

[0050] If the target detection loss does not converge, then the feature extraction unit, the domain feature decoupling unit, the target detection unit, and the domain information features are updated by backpropagation based on the gradient calculated by the target detection loss.

[0051] Return to the execution steps: Obtain training image samples from different data distribution domains and the domain information features of the data distribution domain to which the training image samples belong, until the convergence of the target detection loss is detected.

[0052] Optionally, the model optimization module is further configured to:

[0053] If the target detection loss converges, the target detection model is determined to be optimized.

[0054] If the target detection loss does not converge, then the domain feature decoupling unit and the domain information feature are updated by backpropagation based on the gradient calculated by the target detection loss.

[0055] Return to the execution steps: Obtain training image samples from different data distribution domains and the domain information features of the data distribution domain to which the training image samples belong, until the convergence of the target detection loss is detected.

[0056] Optionally, the acquisition module is further configured to:

[0057] Obtain the domain information of the data distribution domain to which the training image samples belong;

[0058] Domain information features are extracted from the domain information to obtain the domain information features corresponding to the domain.

[0059] This application also provides an electronic device, which is a physical device, comprising: a memory, a processor, and a program of the target detection model optimization method stored in the memory and executable on the processor. When the program of the target detection model optimization method is executed by the processor, it can implement the steps of the target detection model optimization method as described above.

[0060] This application also provides a computer-readable storage medium storing a program for implementing a target detection model optimization method, wherein when the program for the target detection model optimization method is executed by a processor, it implements the steps of the target detection model optimization method as described above.

[0061] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the target detection model optimization method described above.

[0062] This application provides a method, apparatus, electronic device, and readable storage medium for optimizing an object detection model. The object detection model includes a feature extraction unit, a domain feature decoupling unit, and an object detection unit. First, training image samples from different data distribution domains and the domain information features of the data distribution domain to which the training image samples belong are obtained. The feature extraction unit extracts features from the training image samples to obtain domain-shared image features, wherein the training image samples from different data distribution domains share the network parameters of the feature extraction unit. The domain information features and the domain-shared image features are input into the domain feature decoupling unit to decouple the domain-shared image features, resulting in domain decoupling features. This achieves the construction of differentiated domain decoupling features corresponding to each domain using the shared feature extraction unit and the domain information features of each domain. Then, by inputting the domain decoupling features into the object detection unit, object detection is performed on the training image samples to obtain object detection results. This achieves the purpose of differentiated object detection for each domain. Furthermore, the object detection model is optimized based on the object detection loss obtained from the object detection results. This application does not train a single-domain object detection model for each domain to achieve targeted object detection for each domain. In the object detection model of this application, the feature extraction unit, object detection unit, and domain feature decoupling unit are all shared modules of image datasets across different domains. The difference between each domain lies only in the different domain information features used as input. By using the domain information features to decouple the shared image features of the domains, differentiated domain decoupling features can be obtained. Target detection can then be performed using these differentiated domain decoupling features, which can also achieve targeted object detection for each domain. This ensures the accuracy of object detection in each domain. Therefore, the goal of building a shared object detection model for all domains is achieved while ensuring the accuracy of object detection in all domains, reducing the redundancy of the object detection model for training image samples in different data distribution domains. Attached Figure Description

[0063] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0064] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0065] Figure 1 This is a flowchart illustrating the first embodiment of the target detection model optimization method of this application;

[0066] Figure 2 This is a flowchart illustrating the second embodiment of the target detection model optimization method of this application;

[0067] Figure 3 This is a schematic diagram of the hardware operating environment involved in the target detection model optimization method in the embodiments of this application.

[0068] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0069] To make the above-mentioned objectives, features, and advantages of this application more apparent and understandable, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0070] Due to the varying environments, heights, and angles of the current deployment of numerous monitoring devices, a massive amount of cross-domain data is introduced. Currently, for this cross-domain data, a separate object detection model is typically trained for each domain, enabling targeted object detection for each domain. This object detection model usually consists of a feature extraction unit and an object detection unit. However, when the number of domains is large, the number of models also increases, leading to significant model redundancy.

[0071] This application provides a method for optimizing a target detection model. The target detection model includes a feature extraction unit, a domain feature decoupling unit, and a target detection unit. In one embodiment of the target detection model optimization method of this application, refer to... Figure 1 The target detection model optimization method includes:

[0072] Step S10: Obtain training image samples from different data distribution domains and the domain information features of the data distribution domain to which the training image samples belong;

[0073] Step S20: The feature extraction unit extracts features from the training image samples to obtain domain-shared image features, wherein training image samples from different data distribution domains share the network parameters of the feature extraction unit.

[0074] Step S30: By inputting the domain information features and the domain shared image features into the domain feature decoupling unit, the domain shared image features are decoupled to obtain domain decoupling features;

[0075] Step S40: By inputting the domain decoupling features into the target detection unit, target detection is performed on the training image samples to obtain the target detection result;

[0076] Step S50: Optimize the target detection model based on the target detection loss obtained from the target detection results.

[0077] In this embodiment, it should be noted that the data distribution domain can be a sample space where a data distribution exists. Different data distribution domains typically correspond to different target detection tasks. For example, different data distribution domains can correspond to target detection tasks during the day and at night, respectively; different data distribution domains can correspond to target detection tasks in different road scenes; different data distribution domains can correspond to target detection tasks under different weather conditions, etc. The target detection model includes a feature extraction unit, a domain feature decoupling unit, and a target detection unit. The feature extraction unit is used to extract features from the image and output image features. Image data from different data distribution domains share the network parameters of the feature extraction unit. The domain feature decoupling unit is used to decouple the image features output by the feature extraction unit based on the domain information features, thereby obtaining differentiated image features corresponding to each data distribution domain based on the domain information of each data distribution domain, i.e., obtaining domain decoupling features. The target detection unit is used to perform target detection on the image based on the domain decoupling features. Image data from different data distribution domains share the network parameters of the target detection unit. Since the features input to the target detection unit are differentiated image features of each data distribution domain, a targeted target detection process can be performed for each data distribution domain. Therefore, in this embodiment, the feature extraction unit shared by each data distribution domain and the shared domain feature decoupling unit are used in the target detection model to replace the feature extraction units corresponding to each data distribution domain separately. Compared with the feature extraction units corresponding to each data distribution domain separately, the shared feature extraction unit and the shared domain feature decoupling unit can also generate differentiated image features between each data distribution domain, so as to realize targeted target detection for each data distribution domain. Thus, the redundancy of the target detection model for images of different data distribution domains is reduced.

[0078] Furthermore, since the domain decoupling feature is obtained by decoupling the domain-shared image features based on the domain information feature, the domain decoupling feature will simultaneously contain domain information and shared feature information between different data distribution domains. The target detection model is trained based on training image samples from different data distribution domains. Therefore, the target detection model is trained based on image samples from different data distribution domains. It can also improve the generalization of the target detection model based on the multi-task complementarity between different data distribution domains.

[0079] The domain information feature is a feature vector that characterizes the domain information. The domain information can be scene feature information of the image shooting scene, such as shooting position, shooting angle, scene light intensity and camera height.

[0080] As an example, steps S10 to S50 include: acquiring training image samples from different data distribution domains and the domain ID of the data distribution domain to which the training image samples belong; finding the corresponding domain information features based on the domain ID; extracting features from the training image samples by inputting them into the feature extraction unit to transform the pixel matrix corresponding to the training image samples to a preset feature dimension, thereby obtaining domain-shared image features, wherein the training image samples from different data distribution domains share the network parameters of the feature extraction unit; and inputting the domain information features and the domain-shared image features together into the domain feature decoupling unit to decouple the domain-shared image features using the domain information features, thereby enhancing the image features in the domain-shared image features that are strongly correlated with the domain information. Information, and image feature information that is weakly correlated with domain information in the domain-shared image features, thereby obtaining domain decoupling features; by inputting the domain decoupling features into the target detection unit, target detection is performed on the training image samples to obtain target detection results; image sample labels corresponding to the training image samples are obtained, and target detection loss is calculated based on the distance between the image sample labels and the target detection results; if the target detection loss converges, it is determined that the target detection model has been optimized; if the target detection loss does not converge, the target detection model is updated based on the model gradient calculated by the target detection loss, and the execution steps are returned: obtaining training image samples from different data distribution domains and the domain information features of the data distribution domain to which the training image samples belong, until the calculated target detection loss converges.

[0081] As an example, the image sample label includes candidate box location labels and classification labels. The object detection loss can be one or more of location loss and classification loss. The object detection loss can be obtained by summing the location loss and classification loss. The location loss can be the similarity loss between the predicted candidate box location and the candidate box location label during the object detection process. The classification loss can be the similarity loss between the predicted image classification result and the classification label during the object detection process.

[0082] As an example, training image samples from different data distribution domains can be preprocessed image samples. The purpose of preprocessing is to ensure the balance and diversity of training image samples from different data distribution domains. Preprocessing methods can include batch sampling, using the MixUp image fusion augmentation method, using the multi-scale image augmentation method, and using the PhotoDistortion augmentation method, etc.

[0083] As an example, the domain-shared image features can be multi-scale fused image features. The feature extraction unit can include a backbone network and a multi-scale fusion network. The backbone network is used to extract features from image samples to obtain image sample features at different depths. The multi-scale fusion network is used to fuse image sample features at different depths to obtain multi-scale fused image features.

[0084] As an example, inputting the domain decoupling features into the target detection unit for target detection includes:

[0085] At least one candidate image box is selected from the feature map corresponding to the domain decoupling feature. Using each candidate image box, corresponding image instance features are extracted from the feature map. Based on each image instance feature, classification prediction is performed on each candidate image box. Based on the obtained classification prediction results, a target image box is selected from each candidate image box. The image target in the target image box is the detected target, and the classification prediction result can be a classification probability or a classification label. Image classification is performed on the image region selected by the target image box to obtain the image classification result.

[0086] The step of performing domain decoupling on the domain-shared image features to obtain domain-decoupled features includes:

[0087] Step S31: Construct channel attention weights based on domain information according to the domain information features and the domain-shared image features;

[0088] Step S32: Based on the channel attention weights, perform domain decoupling on the domain-shared image features to obtain the domain-decoupled features.

[0089] In this embodiment, it should be noted that each feature value in the domain-shared image features is typically output by a corresponding output channel. The importance of image feature values ​​output by different output channels for target detection usually varies, and the output channels with higher importance corresponding to different data distribution domains are not necessarily the same. For example, the output channels with higher importance corresponding to data distribution domain A are a, b, and c, while the output channels with higher importance corresponding to data distribution domain b are a, d, and f. The domain feature decoupling unit includes an encoder, which is used to convert the domain information features and the domain-shared image features into corresponding channel attention weights. The channel attention weight is a weight characterizing the importance of the output channel corresponding to the domain-shared image feature. The channel attention weight includes at least one weight value corresponding to the output channel of the domain-shared image feature. The larger the weight value of the output channel, the higher the influence of the image feature value output by that output channel on the accuracy of target detection in the corresponding domain.

[0090] As an example, steps S31 to S32 include: inputting the domain information features and the domain-shared image features into the encoder, converting the domain information features and the domain-shared image features into corresponding channel attention weights, wherein the channel attention weights characterize the importance of the output channels of the domain-shared image features for target detection in the corresponding domain; reweighting the output channels corresponding to the domain-shared image features using the channel attention weights to decouple the domain-shared image features from the domain, and using the image features output by each reweighted output channel as the domain decoupling features. This embodiment of the application achieves the purpose of constructing channel attention weights based on domain information, thereby decoupling the domain-shared image features from the domain according to the channel attention weights. It can apply a corresponding interference to the domain-shared image features based on the domain information, making the image features different between different data distribution domains. Therefore, in addition to learning the image-shared features of each data distribution domain, it can also learn the domain information features of each data distribution domain, that is, learn the differentiated features between different data distribution domains, laying the foundation for targeted target detection in different data distribution domains.

[0091] The domain feature decoupling unit includes a first encoder and a second encoder. The step of constructing channel attention weights based on domain information according to the domain information features and the domain shared image features includes:

[0092] Step S311: By inputting the domain information features into the first encoder, the domain information features are encoded to obtain domain information encoded features;

[0093] Step S312: Aggregate the domain information encoding features with the domain shared image features to obtain aggregated domain information features;

[0094] Step S313: The aggregated domain information features are input into the second encoder to encode the aggregated domain information features, thereby obtaining the channel attention weights.

[0095] As an example, steps S311 to S313 include: inputting the domain information features into the first encoder to encode the domain information features, thereby transforming the domain information features to a first preset feature dimension to obtain domain information encoded features; aggregating the domain information encoded features with the domain-shared image features to obtain aggregated domain information features, wherein the aggregation method can be summation or product; and inputting the aggregated domain information features into the second encoder to encode the aggregated domain information features, thereby transforming the aggregated domain information features to a second preset feature dimension to obtain the channel attention weights.

[0096] As an example, the specific formula for calculating the channel attention weights is as follows:

[0097]

[0098] Where attn is the channel attention weight, F emd For the first encoder, F attn For the second encoder, E d For the domain information feature, f c d The domain-shared image features are defined by matmul, which represents matrix multiplication.

[0099] The step of performing domain decoupling on the domain-shared image features based on the channel attention weights to obtain domain-decoupled features includes:

[0100] Step S321: Convert the channel attention weights into reweighted parameters according to the preset constraint function;

[0101] Step S322: Based on the reweighting parameters, the output channels of the domain-shared image features are reweighted to obtain the domain decoupling features. The reweighting parameters are used to enhance the output feature values ​​of output channels that are strongly correlated with domain information and to suppress the output feature values ​​of output channels that are weakly correlated with domain information.

[0102] As an example, steps S321 to S322 include: inputting the channel attention weights into a preset constraint function to constrain the channel attention weights to a preset value range to obtain the reweighting parameters; and reweighting the output channels of the domain-shared image features according to the reweighting parameters to obtain the domain decoupling features. The reweighting parameters are used to enhance the output feature values ​​of output channels that are strongly correlated with domain information and to suppress the output feature values ​​of output channels that are weakly correlated with domain information. That is, a larger weight value is given to the output channels that are strongly correlated with domain information to enhance their output, and a smaller weight value is given to the output channels that are weakly correlated with domain information to suppress their output.

[0103] As an example, the formula for calculating the domain decoupling characteristics is as follows:

[0104]

[0105] Among them, F c d The domain decoupling feature is defined as attn, and the channel attention weight is defined as attn. Let f be the preset constraint function. c d The domain shares image features.

[0106] The process of obtaining the domain information features of the data distribution domain to which the training image samples belong includes:

[0107] Step A10: Obtain the domain information of the data distribution domain to which the training image sample belongs;

[0108] Step A20: Extract domain information features from the domain information to obtain the domain information features corresponding to the domain.

[0109] In this embodiment, it should be noted that the target detection model further includes a domain information feature extraction unit, wherein the domain information features can be retrieved based on the domain ID corresponding to the training image sample, or generated based on known domain information.

[0110] As an example, steps A10 to A20 include: obtaining domain information of the data distribution domain to which the training image sample belongs, wherein the domain information includes information such as image shooting angle, image shooting height, and image shooting illumination, and the domain information can be a domain information vector; inputting the domain information vector into the domain information feature extraction unit, performing feature extraction on the domain information vector to convert the domain information vector into a preset domain information feature dimension, thereby obtaining the domain information features corresponding to the data distribution domain.

[0111] The step of optimizing the target detection model based on the target detection loss obtained from the target detection results includes:

[0112] Step S51: If the target detection loss converges, it is determined that the target detection model has been optimized.

[0113] Step S52: If the target detection loss does not converge, then based on the gradient calculated by the target detection loss, backpropagation is performed to update the feature extraction unit, the domain feature decoupling unit, the target detection unit, and the domain information features.

[0114] Step S53, return to the execution steps: obtain training image samples from different data distribution domains and the domain information features of the data distribution domain to which the training image samples belong, until the target detection loss is detected to converge.

[0115] As an example, steps S51 to S53 include: if the target detection loss converges, it proves that the accuracy of the target detection model has met the requirements, thus determining that the target detection model has been optimized; if the target detection loss does not converge, then based on the target detection loss, calculate the first gradient for the feature extraction unit, the second gradient for the domain feature decoupling unit, the third gradient for the target detection unit, and the fourth gradient for the domain information feature, respectively; update the feature extraction unit based on the first gradient, update the domain feature decoupling unit based on the second gradient, update the target detection unit based on the third gradient, and update the domain information feature based on the fourth gradient; return to the execution step: obtain training image samples from different data distribution domains and the domain information features of the data distribution domain to which the training image samples belong, to perform the next round of iterative updates of the target detection model, until the target detection loss calculated during the iterative update process converges.

[0116] Furthermore, it should be noted that currently, when performing multi-domain learning, due to semantic and distributional differences in image data from different data distribution domains, direct multi-domain learning based on image data from different data distribution domains may lead to interference between the multi-domain data, thus affecting the accuracy of the target detection model built based on multi-domain learning. However, this application embodiment includes a domain feature decoupling unit. This unit reweights the output channels of the domain-shared image features, enhancing the output feature values ​​of output channels with strong domain information correlation and suppressing the output feature values ​​of output channels with weak domain information correlation. Therefore, by using the domain feature decoupling unit to decouple domain features, interference between multi-domain data can be reduced in detection tasks jointly optimized from data from different data distribution domains, solving the information interference problem caused by data distribution differences, and improving the accuracy of the target detection model built based on multi-domain learning, i.e., improving the performance of cross-domain target detection models.

[0117] This application provides a method for optimizing a target detection model. The target detection model includes a feature extraction unit, a domain feature decoupling unit, and a target detection unit. In this embodiment, training image samples from different data distribution domains and the domain information features of the data distribution domain to which the training image samples belong are first acquired. The feature extraction unit extracts features from the training image samples to obtain domain-shared image features, wherein the training image samples from different data distribution domains share the network parameters of the feature extraction unit. The domain information features and the domain-shared image features are input into the domain feature decoupling unit to decouple the domain-shared image features, resulting in domain decoupling features. This achieves the construction of differentiated domain decoupling features corresponding to each domain using the shared feature extraction unit and the domain information features of each domain. Then, by inputting the domain decoupling features into the target detection unit, target detection is performed on the training image samples to obtain target detection results. This achieves the purpose of differentiated target detection for each domain. Furthermore, the target detection model is optimized based on the target detection loss obtained from the target detection results. In this embodiment, a single-domain object detection model is not trained for each domain to achieve targeted object detection for each domain. In the object detection model of this embodiment, the feature extraction unit, object detection unit, and domain feature decoupling unit are all shared modules of image datasets across different domains. The difference between each domain lies only in the different domain information features used as input. By using the domain information features to decouple the shared image features of the domains, differentiated domain decoupling features can be obtained. Thus, using the differentiated domain decoupling features for object detection can also achieve targeted object detection for each domain, ensuring the accuracy of object detection in each domain. Therefore, the goal of building a shared object detection model for all domains is achieved while ensuring the accuracy of object detection in all domains, reducing the redundancy of the object detection model for training image samples in different data distribution domains.

[0118] Furthermore, referring to Figure 2 In another embodiment of this application, content that is the same as or similar to the above embodiment can be referred to the above description, and will not be repeated hereafter. The step of optimizing the target detection model based on the target detection loss obtained from the target detection result includes:

[0119] Step B10: If the target detection loss converges, then the target detection model is determined to be optimized.

[0120] Step B20: If the target detection loss does not converge, then backpropagation is performed to update the domain feature decoupling unit and the domain information features based on the gradient calculated by the target detection loss.

[0121] Step B30, return to the execution steps: obtain training image samples from different data distribution domains and the domain information features of the data distribution domain to which the training image samples belong, until the convergence of the target detection loss is detected.

[0122] In this embodiment, it should be noted that the domain feature decoupling unit is a pluggable module that can be added to an existing target detection model for iterative updates to obtain the final target detection model.

[0123] As an example, steps B10 to B30 include: if the target detection loss converges, it proves that the accuracy of the target detection model has met the requirements, thus determining that the target detection model has been optimized; if the target detection loss does not converge, then based on the target detection loss, calculate the fifth gradient for the domain feature decoupling unit and the sixth gradient for the domain information feature respectively; update the domain feature decoupling unit based on the fifth gradient, and update the domain information feature based on the sixth gradient; return to the execution step: obtain training image samples from different data distribution domains and the domain information feature of the data distribution domain to which the training image samples belong, to perform the next round of iterative update of the target detection model, until the target detection loss calculated during the iterative update process converges.

[0124] This application provides a method for optimizing an object detection model. Specifically, after calculating the object detection loss, if the object detection loss converges, the object detection model is considered optimized. If the object detection loss does not converge, the domain feature decoupling unit and the domain information features are updated via backpropagation based on the gradient calculated from the object detection loss. The process then returns to the previous step: acquiring training image samples from different data distribution domains and the domain information features of the data distribution domain to which the training image samples belong, until the object detection loss converges. By adding a domain feature decoupling unit for iterative updates to the existing object detection model composed of a feature extraction unit and an object detection unit, it is possible to insert a domain decoupling model into the existing object detection model for training image samples of different data distribution domains, thus avoiding the need to train a new object detection model from scratch. This improves the model training efficiency of object detection models for training image samples of different data distribution domains.

[0125] This application embodiment also provides a target detection model optimization device, wherein the target detection model includes a feature extraction unit, a domain feature decoupling unit, and a target detection unit, and the target detection model optimization device includes:

[0126] The acquisition module is used to acquire training image samples from different data distribution domains and the domain information features of the data distribution domain to which the training image samples belong;

[0127] A domain-shared feature extraction module is used to extract features from the training image samples through the feature extraction unit to obtain domain-shared image features, wherein training image samples from different data distribution domains share the network parameters of the feature extraction unit;

[0128] The domain feature decoupling module is used to decouple the domain shared image features by inputting the domain information features and the domain shared image features into the domain feature decoupling unit, thereby obtaining domain decoupling features.

[0129] The target detection module is used to perform target detection on the training image samples by inputting the domain decoupling features into the target detection unit, and obtain the target detection result;

[0130] The model optimization module is used to optimize the target detection model based on the target detection loss obtained from the target detection results.

[0131] Optionally, the domain feature decoupling module is further configured to:

[0132] Based on the domain information features and the domain-shared image features, construct channel attention weights based on domain information;

[0133] Based on the channel attention weights, the domain-shared image features are decoupled to obtain the domain-decoupled features.

[0134] Optionally, the domain feature decoupling unit includes a first encoder and a second encoder, and the domain feature decoupling module is further configured to:

[0135] By inputting the domain information features into the first encoder, the domain information features are encoded to obtain the domain information encoded features;

[0136] The domain information encoding features are aggregated with the domain shared image features to obtain aggregated domain information features;

[0137] The channel attention weights are obtained by inputting the aggregated domain information features into the second encoder and encoding the aggregated domain information features.

[0138] Optionally, the domain feature decoupling module is further configured to:

[0139] According to the preset constraint function, the channel attention weights are converted into reweighted parameters;

[0140] According to the reweighting parameters, the output channels of the domain-shared image features are reweighted to obtain the domain-decoupled features. The reweighting parameters are used to enhance the output feature values ​​of output channels that are strongly correlated with domain information and to suppress the output feature values ​​of output channels that are weakly correlated with domain information.

[0141] Optionally, the model optimization module is further configured to:

[0142] If the target detection loss converges, the target detection model is determined to be optimized.

[0143] If the target detection loss does not converge, then the feature extraction unit, the domain feature decoupling unit, the target detection unit, and the domain information features are updated by backpropagation based on the gradient calculated by the target detection loss.

[0144] Return to the execution steps: Obtain training image samples from different data distribution domains and the domain information features of the data distribution domain to which the training image samples belong, until the convergence of the target detection loss is detected.

[0145] Optionally, the model optimization module is further configured to:

[0146] If the target detection loss converges, the target detection model is determined to be optimized.

[0147] If the target detection loss does not converge, then the domain feature decoupling unit and the domain information feature are updated by backpropagation based on the gradient calculated by the target detection loss.

[0148] Return to the execution steps: Obtain training image samples from different data distribution domains and the domain information features of the data distribution domain to which the training image samples belong, until the convergence of the target detection loss is detected.

[0149] Optionally, the acquisition module is further configured to:

[0150] Obtain the domain information of the data distribution domain to which the training image samples belong;

[0151] Domain information features are extracted from the domain information to obtain the domain information features corresponding to the domain.

[0152] The target detection model optimization device provided in this application, employing the target detection model optimization method described in the above embodiments, solves the technical problem of severe redundancy in the target detection model for training image samples with different data distribution domains. Compared with the prior art, the beneficial effects of the target detection model optimization device provided in this application are the same as those of the target detection model optimization method described in the above embodiments, and other technical features in this target detection model optimization device are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0153] This application provides an electronic device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, which are executed by the at least one processor to enable the at least one processor to perform the target detection model optimization method in the first embodiment described above.

[0154] The following is for reference. Figure 3 The diagram illustrates a structural schematic of an electronic device suitable for implementing embodiments of the present disclosure. The electronic devices in the embodiments of the present disclosure may include, but are not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 3 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0155] like Figure 3 As shown, an electronic device may include a processing unit (such as a central processing unit, graphics processing unit, etc.) that can perform various appropriate actions and processes based on a program stored in read-only memory (ROM) or a program loaded from a storage device into random access memory (RAM). The RAM also stores various programs and data required for the operation of the electronic device. The processing unit, ROM, and RAM are interconnected via a bus. Input / output (I / O) interfaces are also connected to the bus.

[0156] Typically, the following systems can be connected to the I / O interface: input devices including, for example, touchscreens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices including, for example, magnetic tapes, hard disks, etc.; and communication devices. Communication devices allow electronic devices to communicate wirelessly or wiredly with other devices to exchange data. Although electronic devices with various systems are shown in the figures, it should be understood that it is not required to implement or possess all the systems shown. More or fewer systems may be implemented alternatively.

[0157] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from a storage device, or installed from a ROM. When the computer program is executed by a processing device, it performs the functions defined above in the methods of embodiments of this disclosure.

[0158] The electronic device provided in this application employs the target detection model optimization method described in the above embodiments, solving the technical problem of severe redundancy in the target detection model for training image samples with different data distribution domains. Compared with the prior art, the beneficial effects of the electronic device provided in this application are the same as those of the target detection model optimization method described in the above embodiments, and other technical features of the electronic device are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0159] It should be understood that various parts of this disclosure can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics may be combined in any suitable manner in one or more embodiments or examples.

[0160] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0161] This embodiment provides a computer-readable storage medium having computer-readable program instructions stored thereon, which are used to execute the target detection model optimization method described in the above embodiment.

[0162] The computer-readable storage medium provided in this application embodiment may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.

[0163] The aforementioned computer-readable storage medium may be included in an electronic device or may exist independently without being assembled into an electronic device.

[0164] The aforementioned computer-readable storage medium carries one or more programs. When these programs are executed by an electronic device, the electronic device causes the following: it acquires training image samples from different data distribution domains and domain information features of the data distribution domain to which the training image samples belong; it extracts features from the training image samples using the feature extraction unit to obtain domain-shared image features, wherein the training image samples from different data distribution domains share the network parameters of the feature extraction unit; it decouples the domain-shared image features by inputting the domain information features and the domain-shared image features into the domain feature decoupling unit to obtain domain-decoupled features; it performs target detection on the training image samples by inputting the domain-decoupled features into the target detection unit to obtain target detection results; and it optimizes the target detection model based on the target detection loss obtained from the target detection results.

[0165] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0166] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0167] The modules described in the embodiments of this disclosure can be implemented in software or hardware. The names of the modules are not, in some cases, intended to limit the functionality of the module itself.

[0168] The computer-readable storage medium provided in this application stores computer-readable program instructions for executing the above-described object detection model optimization method, solving the technical problem of severe redundancy in the object detection model for training image samples with different data distribution domains. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as the beneficial effects of the object detection model optimization method provided in the above-described embodiments, and will not be repeated here.

[0169] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the target detection model optimization method described above.

[0170] The computer program product provided in this application solves the technical problem of severe redundancy in object detection models for training image samples with different data distribution domains. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as the beneficial effects of the object detection model optimization method provided in the above embodiments, and will not be repeated here.

[0171] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent scope of this application.

Claims

1. A method for optimizing a target detection model, characterized in that, The target detection model includes a feature extraction unit, a domain feature decoupling unit, and a target detection unit. The target detection model optimization method includes: Obtain training image samples from different data distribution domains and the domain information features of the data distribution domain to which the training image samples belong; The feature extraction unit extracts features from the training image samples to obtain domain-shared image features, wherein training image samples from different data distribution domains share the network parameters of the feature extraction unit. By inputting the domain information features and the domain shared image features into the domain feature decoupling unit, the domain shared image features are decoupled to obtain domain decoupling features. The domain feature decoupling unit includes a first encoder and a second encoder. By inputting the domain information features into the first encoder, the domain information features are encoded to obtain domain information encoded features. The domain information encoded features are aggregated with the domain shared image features to obtain aggregated domain information features. By inputting the aggregated domain information features into the second encoder, the aggregated domain information features are encoded to obtain channel attention weights. Based on the channel attention weights, the domain shared image features are decoupled to obtain domain decoupling features. By inputting the domain decoupling features into the target detection unit, target detection is performed on the training image samples to obtain the target detection result; The target detection model is optimized based on the target detection loss obtained from the target detection results.

2. The target detection model optimization method as described in claim 1, characterized in that, The step of performing domain decoupling on the domain-shared image features to obtain domain-decoupled features includes: Based on the domain information features and the domain-shared image features, construct channel attention weights based on domain information; Based on the channel attention weights, the domain-shared image features are decoupled to obtain the domain-decoupled features.

3. The target detection model optimization method as described in claim 2, characterized in that, The step of performing domain decoupling on the domain-shared image features based on the channel attention weights to obtain domain-decoupled features includes: According to the preset constraint function, the channel attention weights are converted into reweighted parameters; According to the reweighting parameters, the output channels of the domain-shared image features are reweighted to obtain the domain-decoupled features. The reweighting parameters are used to enhance the output feature values ​​of output channels that are strongly correlated with domain information and to suppress the output feature values ​​of output channels that are weakly correlated with domain information.

4. The target detection model optimization method as described in claim 1, characterized in that, The step of optimizing the target detection model based on the target detection loss obtained from the target detection result includes: If the target detection loss converges, the target detection model is determined to be optimized. If the target detection loss does not converge, then the feature extraction unit, the domain feature decoupling unit, the target detection unit, and the domain information features are updated by backpropagation based on the gradient calculated by the target detection loss. Return to the execution steps: Obtain training image samples from different data distribution domains and the domain information features of the data distribution domain to which the training image samples belong, until the convergence of the target detection loss is detected.

5. The target detection model optimization method as described in claim 1, characterized in that, The step of optimizing the target detection model based on the target detection loss obtained from the target detection result includes: If the target detection loss converges, the target detection model is determined to be optimized. If the target detection loss does not converge, then the domain feature decoupling unit and the domain information features are updated by backpropagation based on the gradient calculated by the target detection loss. Return to the execution steps: Obtain training image samples from different data distribution domains and the domain information features of the data distribution domain to which the training image samples belong, until the convergence of the target detection loss is detected.

6. The target detection model optimization method as described in claim 1, characterized in that, Obtaining the domain information features of the data distribution domain to which the training image samples belong includes: Obtain the domain information of the data distribution domain to which the training image samples belong; Domain information features are extracted from the domain information to obtain the domain information features corresponding to the domain.

7. A target detection model optimization device, characterized in that, The target detection model includes a feature extraction unit, a domain feature decoupling unit, and a target detection unit. The target detection model optimization device includes: The acquisition module is used to acquire training image samples from different data distribution domains and the domain information features of the data distribution domain to which the training image samples belong; A domain-shared feature extraction module is used to extract features from the training image samples through the feature extraction unit to obtain domain-shared image features, wherein training image samples from different data distribution domains share the network parameters of the feature extraction unit; A domain feature decoupling module is used to decouple the domain shared image features by inputting the domain information features and the domain shared image features into the domain feature decoupling unit, thereby obtaining domain decoupling features. The domain feature decoupling unit includes a first encoder and a second encoder. The domain information features are input into the first encoder and encoded to obtain domain information encoded features. The domain information encoded features are aggregated with the domain shared image features to obtain aggregated domain information features. The aggregated domain information features are input into the second encoder and encoded to obtain channel attention weights. Based on the channel attention weights, the domain shared image features are decoupled to obtain domain decoupling features. The target detection module is used to perform target detection on the training image samples by inputting the domain decoupling features into the target detection unit, and obtain the target detection result; The model optimization module is used to optimize the target detection model based on the target detection loss obtained from the target detection results.

8. An electronic device, characterized in that, The electronic device includes: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the steps of the target detection model optimization method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a program for implementing a target detection model optimization method, which is executed by a processor to implement the steps of the target detection model optimization method as described in any one of claims 1 to 6.

Images