Method for constructing a base model for training a domain adaptive object detection model

Abstract

The application provides a base model for training a domain adaptive target detection model, comprising: a feature extraction network, configured to convert a source domain image and a target domain image into a source domain feature map and a target domain feature map respectively; a feature decomposition network, configured to compare the source domain feature map and the target domain feature map to determine first instance invariant features of the source domain feature map and second instance invariant features of the target domain feature map; a feature adder, configured to combine the first instance invariant features and the source domain feature map to obtain a first sum image, and combine the second instance invariant features and the target domain feature map to obtain a second sum image; and a detector, configured to determine a position of a source domain image detection frame and a category of the source domain image based on the first sum image, and determine a position of a target domain image detection frame and a category of the target domain image based on the second sum image. The target detection model trained by the base model can improve the accuracy of target detection.

Description

Technical Field

[0001] This application relates to the field of object detection technology, and in particular to a basic model for training domain adaptive object detection models. Background Technology

[0002] Currently, object detection models based on convolutional neural networks have achieved great success. However, these high-precision performances all depend on the source domain of the training set, such as large-scale, high-quality datasets like Pascal VOC and Cityscapes.

[0003] However, in practical applications, it often happens that the source domain images used to train the object detection model are taken during the day, while the actual time to use the object detection model is at night. In this case, the object detection model will experience a significant performance degradation. Currently, for this situation, a method called Domain Adaptive Object Detection (DAOD) is used. DAOD has become a way to solve the problem of excessively large domain gaps between the source and target domains. DAOD aligns the data distribution of a labeled source domain with the data distribution of an unlabeled target domain, enabling the domain-adaptive network to generate environment-independent features. In this process, distribution alignment focuses more on the overall representation of the domains at the feature level (mainly focusing on features of the image as a whole) or at the pixel level.

[0004] However, in actual use, the detected target often does not match the actual target, and the location and classification of the target are also inaccurate. Summary of the Invention

[0005] This application provides a basic model for training domain adaptive object detection models to solve the problems mentioned in the background art.

[0006] In the first aspect, this application provides a basic model for training a domain adaptive object detection model, which, in order from input to output, includes: a feature extraction network, a feature decomposition network, a feature summer, and a detector. The feature extraction network is used to receive source domain images and target domain images, convert the source domain images into source domain feature maps, convert the target domain images into target domain feature maps, send the source domain feature maps and the target domain feature maps to the feature decomposition network, and also send the target domain feature maps to the feature summer. The feature decomposition network is used to compare the source domain feature map and the target domain feature map to determine the first domain style feature and the first instance invariant feature of the source domain feature map, and to determine the second domain style feature and the second instance invariant feature of the target domain feature map. The feature summer is used to merge the first instance-invariant feature with the source domain feature map to obtain a first sum image; the feature summer is also used to merge the second instance-invariant feature with the target domain feature map to obtain a second sum image; the feature summer is also used to send the first sum image and the second sum image to the detector; The detector is used to determine the position of the source domain image detection box and the category of the source domain image based on the first image; the detector is also used to determine the position of the target domain image detection box and the category of the target domain image based on the second image.

[0007] Optionally, the base model further includes a detection loss function, which includes a source domain detection loss function and a target domain detection loss function that are parallel and data-connected to the output of the detector. The source domain detection loss function is used to determine the source domain detection loss value, and the target domain detection loss function is used to determine the target domain loss value.

[0008] Optionally, the feature extraction network, feature decomposition network, feature summer, and detector are collectively referred to as the student model, and the basic model also includes a teacher model and an image enhancement network; The teacher model and the student model have the same structure, and the parameters of the student model are student parameters; after training a predetermined number of times, the student model sends the student parameters to the teacher model; the teacher model receives the student parameters and adjusts its own parameters to be the same as the student parameters; The image enhancement network, the teacher model, and the target domain detection loss function are connected sequentially from input to output. The image enhancement network is used to receive and enhance the target domain image to obtain the target domain enhanced image, and then send the target domain image to the teacher model; The teacher model outputs the bounding box position and category of the target domain augmented image based on the target domain augmented image, and records the bounding box position and category of the target domain augmented image as pseudo-labels; and sends the pseudo-labels to the target domain detection loss function so that the target domain detection loss function calculates the target domain loss value based on the pseudo-labels.

[0009] Optionally, the basic model further includes a feature reconstruction network and a reconstruction loss function; The feature reconstruction network receives the second domain style features and the second instance-invariant features output by the feature decomposition network, and reconstructs the target domain image based on the second domain style features and the second instance-invariant features to obtain the reconstructed image, and sends the reconstructed image to the reconstruction loss function; The reconstruction loss function calculates the reconstruction loss value of the reconstructed image based on the reconstructed image and the target domain augmentation image.

[0010] Optionally, the base model further includes a mutual information loss function, which is used to receive the input source domain image feature map and target domain image feature map to calculate the mutual information loss value.

[0011] Optionally, the feature extraction network is ResNet101 or VGG-16.

[0012] Optionally, the detector is Faster R-CNN.

[0013] Secondly, this application provides a computer electronic device, the device comprising: one or more processors, and a memory; The memory is used to store the basic model; The processor is used to train any of the base models as described in the first aspect.

[0014] Thirdly, this application provides a computer-readable storage medium having stored thereon any of the basic models described in the first aspect, the basic models being executable by a processor.

[0015] Fourthly, this application provides a computer program product including any of the basic models described in the first aspect above, wherein the basic model can be executed by a processor.

[0016] The basic model for training the domain-adaptive object detection model provided in this application includes a feature decomposition network. This network compares source and target domain images, identifying similar and dissimilar parts. Similar parts are defined as instance-invariant features, while dissimilar parts are defined as their respective domain-specific style features. Instance-invariant features do not change with environmental variations, thus effectively identifying objects within the target domain image and improving the accuracy of target domain image detection by effectively converting from the source domain to the target domain. In contrast, existing technologies only focus on the overall relationship between the source and target domains at the feature and pixel levels; this application achieves adaptation from the source to the target domain through instance-level features, thereby improving the accuracy of object detection. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This application provides an implementation environment architecture diagram for a basic model used to train a domain adaptive object detection model. Figure 2 A schematic diagram of a basic model for training a domain adaptive object detection model provided in an embodiment of this application; Figure 3 A schematic diagram of a basic model for training a domain adaptive object detection model provided in an embodiment of this application; Figure 4 A schematic diagram of a basic model for training a domain adaptive object detection model provided in an embodiment of this application; Figure 5 This is a schematic diagram of the structure of a computer system according to an embodiment of this application. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions in the embodiments of this application are described clearly and completely below. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are also within the scope of protection of this application. In addition, it should be noted that, unless otherwise specified, the embodiments and features in the embodiments of this application can be combined with each other. The present application will now be described in detail with reference to the accompanying drawings and embodiments.

[0020] Figure 1 This is an implementation environment architecture diagram of a basic model for training a domain adaptive object detection model, as shown in an embodiment of this application. Figure 1 As shown, the implementation environment architecture includes: terminals and servers.

[0021] The hardware structure of the terminal includes a processor, memory, and display device, while the software structure includes program products installed on the terminal.

[0022] The aforementioned program product can be standalone software, such as an APP deployed on a terminal as a client (e.g., a client for a base model used to train a domain adaptive object detection model); the aforementioned program product can call on the hardware on the terminal (e.g., one or more of a display screen, a communication module, etc.) to implement the technical solution provided in this application.

[0023] Taking the application product as an example, which is deployed on the terminal, when the application client is started, the terminal screen can display the application interface. Through the interface, the parameters input by the user can be obtained. For example, as mentioned later, when actually training the basic model, it is necessary to manually collect source domain images and target domain images, manually label the source domain images, and send the input parameters to the server by calling the communication module, or receive one or more messages sent by the server and display them on the terminal screen.

[0024] The types of terminals include, but are not limited to, smartphones, tablets, televisions, laptops, desktop computers, etc., and this application does not specifically limit them.

[0025] The server contains a base model to be trained. If the base model is already trained, the server also contains a pre-trained object detection model. The server receives parameters from the terminal and trains the base model based on these parameters.

[0026] A server can be a single server, a server cluster consisting of multiple servers, or a cloud computing service center.

[0027] The terminal and the server establish a communication connection through a wired or wireless network.

[0028] Besides the terminal + server model described above, the basic model can also be implemented by a separate electronic device, such as a smartphone, tablet, television, laptop, or desktop computer. In this case, the basic model can be installed on the aforementioned individual electronic device.

[0029] Figure 2 This is a basic model for training a domain adaptive target detection model, as shown in the embodiments of this application. Figure 2 The basic model shown can be configured in Figure 1 Within the server, such as Figure 2 As shown, the basic model, in order from input to output, includes: a feature extraction network, a feature decomposition network, a feature summer, and a detector. The feature extraction network is used to receive source domain images and target domain images, convert the source domain images into source domain feature maps, convert the target domain images into target domain feature maps, send the source domain feature maps and the target domain feature maps to the feature decomposition network, and also send the target domain feature maps to the feature summer. The feature decomposition network is used to compare the source domain feature map and the target domain feature map to determine the first domain style feature and the first instance invariant feature of the source domain feature map, and to determine the second domain style feature and the second instance invariant feature of the target domain feature map. The feature summer is used to merge the first instance-invariant feature with the source domain feature map to obtain a first sum image; the feature summer is also used to merge the second instance-invariant feature with the target domain feature map to obtain a second sum image; the feature summer is also used to send the first sum image and the second sum image to the detector; The detector is used to determine the position of the source domain image detection box and the category of the source domain image based on the first image; the detector is also used to determine the position of the target domain image detection box and the category of the target domain image based on the second image.

[0030] The base model can be understood as follows: by training the base model multiple times, a new model with stable parameters is obtained when the model's loss value converges to a predetermined value. Furthermore, this new model can be used to detect objects within the target domain image.

[0031] The source domain image is an image taken when the target object is in a favorable environment, such as during the day or in clear weather. Furthermore, the source domain image can be images from existing large-scale datasets such as Pascal VOC and Cityscapes. The images in these datasets are labeled.

[0032] Among them, the source domain feature map is the image after feature extraction of the source domain image. During the training process, irrelevant backgrounds are continuously weakened while relevant features of the target object are preserved.

[0033] Among them, the target domain image is an image of the target object in a poor environment, such as at night, on a rainy day, or on a foggy day. The target domain image can be selected from publicly available datasets or from datasets collected by ourselves.

[0034] Furthermore, the source image can be a real image, and the target image can be a cartoon image, ink painting image, or other images with significantly different backgrounds.

[0035] Among them, the target domain feature map is the image after feature extraction of the target domain image. During the training process, irrelevant backgrounds are continuously weakened while relevant features of the target object are preserved.

[0036] Among them, the feature decomposition network widens the gap between domain style features (DSF) and instance invariant features (IIF) by minimizing mutual information, making the difference between domain style features and instance invariant features larger, thereby separating domain style invariant features and instance invariant features as much as possible.

[0037] Furthermore, the feature decomposition network can be a simple three-layer convolutional network.

[0038] Among them, domain style features are the different characteristics of the target object in different environmental backgrounds. For example, the source domain image is generally in daylight, and the pixel value of the source domain image is higher, while the target domain image is generally in darkness, and the pixel value of the target domain image is lower.

[0039] Among these, instance-invariant features remain unchanged across domains and are unaffected by specific style features such as weather conditions or brightness within a domain. For example, for vehicles, regardless of whether they are in the source or target domain, they will generally have structures such as rearview mirrors, seats, windshields, and wheels, and their overall outlines are also generally similar.

[0040] In addition, before inputting the source domain feature map, the first instance-invariant feature, the target domain feature map, and the second instance-invariant feature into the feature summer, the first instance-invariant feature is adjusted to have the same size as the source domain feature map, and the second instance-invariant feature is adjusted to have the same size as the target domain feature map.

[0041] Among them, the source domain image detection box is used to select target objects in the source domain image.

[0042] The target domain image detection box is used to select target objects in the target domain image.

[0043] The position of the detection box is the coordinate position of the target object in the image.

[0044] The category is used to classify the target objects in the image. For example, if the target object in the image is a vehicle, the category is vehicle; if the target object in the image is a cat, the type is cat.

[0045] Additionally, it should be noted that the object detection in this application can be understood as: detecting the target object in the target domain image, using a detection box to select the target object, locating the position of the detection box in the target domain image, and classifying the target object.

[0046] Before training, the target domain image set is divided into a training set and a test set. During training, the basic model is trained according to the functions of each structure included in the basic model mentioned above. When the loss value of the basic model decreases to within a predetermined range, the training of the basic model can be stopped, and a new basic model is obtained. The basic model is then tested using the test set. If the loss value of the basic model is still within the predetermined range when tested using the test set, the new basic model is determined as the target detection model for detecting targets.

[0047] It should be noted here that, since during testing, only the target domain image needs to be sent to the base model so that the base model can output the detection box and classification of the target domain image, it is only necessary to divide the target domain image into training and test sets, while the source domain image does not need to be divided into training and test sets.

[0048] It should be noted that while the existing dataset is used for training, in actual use, it is necessary to retrain the model with the relevant images of the actual use scenario. That is, every time the use scenario changes, the source domain images and target domain images need to be collected again and the basic model needs to be trained again.

[0049] The feature extraction network is the backbone network of the basic model. Optionally, the feature extraction network is ResNet101 or VGG-16.

[0050] Optionally, the detector is Faster R-CNN.

[0051] Optionally, the base model further includes a detection loss function, which includes a source domain detection loss function and a target domain detection loss function that are parallel and data-connected to the output of the detector. The source domain detection loss function is used to determine the source domain detection loss value, and the target domain detection loss function is used to determine the target domain loss value.

[0052] In this context, data connectivity can be understood as the ability to transmit data between two structures (networks, models, or devices).

[0053] The source domain detection loss function is used to receive the detection results output by the detector, namely the position of the source domain image detection box and the category of the source domain image. The source domain detection loss function is also used to receive the label of the source domain image, which includes the pre-determined position of the source domain image detection box and the category of the source domain image. Further, the label of the source domain image is compared with the position of the source domain image detection box and the category of the source domain image output by the detector to determine the loss value of the position of the source domain image detection box and the category of the source domain image output by the detector. Further, the loss value of the position of the source domain image detection box and the category of the source domain image can also be called the source domain detection loss value.

[0054] The target detection loss function has the same function as the source domain detection loss function, which will not be described in detail here. Similarly, the target detection loss function outputs the target domain detection loss value.

[0055] Furthermore, the source domain detection loss and the target domain detection loss can be used to determine whether training can be stopped. Further, training of the base model can be stopped when both the source domain detection loss and the target domain detection loss decrease to a predetermined range.

[0056] It should be noted that the above source domain detection loss function and target domain detection loss function can only be used directly when both the source domain image and the target domain image have labels.

[0057] Furthermore, the source domain detection loss is defined as follows: in, This is a regional recommended loss. It is classification loss. It's a location loss.

[0058] Furthermore, the target domain detection loss is similar to the source domain detection loss, as follows: in, This is a regional recommended loss. It is classification loss. It's a location loss.

[0059] Optionally, the base model further includes a mutual information loss function, which is used to receive the input source domain image feature map and target domain image feature map to calculate the mutual information loss value.

[0060] Each time the feature decomposition model decomposes the source domain feature map into first-domain style features and first-instance-invariant features, it can calculate the mutual information loss using a mutual information loss function. This mutual information loss value is then used to further adjust the parameters of the feature decomposition network, thereby improving its decomposition capability. Similarly, after decomposing the target domain feature map, the same mutual information loss function can be used to calculate the loss value, which can then be used to adjust the parameters of the feature decomposition network, further enhancing its decomposition capability.

[0061] Furthermore, the mutual information loss function is a mutual information neural estimator, which aims to include as many instance-invariant features as possible in the IIF. The calculation is based on the Monte Carlo integral method, as follows: in, This indicates sampling from the joint distribution of IIF and DSF. This indicates sampling from the marginal distribution of IIF. Here, a neural network T() is designed to perform the integral method in the formula. This represents network parameters.

[0062] In actual target domain detection, both source and target domain images, serving as training and test sets, need to be manually collected and labeled. However, labeling is a time-consuming and laborious process, severely reducing model training efficiency. Therefore, an image augmentation network and teacher model can be used to automatically label target domain images, and these labels can be recorded as pseudo-labels. The specific structure and method are as follows: Optionally, see Figure 3 The feature extraction network, feature decomposition network, feature summer, and detector are collectively referred to as the student model, and the basic model also includes the teacher model and the image enhancement network. The teacher model and the student model have the same structure, and the parameters of the student model are student parameters; after training a predetermined number of times, the student model sends the student parameters to the teacher model; the teacher model receives the student parameters and adjusts its own parameters to be the same as the student parameters; The image enhancement network, the teacher model, and the target domain detection loss function are connected sequentially from input to output. The image enhancement network is used to receive and enhance the target domain image to obtain the target domain enhanced image, and then send the target domain image to the teacher model; The teacher model outputs the bounding box position and category of the target domain augmented image based on the target domain augmented image, and records the bounding box position and category of the target domain augmented image as pseudo-labels; and sends the pseudo-labels to the target domain detection loss function so that the target domain detection loss function calculates the target domain loss value based on the pseudo-labels.

[0063] At this point, the target domain loss function calculates the target domain detection loss value based on the pseudo-labels, the position of the target image detection box output by the detector, and the category of the target domain image. Furthermore, the parameters of the base model can be adjusted based on the calculated target domain loss value to improve the accuracy of the base model's detection.

[0064] The teacher model and student model can form an average teacher-student model. The teacher model generates pseudo-labels, which not only saves the manpower and time spent on manual labeling, but also guides the student model to tend to reduce the loss value in the target domain. Furthermore, in order to obtain more reliable prediction results for the detector, an image augmentation network is used to enhance the target domain image, which is then used as input to the teacher model to further improve the accuracy of the pseudo-labels and guide the student model to tend to reduce the variance in the target domain, thereby enhancing the robustness of the model.

[0065] The type of image enhancement network can be determined based on the environment of the target domain. For example, if the target domain is at night, the target domain image is a nighttime image, and the image enhancement network is a nighttime image enhancement network. If the target domain is foggy, the image enhancement network is an image dehazing network.

[0066] The student model is trained using a gradient descent optimizer. As the detection accuracy gradually improves, the teacher model updates its parameters accordingly. The enhanced image of the target domain is used as input to the teacher model, and the detection boxes with higher locativeness are selected as the final retained detection boxes, which also serve as pseudo-labels. Note that the parameters of the teacher model are typically the parameters of the student model after a certain number of training iterations.

[0067] Optionally, see Figure 4 The basic model also includes a feature reconstruction network and a reconstruction loss function; The feature reconstruction network receives the second domain style features and the second instance-invariant features output by the feature decomposition network, and reconstructs the target domain image based on the second domain style features and the second instance-invariant features to obtain the reconstructed image, and sends the reconstructed image to the reconstruction loss function; The reconstruction loss function calculates the reconstruction loss value of the reconstructed image based on the reconstructed image and the target domain augmentation image.

[0068] The reconstruction loss is represented as follows: in, Indicates the refactoring module. This indicates the backbone network module. Represents the target domain image for reconstruction. This represents the enhanced target domain image.

[0069] By reconstructing the loss value, we can determine whether the decomposition operation of the feature decomposition model meets the standard. If the loss value is too large, the invariant features of the second instance may have missing content, which will lead to low detection accuracy of the base model.

[0070] Because the source and target domain images use unpaired datasets, IIF extraction may result in two different instances. For example, the source domain image might extract an instance of a car, while the target domain image might extract an instance of a horse-drawn carriage. Suppressing this difference can lead to erroneous features, causing incorrect instance-invariant features to be extracted, thus reducing object detection performance. Therefore, a combination of feature reconstruction and feature decomposition networks is used to reconstruct a clear image from the target domain data. The enhanced target domain image is then used as a label to measure the correctness of the feature decomposition network's decomposition of the target domain image, thereby minimizing the generation of false features.

[0071] The "unpaired" here can be understood as the source domain image and the target domain image containing objects of the same category, but they are not images targeting the same object, and the backgrounds of the images are also different.

[0072] Furthermore, when the various loss values ​​of the base model, such as the source domain detection loss value, the target domain detection loss value, the mutual information loss value, and the reconstruction loss value, converge to a predetermined range, training is stopped. If the various loss values ​​are also within the predetermined range when tested with a tester, then the base model is no longer trained, and the base model can then be used for target detection.

[0073] Furthermore, during actual detection, since the model has already been trained and can detect objects with the required accuracy, only the student model from the above model (which can be referred to as the object detection model) needs to be used. The target domain image is then sent to the object detection model, which outputs the location of the target domain image detection box and the category of the target domain image, thus completing the detection task.

[0074] Furthermore, mean accuracy (mAP) is used as the evaluation metric.

[0075] Additionally, it should be noted that when using the object detection model, the feature decomposition network already has the ability to identify second instance-invariant features, so feature decomposition is not required. It is only necessary to identify the second instance-invariant features, label them, and then send the target domain image labeled with the second instance-invariant features to the feature summer. The feature summer sums the second instance-invariant features and the target domain image to obtain the second sum image, and sends the sum image to the detector. The detector receives the second sum image and determines the position of the target domain image detection box and the category of the target domain image based on the second sum image.

[0076] The basic model for training the domain-adaptive object detection model provided in this application includes a feature decomposition network. This network compares source and target domain images, identifying similar and dissimilar parts. Similar parts are defined as instance-invariant features, while dissimilar parts are defined as their respective domain-specific style features. Instance-invariant features do not change with environmental variations, thus effectively identifying objects within the target domain image and improving the accuracy of target domain image detection by effectively converting from the source domain to the target domain. In contrast, existing technologies only focus on the overall relationship between the source and target domains at the feature and pixel levels; this application achieves adaptation from the source to the target domain through instance-level features, thereby improving the accuracy of object detection.

[0077] Figure 5 This is a schematic diagram illustrating the structure of an electronic device or computer system 500 according to an embodiment of this application. The computer system includes a central processing unit (CPU) 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage portion into a random access memory (RAM) 503. The RAM 503 also stores various programs and data required for system operation. The CPU 501, ROM 502, and RAM 503 are interconnected via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0078] The following components are connected to I / O interface 505: an input section 506 including a keyboard, mouse, etc.; an output section 507 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 508 including a hard disk, etc.; and a communication section 509 including a network interface card such as a LAN card, modem, etc. The communication section 509 performs communication processing via a network such as the Internet. A drive is also connected to I / O interface 505 as needed. A removable medium 511, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 510 as needed so that computer programs read from it can be installed into storage section 508 as needed.

[0079] In particular, the processes described in the flowcharts of the embodiments of this application can be implemented as computer software programs. For example, the method embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts.

[0080] In another aspect, this application also provides a computer-readable medium, which may be included in the electronic device described in the above embodiments; or it may exist independently and not assembled into the electronic device. The computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to implement the methods described in the embodiments of this application.

[0081] It should be noted that the computer-readable medium shown in this application may be a computer-readable signal medium or a computer-readable storage medium or any combination thereof.

[0082] The units described in the embodiments of this application can be implemented in software or hardware, and the described units can also be located in a processor. The names of these units do not necessarily limit the specific unit itself. The described units or modules can also be located in a processor. The names of these units or modules do not necessarily limit the specific unit or module itself.

[0083] In addition, it should be noted that the scope of this application includes feasible technical solutions formed by specific combinations of the above-mentioned technical features, and should also cover other feasible technical solutions formed by arbitrary combinations of the above-mentioned technical features or their equivalent features without departing from the above-mentioned application concept.

[0084] Finally, it should be noted that any content not described in the technical solutions of this application can be implemented using existing technology. Furthermore, the above embodiments are merely illustrative of the technical solutions of this application and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to depart from the scope of the technical solutions of the embodiments of this application.

Claims

1. A method for constructing a base model for training a domain-adaptive object detection model, characterized in that, The basic model, in order from input to output, includes: a feature extraction network, a feature decomposition network, a feature summer, and a detector. The feature extraction network is used to receive source domain images and target domain images, convert the source domain images into source domain feature maps, convert the target domain images into target domain feature maps, send the source domain feature maps and the target domain feature maps to the feature decomposition network, and also send the target domain feature maps to the feature summer. The feature decomposition network is used to compare the source domain feature map and the target domain feature map to determine the first domain style feature and the first instance invariant feature of the source domain feature map, and to determine the second domain style feature and the second instance invariant feature of the target domain feature map. The feature summer is used to merge the first instance-invariant feature with the source domain feature map to obtain a first sum image; the feature summer is also used to merge the second instance-invariant feature with the target domain feature map to obtain a second sum image; the feature summer is also used to send the first sum image and the second sum image to the detector; The detector is used to determine the location of the source domain image detection box and the category of the source domain image based on the first image; the detector is also used to determine the location of the target domain image detection box and the category of the target domain image based on the second image. The basic model further includes a detection loss function, which includes a source domain detection loss function and a target domain detection loss function that are parallel and connected to the output of the detector. The source domain detection loss function is used to determine the source domain detection loss value, and the target domain detection loss function is used to determine the target domain loss value. The feature extraction network, feature decomposition network, feature summer, and detector are collectively referred to as the student model, and the basic model also includes the teacher model and the image enhancement network. The teacher model and the student model have the same structure, and the parameters of the student model are student parameters; after training a predetermined number of times, the student model sends the student parameters to the teacher model; the teacher model receives the student parameters and adjusts its own parameters to be the same as the student parameters; The image enhancement network, the teacher model, and the target domain detection loss function are connected sequentially from input to output. The image enhancement network is used to receive and enhance the target domain image to obtain the target domain enhanced image, and send the target domain enhanced image to the teacher model; The teacher model outputs the bounding box position and category of the target domain augmented image based on the target domain augmented image, and records the bounding box position and category of the target domain augmented image as pseudo-labels; and sends the pseudo-labels to the target domain detection loss function so that the target domain detection loss function calculates the target domain loss value based on the pseudo-labels; The basic model also includes a feature reconstruction network and a reconstruction loss function; The feature reconstruction network receives the second domain style features and the second instance-invariant features output by the feature decomposition network, and reconstructs the target domain image based on the second domain style features and the second instance-invariant features to obtain the reconstructed image, and sends the reconstructed image to the reconstruction loss function; The reconstruction loss function calculates the reconstruction loss value of the reconstructed image based on the reconstructed image and the target domain augmentation image.

2. The method for constructing the basic model according to claim 1, characterized in that, The basic model also includes a mutual information loss function, which is used to receive the input source domain image feature map and target domain image feature map to calculate the mutual information loss value.

3. The method for constructing the basic model according to any one of claims 1-2, characterized in that, The feature extraction network is either ResNet101 or VGG-16.

4. The method for constructing the basic model according to any one of claims 1-2, characterized in that, The detector is Faster R-CNN.

5. A computer electronic device, characterized in that, The device includes: one or more processors, and a memory; The memory is used to store the basic model; The processor is used to train the base model as described in any one of claims 1-4.

6. A computer-readable storage medium, characterized in that, It stores a base model as described in any one of claims 1-4, which can be executed by a processor.

7. A computer program product, characterized in that, Includes the base model as described in any one of claims 1-4, which can be executed by a processor.

Images