Cross-domain target detection method, system and device and storage medium

Abstract

The invention relates to a cross-domain target detection method, system and device, and a storage medium. The method comprises the following steps: converting source domain image data into new source domain image data based on CycleGAN; constructing a framework based on mutual learning of a student model and a teacher model, and updating the student model and the teacher model based on the new source domain image data and a pre-constructed target detection model; and taking the teacher model meeting a preset condition as a final model, and performing target detection on the target domain image data to obtain a target detection result. The method can be widely applied to the technical fields of intelligent algorithms, machine learning and computer vision.

Description

Technical Field

[0001] The present invention relates to a cross-domain target detection method, system, device and storage medium based on CycleGAN, and belongs to the technical fields of intelligent algorithms, machine learning and computer vision. Background Art

[0002] With the application of video surveillance systems in all aspects of society, how to obtain the target in video surveillance in a timely and accurate manner has become a research hotspot. Existing target detection has also made significant progress with the development of deep convolutional neural networks. However, training an accurate target detector requires a lot of manpower and material resources for labeling. A natural idea is whether we can use the existing labeled data to learn unlabeled data? How to use these small amounts of labeled or large amounts of unlabeled data to obtain the best possible target detection performance has become a hot task in target detection recently. like Figure 1 As shown in the figure, the cross-domain target detection method in the industry generally first uses the source data to train the first target detector, then uses the target detector to infer the target domain image and generate pseudo labels, and finally uses the source data and the pseudo-labeled target data to train a target detector for the target domain. Existing cross-domain target detection methods strongly rely on the accuracy of pseudo labels, but the pseudo labels are generated once by the first target detector, and the pseudo labels are not updated and learned during the entire process. Therefore, the resulting target detector has certain limitations. Summary of the invention In view of the above problems, the purpose of the present invention is to provide a cross-domain target detection method, system, device and storage medium based on CycleGAN, which solves the problem of lack of real labels in the target domain and reduces false detections in the target domain by constructing a framework based on mutual learning between student model and teacher model.

[0003] To achieve the above object, the present invention adopts the following technical solutions: In a first aspect, the present invention provides a cross-domain target detection method, comprising the following steps: 1) Convert source domain image data into new source domain image data based on CycleGAN; 2) Build a framework based on mutual learning between the student model and the teacher model, and update the teacher model through the student model based on the new source domain image data and the pre-built object detection model; 3) The teacher model that meets the preset conditions is used as the final model, and target detection is performed on the target domain image data to obtain the target detection result.

[0004] Furthermore, in step 2), a framework based on mutual learning between the student model and the teacher model is constructed, and the student model and the teacher model are updated based on the new source domain image data and the initial target detection model, including the following steps: 2.1) Train the target detection model based on the source domain image data as the initial model of the teacher model and the student model; 2.2) Input the target domain image data into the initial teacher model for inference, and obtain the detection box whose confidence meets the preset requirements as pseudo-label data; 2.3) Based on the pseudo-label data and the new source domain image data, the initial student model is trained to obtain an updated student model; 2.4) Update the initial teacher model based on the updated student model, and determine whether the updated teacher model meets the preset conditions. If not, return to step 2.2), otherwise use the updated teacher model as the final model to perform target detection on the target domain image data.

[0005] Furthermore, in the step 2.1), when the target detection model is obtained by training based on the source domain image data, the Yolov5 detection algorithm is used for training.

[0006] Furthermore, in step 2.4), when the initial teacher model is updated based on the updated student model, the EMA method is adopted.

[0007] In a second aspect, the present invention provides a cross-domain target detection system, comprising: A data conversion module, used to convert source domain image data into new source domain image data based on CycleGAN; The model updating module is used to build a framework based on mutual learning between the student model and the teacher model, and to update the teacher model through the student model based on the new source domain image data and the pre-built object detection model; The target detection module is used to use the teacher model that meets the preset conditions as the final model, perform target detection on the target domain image data, and obtain the target detection result.

[0008] Furthermore, the model updating module includes: The initial model training module is used to train the target detection model based on the source domain image data, which serves as the initial model of the teacher model and the student model; The pseudo-label acquisition module is used to input the target domain image data into the initial teacher model for reasoning, and obtain the detection box whose confidence meets the preset requirements as pseudo-label data; A student model updating module is used to train the initial student model based on pseudo-label data and new source domain image data to obtain an updated student model; The teacher model update module is used to update the initial teacher model based on the updated student model and determine whether the updated teacher model meets the preset conditions. If not, it returns to the pseudo-label acquisition module. Otherwise, the updated teacher model is used as the final model to perform target detection on the target domain image data.

[0009] In a third aspect, the present invention provides a computer-readable storage medium storing one or more programs, wherein the one or more programs include instructions, which, when executed by a computing device, cause the computing device to perform any method.

[0010] In a fourth aspect, the present invention provides a computing device, comprising: one or more processors and a memory, wherein the memory stores one or more programs and is configured to be executed by the one or more processors, and the one or more programs include instructions for executing any method.

[0011] Due to the adoption of the above technical scheme, the present invention has the following advantages: compared with the detection model method of the target domain directly generated by using pseudo-labels, the present invention not only reduces the cost of manually annotating the target domain images, but also reduces the false alarm rate of the target domain, and improves the accuracy and robustness of the target detection algorithm.

[0012] Therefore, the present invention can be widely applied to the fields of intelligent algorithms, machine learning, and computer vision technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present invention. Throughout the accompanying drawings, the same reference numerals are used to represent the same components. In the accompanying drawings: Figure 1 This is a flow chart of a conventional cross-domain target detection method; Figure 2 It is a flow chart of a cross-domain target detection method based on CycleGAN provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0014] In order to make the purpose, technical solution and advantages of the embodiment of the present invention clearer, the technical solution of the embodiment of the present invention will be clearly and completely described below in conjunction with the drawings of the embodiment of the present invention. Obviously, the described embodiment is a part of the embodiment of the present invention, not all of the embodiments. Based on the described embodiment of the present invention, all other embodiments obtained by ordinary technicians in this field belong to the scope of protection of the present invention.

[0015] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0016] In order to better understand the present invention, the principle of CycleGAN is briefly introduced first: CycleGAN: CycleGAN is an image conversion model consisting of two generators and two discriminators. Assume that y represents samples in domain Y and x represents samples in domain X. Distribution X and distribution Y are from different domains. Then CycleGAN has: (1) Generator G: X->Y, generator G is used to achieve migration from X to Y; (2) Generator F: Y->X, generator F is used to realize the migration from Y to X; (3) Discriminator Dx: Distinguishes between X and F(y). Discriminator Dx is used to determine whether it is the true X or the data generated by generator F based on Y and has the same distribution as X. (4) Discriminator Dy: Distinguishes Y from G(x). Discriminator Dy is used to determine whether it is the true Y or the data generated by generator G based on X and has the same distribution as Y.

[0017] The generator G hopes to generate as many realistic samples G(x) as possible to deceive the discriminator Dy, so the generator G hopes that the GAN loss is as small as possible. On the contrary, F(y) tries to deceive the discriminator Dx. In the process of confrontation, the generator F(y) and the discriminator Dx become stronger and stronger. Finally, the samples generated by the generator F(y) are so realistic that the discriminator Dx cannot distinguish them. In this way, the distribution difference between the two domains can be further reduced, ensuring that the samples generated by the generator are distributed in the same way as the real samples.

[0018] Based on this, in some embodiments of the present invention, a cross-domain target detection method based on CycleGAN is provided. The architecture of training target detectors is redesigned based on source data and target domain data with pseudo labels. First, a target detection model is trained using source domain image data as the initial model of the teacher model and the student model; then, the teacher model is used to infer the target domain image, and the detection box with higher confidence is selected as the pseudo label, and the new student model is trained in conjunction with the new source domain image data; finally, the new student model updates the teacher model through the EMA method, and the teacher model and the student model are continuously updated iteratively to improve the detection accuracy and recall rate of the teacher model and the student model in the target domain image. On the one hand, the present invention uses CycleGAN to generate image data in the target domain style to help the student model learn the characteristics of the target domain. On the other hand, by introducing a mutual learning method between the teacher model and the student model, the teacher model and the student model are continuously updated together to improve the accuracy of target detection.

[0019] Correspondingly, in some other embodiments of the present invention, a cross-domain target detection system, device and storage medium based on CycleGAN are provided.

[0020] Example 1 like Figure 2 As shown, this embodiment provides a cross-domain target detection method based on CycleGAN, which includes the following steps: 1) Convert source domain image data into new source domain image data based on CycleGAN; 2) Build a framework based on mutual learning between the student model and the teacher model, and update the teacher model through the student model based on the new source domain image data and the pre-built object detection model; 3) The teacher model that meets the preset conditions is used as the final model, and target detection is performed on the target domain image data to obtain the target detection result.

[0021] Furthermore, in the above step 2), a framework based on mutual learning between the student model and the teacher model is constructed, and the student model and the teacher model are updated based on the new source domain image data and the initial target detection model, including the following steps: 2.1) Train the target detection model based on the source domain image data as the initial model of the teacher model and the student model; 2.2) Input the target domain image data into the initial teacher model for inference, and obtain the detection box whose confidence meets the preset requirements as pseudo-label data; 2.3) Based on the pseudo-label data and the new source domain image data, the initial student model is trained to obtain an updated student model; 2.4) Update the initial teacher model based on the updated student model, and determine whether the updated teacher model meets the preset conditions. If not, return to step 2.2), otherwise use the updated teacher model as the final model to perform target detection on the target domain image data.

[0022] Furthermore, in the above step 1), the source domain image data refers to a real labeled dataset containing images; the new source domain image data has the same style as the target domain image data, and retains the target position information of the source domain image data.

[0023] Furthermore, in the above step 2.1), when the target detection model is trained based on the source domain image data, the general Yolov5 detection algorithm can be used for training, and this embodiment does not impose any limitation on this.

[0024] Furthermore, in the above step 2.2), the target domain image data refers to the image dataset to be labeled.

[0025] Furthermore, in the above step 2.4), EMA is an exponential moving average. In each epoch, the teacher model updates the parameters of the teacher model by performing EMA on the weights of the student model.

[0026] Example 2 The above-mentioned embodiment 1 provides a cross-domain target detection method based on CycleGAN, and correspondingly, this embodiment provides a cross-domain target detection system based on CycleGAN. The system provided in this embodiment can implement the cross-domain target detection method based on CycleGAN in embodiment 1, and the system can be implemented by software, hardware, or a combination of software and hardware. For example, the system may include integrated or separate functional modules or functional units to perform the corresponding steps in each method of embodiment 1. Since the system of this embodiment is basically similar to the method embodiment, the process described in this embodiment is relatively simple, and the relevant parts can refer to the partial description of embodiment 1. The embodiment of the system provided in this embodiment is only schematic.

[0027] The cross-domain object detection system based on CycleGAN provided in this embodiment includes: A data conversion module, used to convert source domain image data into new source domain image data based on CycleGAN; The model updating module is used to build a framework based on mutual learning between the student model and the teacher model, and to update the teacher model through the student model based on the new source domain image data and the pre-built object detection model; The target detection module is used to use the teacher model that meets the preset conditions as the final model, perform target detection on the target domain image data, and obtain the target detection result.

[0028] Furthermore, the model updating module includes: Example 3 This embodiment provides a processing device corresponding to the cross-domain target detection method based on CycleGAN provided in this embodiment 1. The processing device may be a processing device for a client, such as a mobile phone, a laptop computer, a tablet computer, a desktop computer, etc., to execute the method of embodiment 1.

[0029] The processing device includes a processor, a memory, a communication interface and a bus, and the processor, the memory and the communication interface are connected through the bus to complete mutual communication. The memory stores a computer program that can be run on the processor, and when the processor runs the computer program, it executes the cross-domain target detection method based on CycleGAN provided in this embodiment 1.

[0030] Preferably, the memory may be a high-speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk memory.

[0031] Preferably, the processor may be a central processing unit (CPU), a digital signal processor (DSP) or other general-purpose processors of various types, which are not limited here.

[0032] Example 4 The cross-domain target detection method based on CycleGAN of this embodiment 1 can be specifically implemented as a computer program product, which may include a computer-readable storage medium carrying computer-readable program instructions for executing the cross-domain target detection method based on CycleGAN described in this embodiment 1.

[0033] Computer readable storage media can be tangible devices that hold and store instructions used by instruction execution devices. Computer readable storage media can be, for example, but not limited to, electronic storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any combination thereof.

[0034] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks. These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 One or more processes and / or boxes Figure 1 A function specified in one or more boxes. These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0035] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A cross-domain target detection method, characterized in that: The following steps are involved: 1) Convert source domain image data into new source domain image data based on CycleGAN; 2) Build a framework based on mutual learning between the student model and the teacher model, and update the teacher model through the student model based on the new source domain image data and the pre-built object detection model; 3) The teacher model that meets the preset conditions is used as the final model, and target detection is performed on the target domain image data to obtain the target detection result.

2. A cross-domain target detection method as claimed in claim 1, characterized in that: In the step 2), a framework based on mutual learning between the student model and the teacher model is constructed, and the student model and the teacher model are updated based on the new source domain image data and the initial target detection model, including the following steps: 2.1) Train the target detection model based on the source domain image data as the initial model of the teacher model and the student model; 2.2) Input the target domain image data into the initial teacher model for inference, and obtain the detection box whose confidence meets the preset requirements as pseudo-label data; 2.3) Based on the pseudo-label data and the new source domain image data, the initial student model is trained to obtain an updated student model; 2.4) Update the initial teacher model based on the updated student model, and determine whether the updated teacher model meets the preset conditions. If not, return to step 2.2), otherwise use the updated teacher model as the final model to perform target detection on the target domain image data.

3. A cross-domain target detection method as claimed in claim 2, characterized in that: In the step 2.1), when the target detection model is obtained by training based on the source domain image data, the Yolov5 detection algorithm is used for training.

4. A cross-domain target detection method as claimed in claim 2, characterized in that: In the step 2.4), when the initial teacher model is updated based on the updated student model, the EMA method is adopted.

5. A cross-domain target detection system, characterized in that: include: A data conversion module, used to convert source domain image data into new source domain image data based on CycleGAN; The model updating module is used to build a framework based on mutual learning between the student model and the teacher model, and to update the teacher model through the student model based on the new source domain image data and the pre-built object detection model; The target detection module is used to use the teacher model that meets the preset conditions as the final model, perform target detection on the target domain image data, and obtain the target detection result.

6. A cross-domain target detection system as claimed in claim 5, characterized in that: The model updating module comprises: The initial model training module is used to train the target detection model based on the source domain image data, which serves as the initial model of the teacher model and the student model; The pseudo-label acquisition module is used to input the target domain image data into the initial teacher model for reasoning, and obtain the detection box whose confidence meets the preset requirements as pseudo-label data; A student model updating module is used to train the initial student model based on pseudo-label data and new source domain image data to obtain an updated student model; The teacher model update module is used to update the initial teacher model based on the updated student model and determine whether the updated teacher model meets the preset conditions. If not, it returns to the pseudo-label acquisition module. Otherwise, the updated teacher model is used as the final model to perform target detection on the target domain image data.

7. A computer-readable storage medium storing one or more programs, characterized in that: The one or more programs include instructions, which, when executed by a computing device, cause the computing device to perform any one of the methods of claims 1 to 4 .

8. A computing device, characterized in that include: One or more processors and a memory, wherein the memory stores one or more programs and is configured to be executed by the one or more processors, wherein the one or more programs include instructions for executing any one of the methods described in claims 1 to 4.

Images