Cross-domain target detection method and system, device, and storage medium

Abstract

The present invention relates to a cross-domain target detection method and system, a device, and a storage medium. The method comprises the following steps: converting source-domain image data into new source-domain image data on the basis of CycleGAN; constructing a framework based on mutual learning between a student model and a teacher model, and updating the student model and the teacher model on the basis of the new source-domain image data and a pre-constructed target detection model; and using the teacher model meeting a preset condition as a final model to perform target detection on target-domain image data, to obtain a target detection result. The present invention can be widely used in the technical fields of intelligent algorithms, machine learning, and computer vision.

Description

A cross-domain target detection method, system, device, and storage medium Technical Field

[0001] This invention belongs to the fields of intelligent algorithms, machine learning, and computer vision technology, and relates to a cross-domain target detection method, system, device, and storage medium based on CycleGAN. Background Technology

[0002] With the widespread application of video surveillance systems in all aspects of society, timely and accurate target acquisition in video surveillance has become a research hotspot. Existing target detection technologies have made significant progress with the development of deep convolutional neural networks. However, training an accurate target detector requires substantial investment of human and material resources for annotation. A natural question arises: can we utilize existing labeled data to learn from unlabeled data? How to achieve the best possible target detection performance using these limited labeled or large amounts of unlabeled data has become a recent hot topic in target detection.

[0003] As shown in Figure 1, common cross-domain object detection methods in the industry typically involve first training an initial object detector using source data, then using this detector to infer meaning from the target domain image to generate pseudo-labels, and finally training a target detector specifically for the target domain using the source data and the pseudo-labeled target data. Existing cross-domain object detection methods heavily rely on the accuracy of the pseudo-labels; however, the pseudo-labels are generated only once by the initial object detector, and are not updated or learned throughout the process. Therefore, the resulting object detector has certain limitations. Summary of the Invention

[0004] To address the aforementioned problems, the present invention aims to provide a cross-domain object detection method, system, device, and storage medium based on CycleGAN. By constructing a framework based on mutual learning between student and teacher models, the invention solves the problem of the lack of real labels in the target domain and reduces false detections in the target domain.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: Firstly, the present invention provides a cross-domain target detection method, comprising the following steps:

[0006] 1) Based on CycleGAN, source domain image data is transformed into new source domain image data;

[0007] 2) Construct a framework based on mutual learning between the student model and the teacher model, and update the teacher model through the student model based on new source domain image data and a pre-built object detection model;

[0008] 3) Using the teacher model that meets the preset conditions as the final model, target detection is performed on the target domain image data to obtain the target detection results.

[0009] Furthermore, in step 2), a framework based on mutual learning between the student model and the teacher model is constructed, and the student model and the teacher model are updated based on the new source domain image data and the initial object detection model, including the following steps:

[0010] 2.1) A target detection model is trained based on source domain image data, which serves as the initial model for both the teacher and student models;

[0011] 2.2) Input the target domain image data into the initial teacher model for inference, and obtain the detection boxes with confidence levels that meet the preset requirements as pseudo-label data;

[0012] 2.3) Based on pseudo-label data and new source domain image data, the initial student model is trained to obtain an updated student model;

[0013] 2.4) Update the initial teacher model based on the updated student model, and determine whether the updated teacher model meets the preset conditions. If it does not meet the conditions, return to step 2.2. Otherwise, use the updated teacher model as the final model to perform target detection on the target domain image data.

[0014] Furthermore, in step 2.1), when training the target detection model based on the source domain image data, the Yolov5 detection algorithm is used for training.

[0015] Furthermore, in step 2.4), when updating the initial teacher model based on the updated student model, the EMA method is used.

[0016] Secondly, the present invention provides a cross-domain target detection system, comprising:

[0017] The data conversion module is used to convert source domain image data into new source domain image data based on CycleGAN;

[0018] The model update module is used to build a framework based on mutual learning between the student model and the teacher model. Based on new source domain image data and a pre-built object detection model, the teacher model is updated through the student model.

[0019] The target detection module is used to use a teacher model that meets preset conditions as the final model to perform target detection on the target domain image data and obtain the target detection results.

[0020] Furthermore, the model update module includes:

[0021] The initial model training module is used to train an object detection model based on source domain image data, which serves as the initial model for both the teacher and student models.

[0022] The pseudo-label acquisition module is used to input the target domain image data into the initial teacher model for inference and obtain the detection box with confidence that meets the preset requirements as pseudo-label data.

[0023] The student model update module is used to train the updated student model based on the initial student model using pseudo-labeled data and new source domain image data.

[0024] The teacher model update module is used to update the initial teacher model based on the updated student model, and to determine whether the updated teacher model meets the preset conditions. If it does not meet the conditions, it returns to the pseudo-label acquisition module; otherwise, it uses the updated teacher model as the final model to perform target detection on the target domain image data.

[0025] Thirdly, the present invention provides a computer-readable storage medium for storing one or more programs, said one or more programs including instructions that, when executed by a computing device, cause the computing device to perform any method.

[0026] Fourthly, the present invention provides a computing device comprising: one or more processors and a memory, wherein the memory stores one or more programs and is configured to be executed by the one or more processors, the one or more programs including instructions for performing any method.

[0027] The present invention has the following advantages due to the adoption of the above technical solutions:

[0028] Compared with the detection model method that directly uses pseudo-labels to generate target domains, this invention not only reduces the cost of manually annotating target domain images, but also reduces the false alarm rate of target domains, and improves the accuracy and robustness of target detection algorithms.

[0029] Therefore, this invention can be widely applied in the fields of intelligent algorithms, machine learning, and computer vision. Attached Figure Description

[0030] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Throughout the drawings, the same reference numerals denote the same parts. In the drawings:

[0031] Figure 1 is a flowchart of a conventional cross-domain target detection method;

[0032] Figure 2 is a flowchart of the cross-domain target detection method based on CycleGAN provided in an embodiment of the present invention. The best embodiment of the present invention

[0033] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the described embodiments of the present invention are within the scope of protection of the present invention.

[0034] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0035] To better understand this invention, we will first give a brief introduction to the principles of CycleGAN:

[0036] CycleGAN: CycleGAN is an image transformation model consisting of two generators and two discriminators. Let y represent a sample in domain Y, and x represent a sample in domain X. Distributions X and Y come from different domains. Then CycleGAN contains:

[0037] (1) Generator G: X->Y, generator G is used to implement the migration from X to Y;

[0038] (2) Generator F: Y->X, generator F is used to implement the migration from Y to X;

[0039] (3) Discriminator Dx: Discriminates X and F(y). Discriminator Dx is used to determine whether it is true X or data generated by generator F based on Y that is distributed in the same way as X.

[0040] (4) Discriminator Dy: Discriminates Y and G(x). The discriminator Dy is used to determine whether it is true Y or data generated by the generator G based on X and is distributed in the same way as Y.

[0041] The generator G aims to generate highly realistic samples G(x) to deceive the discriminator Dy, thus minimizing the GAN loss. Conversely, F(y) attempts to deceive the discriminator Dx. In this adversarial process, the generator F(y) becomes increasingly powerful, eventually producing highly realistic samples that the discriminator Dx cannot distinguish. This further reduces the distributional difference between the two domains, ensuring that the samples generated by the generator are distributed similarly to the real samples.

[0042] Based on this, some embodiments of the present invention provide a cross-domain object detection method based on CycleGAN. The architecture for training the object detector is redesigned based on source data and target domain data with pseudo-labels. First, an object detection model is trained using source domain image data, serving as the initial model for both the teacher and student models. Then, the teacher model is used to infer from the target domain images, selecting high-confidence detection boxes as pseudo-labels, and training a new student model using new source domain image data. Finally, the new student model updates the teacher model using EMA (Effective Mapping), continuously iterating and updating both the teacher and student models to improve their detection accuracy and recall in the target domain images. On one hand, the present invention utilizes CycleGAN to generate target domain-style image data, helping the student model learn the features of the target domain. On the other hand, by introducing a mutual learning mechanism between the teacher and student models, they continuously update their parameters together, thereby improving the accuracy of object detection.

[0043] Correspondingly, in other embodiments of the present invention, a cross-domain target detection system, device, and storage medium based on CycleGAN are provided.

[0044] Example 1

[0045] As shown in Figure 2, this embodiment provides a cross-domain target detection method based on CycleGAN, which includes the following steps:

[0046] 1) Based on CycleGAN, source domain image data is transformed into new source domain image data;

[0047] 2) Construct a framework based on mutual learning between the student model and the teacher model, and update the teacher model through the student model based on new source domain image data and a pre-built object detection model;

[0048] 3) Using the teacher model that meets the preset conditions as the final model, target detection is performed on the target domain image data to obtain the target detection results.

[0049] Furthermore, in step 2) above, a framework based on mutual learning between the student model and the teacher model is constructed. The student model and the teacher model are updated based on the new source domain image data and the initial object detection model, including the following steps:

[0050] 2.1) An object detection model is trained based on source domain image data, which serves as the initial model for the teacher model and the student model;

[0051] 2.2) Input the target domain image data into the initial teacher model for inference, and obtain the detection boxes with confidence levels that meet the preset requirements as pseudo-label data;

[0052] 2.3) Based on pseudo-label data and new source domain image data, the initial student model is trained to obtain an updated student model;

[0053] 2.4) Update the initial teacher model based on the updated student model, and determine whether the updated teacher model meets the preset conditions. If it does not meet the conditions, return to step 2.2. Otherwise, use the updated teacher model as the final model to perform target detection on the target domain image data.

[0054] Furthermore, in step 1) above, the source domain image data refers to the real label dataset containing images; the new source domain image data has the same style as the target domain image data and retains the target location information of the source domain image data.

[0055] Furthermore, in step 2.1) above, when training the target detection model based on source domain image data, the general Yolov5 detection algorithm can be used for training, and this embodiment does not impose any restrictions on this.

[0056] Furthermore, in step 2.2 above, the target domain image data refers to the image dataset to be labeled.

[0057] Furthermore, in step 2.4 above, EMA stands for exponential moving average. In each epoch, the teacher model updates its parameters by applying EMA to the weights of the student model.

[0058] Example 2

[0059] The above-described embodiment 1 provides a cross-domain object detection method based on CycleGAN. Correspondingly, this embodiment provides a cross-domain object detection system based on CycleGAN. The system provided in this embodiment can implement the cross-domain object detection method based on CycleGAN of embodiment 1. The system can be implemented through software, hardware, or a combination of both. For example, the system may include integrated or separate functional modules or units to execute the corresponding steps in the methods of embodiment 1. Since the system in this embodiment is basically similar to the method embodiment, the description process in this embodiment is relatively simple. Relevant details can be found in the description of embodiment 1. The system embodiment provided in this embodiment is merely illustrative.

[0060] The CycleGAN-based cross-domain target detection system provided in this embodiment includes:

[0061] The data conversion module is used to convert source domain image data into new source domain image data based on CycleGAN;

[0062] The model update module is used to build a framework based on mutual learning between the student model and the teacher model. Based on new source domain image data and a pre-built object detection model, the teacher model is updated through the student model.

[0063] The target detection module is used to use a teacher model that meets preset conditions as the final model to perform target detection on the target domain image data and obtain the target detection results.

[0064] Furthermore, the model update module includes:

[0065] The initial model training module is used to train an object detection model based on source domain image data, which serves as the initial model for both the teacher and student models.

[0066] The pseudo-label acquisition module is used to input the target domain image data into the initial teacher model for inference and obtain the detection box with confidence that meets the preset requirements as pseudo-label data.

[0067] The student model update module is used to train the updated student model based on the initial student model using pseudo-labeled data and new source domain image data.

[0068] The teacher model update module is used to update the initial teacher model based on the updated student model, and to determine whether the updated teacher model meets the preset conditions. If it does not meet the conditions, it returns to the pseudo-label acquisition module; otherwise, it uses the updated teacher model as the final model to perform target detection on the target domain image data.

[0069] Example 3

[0070] This embodiment provides a processing device corresponding to the CycleGAN-based cross-domain target detection method provided in Embodiment 1. The processing device can be a client-side processing device, such as a mobile phone, laptop, tablet computer, desktop computer, etc., to execute the method of Embodiment 1.

[0071] The processing device includes a processor, a memory, a communication interface, and a bus. The processor, memory, and communication interface are connected via the bus to enable communication between them. The memory stores a computer program that can run on the processor. When the processor runs the computer program, it executes the CycleGAN-based cross-domain target detection method provided in Embodiment 1.

[0072] Preferably, the memory may be high-speed random access memory (RAM), and may also include non-volatile memory, such as at least one disk storage device.

[0073] Preferably, the processor can be any type of general-purpose processor such as a central processing unit (CPU) or a digital signal processor (DSP), and there is no limitation herein.

[0074] Example 4

[0075] The CycleGAN-based cross-domain target detection method of this embodiment 1 can be specifically implemented as a computer program product. The computer program product may include a computer-readable storage medium on which computer-readable program instructions for executing the CycleGAN-based cross-domain target detection method of this embodiment 1 are loaded.

[0076] A computer-readable storage medium can be a tangible device that holds and stores instructions for use by an instruction execution device. A computer-readable storage medium can be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any combination thereof.

[0077] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0078] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in one or more blocks of the flowchart illustrations and / or one or more blocks of the block diagrams.

[0079] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that implement the functions specified in one or more flowcharts and / or one or more block diagrams.

[0080] These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable apparatus, provide steps for implementing the functions specified in one or more flowcharts and / or one or more block diagrams.

[0081] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

[0082] Cross-references to related applications

[0083] This application claims priority to Chinese patent application No. 202411549567.4, filed on November 1, 2024, the entire contents of which are incorporated herein by reference. Industrial applicability

[0084] This invention provides a cross-domain target detection method, system, device, and medium based on CycleGAN. Compared with the target domain detection model method that directly uses pseudo-labels to generate the target domain, this invention not only reduces the cost of manually annotating target domain images, but also reduces the false alarm rate of the target domain, and improves the accuracy and robustness of the target detection algorithm.

[0085] Therefore, this invention can be widely applied in the fields of intelligent algorithms, machine learning, and computer vision.

Claims

1. A cross-domain target detection method, characterized in that, Includes the following steps: 1) Based on CycleGAN, source domain image data is transformed into new source domain image data; 2) Construct a framework based on mutual learning between the student model and the teacher model, and update the teacher model through the student model based on new source domain image data and a pre-built object detection model; 3) Using the teacher model that meets the preset conditions as the final model, target detection is performed on the target domain image data to obtain the target detection results.

2. The cross-domain target detection method as described in claim 1, characterized in that, In step 2), a framework based on mutual learning between the student model and the teacher model is constructed. The student model and the teacher model are updated based on the new source domain image data and the initial object detection model, including the following steps: 2.1) A target detection model is trained based on source domain image data, which serves as the initial model for both the teacher and student models; 2.2) Input the target domain image data into the initial teacher model for inference, and obtain the detection boxes with confidence levels that meet the preset requirements as pseudo-label data; 2.3) Based on pseudo-label data and new source domain image data, the initial student model is trained to obtain an updated student model; 2.4) Update the initial teacher model based on the updated student model, and determine whether the updated teacher model meets the preset conditions. If it does not meet the conditions, return to step 2.

2. Otherwise, use the updated teacher model as the final model to perform target detection on the target domain image data.

3. The cross-domain target detection method as described in claim 2, characterized in that, In step 2.1), when training the target detection model based on the source domain image data, the Yolov5 detection algorithm is used for training.

4. The cross-domain target detection method as described in claim 2, characterized in that, In step 2.4), when updating the initial teacher model based on the updated student model, the EMA method is used.

5. A cross-domain target detection system, characterized in that, include: The data conversion module is used to convert source domain image data into new source domain image data based on CycleGAN; The model update module is used to build a framework based on mutual learning between the student model and the teacher model. Based on new source domain image data and a pre-built object detection model, the teacher model is updated through the student model. The target detection module is used to use a teacher model that meets preset conditions as the final model to perform target detection on the target domain image data and obtain the target detection results.

6. The cross-domain target detection system as described in claim 5, characterized in that, The model update module includes: The initial model training module is used to train an object detection model based on source domain image data, which serves as the initial model for both the teacher and student models. The pseudo-label acquisition module is used to input the target domain image data into the initial teacher model for inference and obtain the detection box with confidence that meets the preset requirements as pseudo-label data. The student model update module is used to train the updated student model based on the initial student model using pseudo-labeled data and new source domain image data. The teacher model update module is used to update the initial teacher model based on the updated student model, and to determine whether the updated teacher model meets the preset conditions. If it does not meet the conditions, it returns to the pseudo-label acquisition module; otherwise, it uses the updated teacher model as the final model to perform target detection on the target domain image data.

7. A computer-readable storage medium for storing one or more programs, characterized in that, The one or more programs include instructions that, when executed by a computing device, cause the computing device to perform any of the methods described in claims 1 to 4.

8. A computing device, characterized in that, include: One or more processors and a memory, wherein the memory stores one or more programs and is configured to be executed by the one or more processors, the one or more programs including instructions for performing any of the methods described in claims 1 to 4.

Images