Multiple domain object detection method and apparatus therefor

Abstract

To provide a multiple domain object detection method and an apparatus therefor which can perform effective learning regardless of a domain and can enhance an object detection performance for data of a new domain.SOLUTION: A detection method carried out by a multiple domain object detection apparatus has the steps of: generating a teacher model and a student model; inputting, to the teacher model, an image applied with weak enhancement for a target image; determining whether or not a pseudo label has a value smaller than a preset critical value; if it is determined that the pseudo label has a value smaller than the critical value, subjecting a class corresponding to the pseudo label to negative learning; inputting an image applied with strong enhancement for the target image to the student model; calculating a loss of unsupervised learning by comparing a first prediction result with the pseudo label; updating the teacher model by using an index moving average; and detecting an object in an image of another domain by using the teacher model.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a method and apparatus for multi-domain object detection using category-based domain learning. [Background technology]

[0002] Deep learning models require large amounts of labeled image data to accurately detect or classify objects. However, the process of collecting and creating labeled images has limitations in terms of time and cost. Obtaining data from diverse domains is particularly challenging. Domain adaptation is a technology proposed to solve this problem, and aims to reduce the discrepancy between a labeled source domain and an unlabeled target domain. Generally, domain adaptation works by calculating the difference between the domains based on the feature maps of the entire source and target domain images and training the model.

[0003] However, in the field of object detection, general domain adaptation methods can be difficult to achieve sufficient learning results. This is because each object category has its own unique features, and there are large differences between domains. For example, even the same object can be represented in different ways depending on the domain, such as RGB, IR (Infrared), and thermal images, and it is highly likely that a model will not be able to effectively learn these differences. As a result, it can be difficult to guarantee sufficient performance in the field of object detection using existing domain adaptation methodologies alone. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2023-145404 Summary of the Invention [Problem to be solved by the invention]

[0005] One problem to be solved is to provide a multi-domain object detection method and apparatus that can effectively learn the unique features of objects regardless of the domain and improve object detection performance for data in a new domain. [Means for solving the problem]

[0006] According to one embodiment, a method for detecting a multi-domain object using category-based domain learning is implemented by a computing device including a processor and a memory, the method including the steps of: generating a teacher model and a student model from a pre-trained model by the processor; inputting an image to which weak augmentation is applied to a target image in which an object is to be detected into the teacher model by the processor; determining whether a pseudolabel generated by the teacher model is less than a predetermined threshold; if it is determined that the pseudolabel is less than the threshold, performing negative learning on a class corresponding to the pseudolabel; inputting an image to which strong augmentation is applied to the target image into the student model by the processor; comparing a first prediction result generated from the student model with the pseudolabel by the processor to calculate an unsupervised loss; and calculating an exponential moving average (EMA) defined by the student model by the processor. updating the teacher model using EMA (electron microarray average, electromyographic average); and detecting objects in images of other domains using the teacher model.

[0007] In one embodiment, the method may further include a step in which the processor passes a feature map generated by the student model through a first classifier and then transmits the result to a first head that generates the first prediction result.

[0008] In one embodiment, the step of determining whether the pseudo labels in the teacher model are less than a predetermined threshold value includes the step of determining whether a first pseudo label having the highest class probability value among the pseudo labels is less than the threshold value by the processor, and the step of performing the negative learning may include the step of selecting k pseudo labels (where k is a natural number) from the remaining pseudo labels excluding the first pseudo label if it is determined that the first pseudo label is less than the threshold value; and the step of performing the negative learning on classes corresponding to the first pseudo label and the k pseudo labels by the processor.

[0009] In one embodiment, the step of performing the negative learning may include the step of the processor performing the negative learning based on a negative learning loss according to the following formula:

number

[0010] In one embodiment, the first prediction result may include a class prediction value and a bounding box prediction value.

[0011] In one embodiment, the method may further include a step in which the processor performs pre-learning based on a second discriminator provided in advance.

[0012] In one embodiment, the pre-learning step may include the steps of: the processor inputting a pre-defined dataset into a backbone to generate a feature map; the processor passing the feature map through the second classifier and then transmitting it to a second head that generates a second prediction; and the processor comparing the second prediction with a ground truth to calculate a supervised loss and update weights through an inverse wave.

[0013] In one embodiment, the step of performing the pre-learning may include repeatedly performing the pre-learning for a predetermined number of iterations.

[0014] In one embodiment, the second prediction result may include a class prediction value and a bounding box prediction value.

[0015] In one embodiment, detecting an object in an image of another domain using the teacher model may include detecting the object in an IR domain for an IR (Infrared) image, a thermal image domain for a thermal image, or a LiDAR (Light Detection and Ranging) image for a LiDAR image using the teacher model trained in an RGB domain for an RGB image.

[0016] According to one embodiment, a multi-domain object detection apparatus performs object detection using category-based domain learning by executing one or more instructions loaded into one or more memory devices through one or more processors. The one or more instructions are executed to generate a teacher model and a student model from a pre-trained model, input an image to which weak augmentation is applied for a target image for which an object is to be detected to the teacher model, determine whether a pseudolabel generated by the teacher model is below a predetermined threshold, and if it is determined that the pseudolabel is below the threshold, perform negative learning on a class corresponding to the pseudolabel, input an image to which strong augmentation is applied for the target image to the student model, compare a first prediction result generated by the student model with the pseudolabel to calculate an unsupervised loss, and calculate an exponential moving average (EMA) determined by the student model. The training model can be updated using EMA (Equation Average, EMA) and used to detect objects in images of other domains.

[0017] In one embodiment, the one or more instructions are executed and can further include passing a feature map generated by the student model through a first classifier and then transmitting the result to a first head, which generates the first prediction result.

[0018] In one embodiment, determining whether pseudo labels in the teacher model are less than a predetermined threshold value includes determining whether a first pseudo label having the highest class probability value among the pseudo labels is less than the threshold value, and performing the negative learning may include, if it is determined that the first pseudo label is less than the threshold value, selecting k pseudo labels (where k is a natural number) from the remaining pseudo labels excluding the first pseudo label, and performing the negative learning on classes corresponding to the first pseudo label and the k pseudo labels.

[0019] In one embodiment, performing the negative learning may include performing the negative learning based on a negative learning loss according to the following formula:

number

[0020] In one embodiment, the first prediction result may include a class prediction value and a bounding box prediction value.

[0021] In one embodiment, the one or more instructions are executed and may further include pre-training based on a second pre-defined discriminator.

[0022] In one embodiment, performing the pre-learning may include inputting a pre-established dataset into a backbone to generate a feature map, passing the feature map through the second classifier, and then transmitting the feature map to a second head that generates a second prediction result, comparing the second prediction result with a ground truth to calculate a supervised loss, and updating weights through an inverse wave.

[0023] In one embodiment, performing the pre-learning may include repeatedly performing the pre-learning for a predetermined number of iterations.

[0024] In one embodiment, the second prediction result may include a class prediction value and a bounding box prediction value.

[0025] In one embodiment, detecting an object in an image of another domain using the teacher model may include detecting the object in an IR domain for an IR (Infrared) image, a thermal image domain for a thermal image, or a LiDAR (Light Detection and Ranging) image for a LiDAR image using the teacher model trained in an RGB domain for an RGB image. [Effects of the Invention]

[0026] According to the embodiment, even in a situation where the target domain data is unlabeled, it is possible to provide strong object detection performance in the target domain using only source domain data. In particular, by learning the unique features of the object using a category identifier, it is possible to effectively overcome inter-domain differences and significantly improve object detection performance in the target domain. Furthermore, by using the improved target domain object detector together with the source domain object detector, it is possible to achieve strong and reliable object detection even in various domain environments. [Brief explanation of the drawings]

[0027] [Figure 1] 1 is a block diagram illustrating an apparatus for detecting a multi-domain object according to an embodiment; [Figure 2] 1 is a diagram illustrating an example implementation of a multi-domain object detection device according to an embodiment; [Figure 3]1 is a flowchart illustrating a method for detecting a multi-domain object according to an embodiment. [Figure 4] 1 is a flowchart illustrating a method for detecting a multi-domain object according to an embodiment. [Figure 5] 1 is a flowchart illustrating a method for detecting a multi-domain object according to an embodiment. [Figure 6] FIG. 1 is a diagram illustrating a computing device according to an embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0028] Hereinafter, the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily understand the embodiments of the present invention. However, the present invention may be embodied in various different forms and is not limited to the embodiments described herein. In order to clearly explain the present invention, parts that are not necessary for the explanation will be omitted, and similar parts will be designated by similar reference numerals throughout the specification.

[0029] Throughout the specification and claims, when a part "comprises" a certain element, this does not mean excluding other elements, but may further include other elements, unless specifically stated to the contrary. Terms including ordinal numbers such as first, second, etc. may be used to describe various elements, but the elements are not limited by such terms. Such terms are used only to distinguish one element from another.

[0030] The terms "unit," "device," "module," etc. used in the specification may refer to a unit capable of processing at least one function or operation described in the specification, which may be embodied in hardware, a circuit, software, or a combination of hardware, a circuit, and software. Furthermore, at least some of the configurations or functions of the method and apparatus for detecting multi-domain objects using category-based domain learning according to the embodiments described below may be embodied in a program or software, and the program or software may be stored in a computer-readable recording medium or storage medium.

[0031] FIG. 1 is a block diagram illustrating a multi-domain object detection device according to an embodiment.

[0032] Referring to FIG. 1, the multi-domain object detection apparatus 10 according to an embodiment may execute program code or instructions loaded into one or more memory devices through one or more processors. For example, the multi-domain object detection apparatus 10 may be embodied as a computing device 50 as described below with reference to FIG. 6. In this case, the one or more processors may correspond to processor 510 of the computing device 50, and the one or more memory devices may correspond to memory 530 of the computing device 50. The program code or instructions may be executed by the one or more processors to perform object detection using category-based domain learning. In this specification, the term "module" is used to logically divide the functions performed by the program code.

[0033] The multi-domain object detection device 10 may include a pre-training module 11, a main training module 12, a model updating module 13, and a multi-domain object detection module 14.

[0034] The pre-training module 11 can perform pre-training based on a pre-defined discriminator. The discriminator used for pre-training is also called a source discriminator. In contrast, the discriminator used in the main training described below is also called a target discriminator to distinguish it from the source discriminator.

[0035] In one embodiment, the source classifier and the target classifier may have the same structure. Specifically, the source classifier and the target classifier may learn specific features for the same class (e.g., automobile, person, bicycle, etc.), have the same physical network architecture, and use different weights. In this embodiment, weights of the source classifier trained in the pre-training process may be used as initial values in the main training process, thereby improving training efficiency. However, the scope of the present invention is not limited to the source classifier and the target classifier having the same structure.

[0036] The pre-learning module 11 may input a pre-defined data set into a backbone to generate a feature map. The backbone processes the input data to hierarchically extract features from low to high levels, and may be implemented, for example, with a convolutional neural network structure. The backbone extracts key features, such as the shape, boundary, and texture of an object, to generate a feature map. The feature map stores important information from the input data and can provide base data for object detection and classification in subsequent steps.

[0037] The pre-learning module 11 passes the feature map through a source classifier and then transmits the result to a head that generates a prediction. Here, the prediction may include a class prediction and a bounding box prediction. The head calculates the class and position information of an object included in input data based on the feature map and may be configured, for example, with a fully connected layer or a convolutional layer. The class prediction indicates the probability that the input data belongs to a specific class, and the bounding box prediction may include coordinate information indicating the position and size of the object. The prediction may be combined with the output of the source classifier to improve the accuracy of object detection and classification.

[0038] Specifically, the pre-learning module 11 does not immediately transmit the feature map to the head, but instead passes it through a source classifier before transmitting it to the head. This makes it possible to distinguish which domain the features extracted from the backbone come from, thereby more effectively learning the unique features of each object class. The source classifier distinguishes the domain of the feature map in the initial learning step and, based on this, can strengthen the learning of unique features for each object class. However, as learning progresses, the classifier gradually becomes unable to distinguish between domains, which leads the model to learn only the unique features of the object, regardless of the source domain and the target domain. As a result, the model can generalize and learn the unique features of each class without being affected by differences between domains. This allows a model trained in the source domain to perform well in the target domain.

[0039] The pre-training module 11 can compare the predicted results with the ground truth to calculate a supervised loss and update the weights through a reverse wave. The supervised loss can numerically express the error of the model by calculating the difference between the predicted results (e.g., classes and bounding boxes) and the ground truth. The reverse wave is an algorithm that updates the network weights based on the calculated loss, and can adjust the weights and biases of each layer using, for example, gradient descent.

[0040] The pre-learning module 11 can repeat the above-described pre-learning for a predetermined number of iterations.

[0041] The learning module 12 can generate a teacher model and a student model from a model pre-trained by the pre-training module 11. Here, the teacher model freezes the weights of the pre-trained model and is maintained in an unupdated state during learning, and can mainly serve to generate pseudolabels for target data. The teacher model receives an image to which weak augmentation has been applied as input and generates initial prediction results such as class prediction values and bounding box prediction values. These prediction results can be used as reference data for learning the student model.

[0042] The student model has the same initial weights as the teacher model, but the weights can be updated through inversion during the learning process. The student model receives as input an image to which strong augmentation has been applied, compares it with the pseudo-labels generated by the teacher model to calculate a loss, and can gradually improve domain adaptation and object detection performance based on this. In addition, during the learning process, the weights of the student model are periodically updated with the weights of the teacher model using an exponential moving average (EMA), allowing the teacher model to maintain a more generalized state as learning progresses.

[0043] The learning module 12 may input an image to which weak augmentation has been applied to a target image from which an object is to be detected, to the training model. Here, weak augmentation may refer to performing minor transformations such as brightness adjustment, contrast adjustment, Gaussian blur, etc. while preserving the main features of the input data.

[0044] The learning module 12 determines whether a pseudolabel generated by the teacher model is below a predetermined threshold, and if it is determined that the pseudolabel is below the threshold, it performs negative learning on the class corresponding to the pseudolabel. Negative learning is a process in which the model learns that certain data does not belong to a certain class. In other words, when the reliability of the pseudolabel generated by the teacher model is low, information that does not belong to the class can be learned based on the data, which can help reduce the model's prediction error.

[0045] To determine whether the pseudo labels in the teacher model are below a predetermined threshold, the learning module 12 determines whether a first pseudo label having the highest class probability value among the pseudo labels is below the threshold. If it is determined that the first pseudo label is below the threshold, the learning module 12 selects k pseudo labels (where k is a natural number) from the remaining pseudo labels excluding the first pseudo label, and performs negative learning on the classes corresponding to the first pseudo label and the k pseudo labels.

[0046] Specifically, among the pseudo labels generated by the teacher model, the first pseudo label with the highest class probability value may indicate the possibility that the data belongs to a specific class. However, if the probability value of the first pseudo label is below a set critical value, the teacher model's prediction result may be deemed unreliable. For example, if the model predicts the probability of the "bicycle" class for specific data as 0.4, but the critical value is set to 0.5, the prediction may be deemed unreliable. By further selecting the top k pseudo labels that the model determines to be the most likely from the remaining classes excluding the first pseudo label (the class corresponding to the highest probability value), the negative learning target class can be diversified, allowing the model to better learn the differences between multiple classes.

[0047] In one embodiment, the learning module 12 can perform negative learning based on the negative learning loss according to the following formula:

number

[0048] The learning module 12 can input images to the student model that have strong augmentation applied to the target image.

[0049] The learning module 12 can calculate unsupervised loss by comparing the predicted results generated from the student model with the pseudo-labels. Here, the predicted results can include class predictions and bounding box predictions. Strong augmentation can include various techniques that increase the strength of the transformation on the input image and modify or distort parts of the image, such as rotation, enlargement, reduction, cropping, color jitter, contrast adjustment, and Gaussian noise addition.

[0050] The learning module 12 can pass the feature map generated by the student model through a classifier, i.e., an object classifier, before transmitting it to the head that generates the prediction results. In other words, the learning module 12 can pass the feature map through the object classifier first, rather than immediately transmitting it to the head. This process can improve the object detection performance of the student model in the object domain. The object classifier is trained to distinguish whether the feature map is from the source domain or the target domain, allowing the student model to understand the differences between domains and learn the unique features of objects regardless of the domain. In addition, the feature map passed through the object classifier is adjusted to better reflect the unique features of the object category. Predictions generated based on this can contribute to improving the accuracy of class predictions and bounding box predictions. As learning progresses, the object classifier gradually loses its domain discrimination ability, which can lead to the student model learning the unique features of each class regardless of the boundary between the source domain and the target domain, thereby improving cross-domain adaptation performance. This design allows the learning module 12 to provide consistent object detection performance in various domains.

[0051] The model updating module 13 can update the teacher model using an exponential moving average (EMA) defined in the student model. In this process, through the interaction between the teacher model and the student model, a strong model that can effectively detect objects in the target domain can be generated.

[0052] The multi-domain object detection module 14 can detect objects in images of other domains using a teacher model. For example, the multi-domain object detection module 14 can detect objects in an IR (Infrared) domain for IR images, a thermal image domain for thermal image images, or a LiDAR (Light Detection and Ranging) domain for LiDAR images using a teacher model trained in the RGB domain for RGB images. According to this embodiment, even in a situation where the target domain data is unlabeled, strong object detection performance can be provided in the target domain using only source domain data. In particular, by learning the unique features of objects using a category identifier, inter-domain differences can be effectively overcome, significantly improving object detection performance in the target domain. Furthermore, by using an object detector for the target domain with improved performance together with an object detector for the source domain, strong and reliable object detection can be achieved even in diverse domain environments.

[0053] FIG. 2 is a diagram illustrating an example implementation of a multi-domain object detection device according to an embodiment.

[0054] Referring to FIG. 2 , a multi-domain object detection apparatus according to an embodiment generates a teacher model 21 and a student model 22 from a pre-trained model, inputs an image to which weak augmentation is applied to a target image to the teacher model 21, determines whether a pseudo label 25 generated by the teacher model is below a predetermined critical value, and, if it is determined that the pseudo label 25 is below the critical value, performs negative learning 32 on a class corresponding to the pseudo label 25. Meanwhile, the multi-domain object detection apparatus may input an image to which strong augmentation is applied to a target image to the student model 22, and calculate an unsupervised loss 26 by comparing a prediction result 27 generated from the student model 22 with the pseudo label 25. Thereafter, the multi-domain object detection apparatus may update the teacher model 21 using an exponential moving average determined by the student model 22. In particular, the multi-domain object detection apparatus may pass a feature map generated by the student model 22 through a classifier 31 and then transmit the feature map to a head 24 to generate a prediction result 27.

[0055] On the other hand, in the same figure, the supervised loss 28 can be generated by inputting a pre-established dataset into the backbone in the pre-learning step, generating a feature map, passing it through a classifier 31, and then transmitting it to the head 24 to generate a prediction result 27, which is then compared with the correct answer.

[0056] 3 and 4 are flowcharts illustrating a method for detecting a multi-domain object according to an embodiment.

[0057] Referring to FIG. 3, a multi-domain object detection method according to one embodiment may include the steps of: generating a teacher model and a student model from a pre-trained model (S301); inputting an image to which weak enhancement has been applied to a target image into the teacher model (S302); determining whether a pseudo label generated by the teacher model is less than a predetermined threshold value (S303); and, if it is determined that the pseudo label is less than the threshold value, performing negative learning on a class corresponding to the pseudo label (S304).

[0058] Next, referring to FIG. 4, the multi-domain object detection method may further include a step of inputting an image to which strong enhancement has been applied to a target image into a student model (S401), a step of passing a feature map generated by the student model through a classifier and then transmitting the feature map to a head that generates a prediction result (S402), a step of calculating an unsupervised loss by comparing the prediction result generated from the student model with a pseudo label (S403), a step of updating the teacher model using an exponential moving average determined by the student model (S404), and a step of detecting objects in images of other domains using the teacher model (S405).

[0059] Steps S301 to S304 and steps S401 to S404 correspond to the main learning and model update process, and step S405 corresponds to the process of performing object detection on data of a new domain.

[0060] For more detailed information about the method, please refer to the information about other embodiments described in this specification, and therefore, the overlapping information will be omitted here.

[0061] FIG. 5 is a flowchart illustrating a method for detecting a multi-domain object according to an embodiment.

[0062] 5, a multi-domain object detection method according to an embodiment may include the steps of: inputting a pre-established dataset into a backbone to generate a feature map (S501); passing the feature map through a classifier and then transmitting the result of prediction to a head that generates a prediction (S502); and comparing the result of prediction with a ground truth to calculate a supervised loss and update weights through an inverse wave (S503). Steps S501 to S503 may correspond to a pre-learning process.

[0063] For more detailed information about the method, please refer to the information about other embodiments described in this specification, and therefore, duplicated information will be omitted here.

[0064] FIG. 6 is a diagram illustrating a computing device according to an embodiment. 6, the method and apparatus for detecting a multi-domain object according to the embodiment may be implemented using a computing device 50. The computing device 50 may be implemented as various types of electronic devices, servers, or similar devices, and its functions may be implemented by combining software and hardware.

[0065] Computing device 50 may include at least one of a processor 510, a memory 530, a user interface input device 540, a user interface output device 550, and a storage device 560, which communicate over a bus 520. Computing device 50 may also include a network interface 570 that electrically connects to network 40. Network interface 570 may send and receive signals to and from other entities over network 40.

[0066] The processor 510 may be implemented as various types of computing devices, such as a microcontroller unit (MCU), an application processor (AP), a central processing unit (CPU), a graphic processing unit (GPU), a neural processing unit (NPU), or a quantum processing unit (QPU). The processor 510 is a semiconductor device that executes instructions stored in the memory 530 or the storage device 560, and may play a central role in the system. The program code and data stored in the memory 530 or the storage device 560 instruct the processor 510 to perform specific tasks, thereby enabling the overall operation of the system. The processor 510 may be configured to implement various functions and methods described above with reference to FIGS. 1 to 5.

[0067] The memory 530 and storage device 560 may include various types of volatile or non-volatile storage media for system data storage and access. For example, the memory 530 may include a read only memory (ROM) 531 and a random access memory (RAM) 532. In one embodiment, the memory 530 may be integrated into the processor 510, which may enable very fast data transfer between the memory 530 and the processor 510. In another embodiment, the memory 530 may be located external to the processor 510, which may enable connection to the processor 510 via various data buses or interfaces. This connection may be achieved through various known means, such as a Peripheral Component Interconnect Express (PCIe) interface for high-speed data transfer or a memory controller.

[0068] In one embodiment, at least some of the configurations or functions of the method and apparatus for detecting a multi-domain object according to the embodiment may be embodied as a program or software executed by the computing device 50, and the program or software may be stored in a computer-readable recording medium or storage medium. Specifically, a computer-readable recording medium or storage medium according to an embodiment may have a program recorded thereon for causing a computer including a processor 510 that executes a program or instructions stored in the memory 530 or the storage device 560 to execute steps included in implementing the method and apparatus for detecting a multi-domain object according to the embodiment.

[0069] In one embodiment, at least some of the configurations or functions of the multi-domain object detection method and apparatus according to the embodiment may be implemented using hardware or circuits of the computing device 50, or may be implemented using separate hardware or circuits that can be electrically connected to the computing device 50.

[0070] According to the embodiment, even in a situation where the target domain data is unlabeled, it is possible to provide strong object detection performance in the target domain using only source domain data. In particular, by learning the unique features of the object using a category identifier, it is possible to effectively overcome inter-domain differences and significantly improve object detection performance in the target domain. Furthermore, by using the improved target domain object detector together with the source domain object detector, it is possible to achieve strong and reliable object detection even in various domain environments.

[0071] Although the embodiments of the present invention have been described in detail above, the scope of the present invention is not limited thereto, and various modifications and improvements made by those skilled in the art to which the present invention pertains, using the basic concept of the present invention defined in the following claims, also fall within the scope of the present invention. [Explanation of symbols]

[0072] 10. Multi-domain object detection method and apparatus 11 Pre-Learning Modules 12 learning modules 13 Model Update Module 14 Multi-Domain Object Detection Module 21 Teacher Model 22 Student Model 23, 24 heads 25 pseudo labels 26 Unsupervised Loss 27 Prediction results 28 Supervised Loss 31 Classifier 32 Negative Learning 40 Network 50 Computing Devices 510 processor 520 Bus 530 memory 531 ROM 532 RAM 540 User Interface Input Device 550 User interface output device 560 Storage device 570 Network Interface

Claims

1. 1. A method for multi-domain object detection using category-based domain learning performed by a computing device including a processor and a memory, comprising: generating a teacher model and a student model from the pre-trained model; The processor inputs an image to which weak augmentation is applied to a target image in which an object is to be detected, into the training model; The processor determines whether the pseudolabels generated by the teacher model are less than a predetermined threshold value; If it is determined that the pseudo label is less than the threshold value, the processor performs negative learning on the class corresponding to the pseudo label; The processor inputs an image to the student model, the image having strong augmentation applied to the target image; the processor calculating an unsupervised loss by comparing a first prediction generated from the student model with the pseudo-label; the processor updating the teacher model using an exponential moving average (EMA) determined by the student model; and and detecting an object in an image of another domain using the teacher model.

2. 2. The method of claim 1, further comprising the step of: passing a feature map generated by the student model through a first classifier, and then transmitting the result to a first head that generates the first prediction result.

3. The step of determining whether the pseudo label in the teacher model is less than the predetermined critical value includes: the processor determining whether a first pseudo label having a highest class probability value among the pseudo labels is less than the critical value; The step of performing negative learning includes: If it is determined that the first pseudo label is less than the threshold value, the processor selects k pseudo labels (where k is a natural number) from the remaining pseudo labels excluding the first pseudo label; and The method of claim 1 , further comprising: performing the negative learning on classes corresponding to the first pseudo label and the k pseudo labels.

4. The step of performing negative learning includes:

2. The method of claim 1, further comprising: performing the negative learning based on a negative learning loss according to the following formula: [Equation 1] 【number】

5. The method of claim 2 , wherein the first prediction result includes a class prediction value and a bounding box prediction value.

6. The method of claim 1 , further comprising the step of: the processor pre-learning based on a second discriminator provided in advance.

7. The step of performing pre-learning includes: The processor inputs a predefined dataset into a backbone to generate a feature map; the processor passes the feature map through the second classifier and then transmits it to a second head, which generates a second prediction; and 7. The method of claim 6, further comprising: calculating a supervised loss by comparing the second prediction result with a ground truth, and updating weights through a reverse wave.

8. The step of performing pre-learning includes: The method of claim 6, further comprising repeating the pre-learning for a predetermined number of iterations.

9. The method of claim 7 , wherein the second prediction result includes a class prediction value and a bounding box prediction value.

10. detecting an object in an image of another domain using the teacher model, 2. The method of claim 1, further comprising: detecting the object in an IR (Infrared) domain for an IR image, a thermal image domain for a thermal image, or a Light Detection and Ranging (LiDAR) image for a LiDAR image, using the teacher model trained in an RGB domain for an RGB image.

11. A multi-domain object detection device that performs object detection using category-based domain learning by executing one or more instructions loaded in one or more memory devices through one or more processors, The one or more instructions are executed to generate a teacher model and a student model from the pre-trained model; An image to which weak augmentation is applied is input to the training model, the image being a target image from which an object is to be detected; Determine whether the pseudolabel generated by the teacher model is less than a predetermined threshold value; If it is determined that the pseudo label is less than the threshold value, negative learning is performed on the class corresponding to the pseudo label; inputting an image to which strong augmentation has been applied to the target image into the student model; Calculating an unsupervised loss by comparing a first prediction generated from the student model with the pseudo-label; updating the teacher model using an exponential moving average (EMA) determined by the student model; A multi-domain object detection device, characterized in that the teacher model is used to detect objects in images of other domains.

12. 12. The multi-domain object detection device of claim 11, wherein the one or more instructions are executed, and further comprising passing a feature map generated by the student model through a first classifier and then transmitting the result to a first head that generates the first prediction result.

13. Determining whether the pseudo label in the teacher model is less than the predetermined critical value includes: determining whether a first pseudo label having a highest class probability value among the pseudo labels is less than the threshold value; The negative learning is performed by If it is determined that the first pseudo label is less than the threshold value, the processor selects k pseudo labels (where k is a natural number) from the remaining pseudo labels excluding the first pseudo label; The multi-domain object detection apparatus of claim 11 , further comprising: performing the negative learning on classes corresponding to the first pseudo label and the k pseudo labels.

14. The negative learning is performed by The multi-domain object detection apparatus of claim 11 , further comprising: performing the negative learning based on a negative learning loss according to the following equation: [Equation 1] 【number】

15. The apparatus of claim 12 , wherein the first prediction result includes a class prediction value and a bounding box prediction value.

16. The multi-domain object detection device of claim 11, wherein the one or more instructions are executed, and further comprises pre-learning based on a second discriminator provided in advance.

17. The pre-learning is performed by A pre-defined dataset is input into the backbone to generate a feature map; The feature map is passed through the second classifier and then transmitted to a second head that generates a second prediction; 17. The multi-domain object detection apparatus of claim 16, further comprising: calculating a supervised loss by comparing the second prediction result with a ground truth, and updating weights through a reverse wave.

18. The pre-learning is performed by The multi-domain object detection apparatus according to claim 16, further comprising repeating the pre-learning for a predetermined number of iterations.

19. The apparatus of claim 17 , wherein the second prediction result includes a class prediction value and a bounding box prediction value.

20. Detecting an object in an image of another domain using the teacher model includes:

12. The multi-domain object sensing device of claim 11, further comprising detecting the object in an IR (Infrared) domain for an IR image, a thermal image domain for a thermal image, or a Light Detection and Ranging (LiDAR) image for a LiDAR image, using the teacher model trained in an RGB domain for an RGB image.

Images