Pseudo label generation method, target detection model training method and device

Abstract

The present disclosure provides a pseudo-label generation method and a target detection model training method and device, relates to the technical field of artificial intelligence, and in particular to the technical field of image processing, computer vision, deep learning and the like. The implementation scheme is as follows: a pseudo-label generation method comprises the following steps: obtaining at least one first detection box by performing first target detection on image data without labels, and obtaining at least one second detection box by performing second target detection on the image data without labels; determining an independent detection box in the detection boxes comprising the at least one first detection box and the at least one second detection box, wherein the independent detection box has no corresponding relationship with the remaining detection boxes in the detection boxes comprising the at least one first detection box and the at least one second detection box except the independent detection box; and determining the independent detection box as a pseudo-label of the image data without labels.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, particularly to the fields of image processing, computer vision, and deep learning, and specifically to a method for generating pseudo-labels and a method, apparatus, electronic device, computer-readable storage medium, and computer program product for training an object detection model. Background Technology

[0002] Artificial intelligence (AI) is the study of enabling computers to simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, and planning). It encompasses both hardware and software technologies. AI hardware technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, and big data processing. AI software technologies mainly include computer vision, speech recognition, natural language processing, machine learning / deep learning, big data processing, and knowledge graph technologies.

[0003] In computer vision, object detection has a wide range of applications. Tasks such as image segmentation, object tracking, and keypoint detection typically rely on object detection. Object detection is generally achieved through a trained object detection model. Training this model requires labeling the training image data, i.e., generating labels. This labeling process is usually done manually, for example, manually marking the target object as a bounding box. However, in practical applications, factors such as high labor costs may lead to a lack of such labeled training data, thus limiting the training of the model.

[0004] The methods described in this section are not necessarily methods that had been previously conceived or adopted. Unless otherwise specified, no method described in this section should be assumed to be prior art simply because it is included in this section. Similarly, unless otherwise specified, the issues mentioned in this section should not be considered to be accepted in any prior art. Summary of the Invention

[0005] This disclosure provides a method for generating pseudo-labels and a method, apparatus, electronic device, computer-readable storage medium, and computer program product for training object detection models.

[0006] According to one aspect of this disclosure, a method for generating pseudo-labels is provided, comprising: obtaining at least one first detection box by performing a first target detection on unlabeled image data, and obtaining at least one second detection box by performing a second target detection on the unlabeled image data; determining an independent detection box among the detection boxes including the at least one first detection box and the at least one second detection box, wherein the independent detection box does not correspond to any other detection boxes in the detection boxes including the at least one first detection box and the at least one second detection box; and determining the independent detection box as a pseudo-label for the unlabeled image data.

[0007] According to another aspect of this disclosure, a method for training an object detection model is provided, comprising: acquiring image training data with pseudo-labels, wherein the pseudo-labels are generated according to the pseudo-label generation method described above; wherein a first object detection is performed via a pre-trained first teacher model, and a second object detection is performed via a pre-trained second teacher model; training a first student model corresponding to the first teacher model based on the image training data with pseudo-labels and image training data with artificial labels to obtain first student model update parameters of the first student model, and training a second student model corresponding to the second teacher model to obtain second student model update parameters of the second student model; and updating the first teacher model based on the first student model update parameters and a first current model parameter of the first teacher model, and updating the second teacher model based on the second student model update parameters and a second current model parameter of the second teacher model.

[0008] According to another aspect of this disclosure, a pseudo-label generation apparatus is provided, comprising: a target detection module configured to obtain at least one first detection box by performing a first target detection on unlabeled image data, and to obtain at least one second detection box by performing a second target detection on the unlabeled image data; a first determination module configured to determine an independent detection box among the detection boxes including the at least one first detection box and the at least one second detection box, wherein the independent detection box does not correspond to any other detection boxes in the detection boxes including the at least one first detection box and the at least one second detection box; and a first processing module configured to determine the independent detection box as a pseudo-label of the unlabeled image data.

[0009] According to another aspect of this disclosure, an object detection model training apparatus is provided, comprising: a training data acquisition module configured to acquire image training data with pseudo-labels, wherein the pseudo-labels are generated according to the pseudo-label generation apparatus described above, wherein a first object detection is performed via a pre-trained first teacher model, and a second object detection is performed via a pre-trained second teacher model; a student model training module configured to train a first student model corresponding to the first teacher model to obtain first student model update parameters of the first student model based on the image training data with pseudo-labels and image training data with artificial labels, and to train a second student model corresponding to the second teacher model to obtain second student model update parameters of the second student model; and a teacher model updating module configured to update the first teacher model based on the first student model update parameters and a first current model parameter of the first teacher model, and to update the second teacher model based on the second student model update parameters and a second current model parameter of the second teacher model.

[0010] According to another aspect of this disclosure, an electronic device is provided, including at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method described above.

[0011] According to another aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are used to cause a computer to perform the methods described above.

[0012] According to another aspect of this disclosure, a computer program product is provided, including a computer program, wherein the computer program, when executed by a processor, implements the method described above.

[0013] According to one or more embodiments of this disclosure, pseudo-labels for unlabeled image data can be accurately generated, thereby improving the training effect of object detection models.

[0014] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0015] The accompanying drawings exemplify embodiments and form part of the specification, serving together with the textual description to explain exemplary implementations of the embodiments. The illustrated embodiments are for illustrative purposes only and do not limit the scope of the claims. Throughout the drawings, the same reference numerals refer to similar but not necessarily identical elements.

[0016] Figure 1 A schematic diagram of an exemplary system in which various methods described herein may be implemented according to embodiments of this disclosure is shown;

[0017] Figure 2 A schematic diagram of a detection frame for target detection according to an embodiment of the present disclosure is shown;

[0018] Figure 3 A flowchart of a pseudo-tag generation method according to an embodiment of the present disclosure is shown;

[0019] Figures 4A to 4C A schematic diagram illustrating an example of an independent detection frame according to an embodiment of the present disclosure is shown;

[0020] Figure 5 A flowchart illustrating the independent detection frame determination step according to an embodiment of the present disclosure is shown;

[0021] Figure 6 A schematic diagram illustrating an example scenario of the independent detection frame determination step according to an embodiment of the present disclosure is shown;

[0022] Figure 7 A flowchart of a target detection model training method according to an embodiment of the present disclosure is shown;

[0023] Figure 8 A schematic diagram of a target detection model training method according to an embodiment of the present disclosure is shown;

[0024] Figure 9 A structural block diagram of a pseudo-label generation apparatus according to an embodiment of the present disclosure is shown;

[0025] Figure 10 A structural block diagram of a pseudo-label generation apparatus according to another embodiment of the present disclosure is shown;

[0026] Figure 11 A structural block diagram of a target detection model training apparatus according to an embodiment of the present disclosure is shown;

[0027] Figure 12 A structural block diagram of an exemplary electronic device that can be used to implement embodiments of the present disclosure is shown. Detailed Implementation

[0028] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0029] In this disclosure, unless otherwise stated, the use of terms such as "first," "second," etc., to describe various elements is not intended to limit the positional, temporal, or importance relationships of these elements; such terms are merely used to distinguish one element from another. In some examples, the first element and the second element may refer to the same instance of that element, while in other cases, based on the context, they may refer to different instances.

[0030] The terminology used in the description of the various examples described in this disclosure is for the purpose of describing particular examples only and is not intended to be limiting. Unless the context explicitly indicates otherwise, an element may be one or more unless the number of elements is specifically limited. Furthermore, the term "and / or" as used in this disclosure covers any one of the listed items and all possible combinations thereof.

[0031] In recent years, in order to address the problem of the lack of labeled training data in practical applications, semi-supervised training methods have been proposed in object detection technologies. These methods generate pseudo-labels for unlabeled training data in order to utilize this unlabeled training data for model training.

[0032] Traditional semi-supervised training methods typically involve a combination of a teacher model and a student model. For example, the teacher model can first predict unlabeled training data, and the predictions can be used as pseudo-labels for this unlabeled data. Then, another model with the same structure as the teacher model can be used as the student model to train and update it. Here, labeled training data, unlabeled training data, and the generated pseudo-labels can be input into the student model to train it. The trained parameters are then used to update the corresponding teacher model, thus completing the training process.

[0033] However, traditional semi-supervised training methods may suffer from error accumulation. Since the teacher and student models have identical structures, and the teacher model's parameters are obtained through training the student model, it becomes difficult to correct errors made by the teacher model during pseudo-label generation (such as missing some targets in object detection). This can lead to poor object detection performance in the trained model.

[0034] To address the aforementioned technical issues, this disclosure proposes a pseudo-label generation method and a target detection model training method.

[0035] Before describing the embodiments of this disclosure in detail, firstly, in conjunction with Figure 1 Exemplary systems in which the various methods described herein can be implemented are introduced.

[0036] Figure 1 A schematic diagram of an exemplary system 100 in which the various methods and apparatus described herein can be implemented according to embodiments of this disclosure is shown. Reference Figure 1 The system 100 includes one or more client devices 101, 102, 103, 104, 105 and 106, a server 120, and one or more communication networks 110 coupling the one or more client devices to the server 120. The client devices 101, 102, 103, 104, 105 and 106 can be configured to execute one or more applications.

[0037] In embodiments of this disclosure, server 120 may run one or more services or software applications that enable the execution of methods for generating pseudo-labels and training object detection models.

[0038] In some embodiments, server 120 may also provide other services or software applications, which may include non-virtual and virtual environments. In some embodiments, these services may be provided as web-based services or cloud services, such as to users of client devices 101, 102, 103, 104, 105, and / or 106 under a Software as a Service (SaaS) model.

[0039] exist Figure 1 In the configuration shown, server 120 may include one or more components that implement the functions performed by server 120. These components may include software components, hardware components, or combinations thereof that can be executed by one or more processors. Users operating client devices 101, 102, 103, 104, 105, and / or 106 can sequentially interact with server 120 using one or more client applications to utilize the services provided by these components. It should be understood that various different system configurations are possible and may differ from system 100. Therefore, Figure 1 This is an example of a system used to implement the various methods described herein, and is not intended to be limiting.

[0040] Users can use client devices 101, 102, 103, 104, 105, and / or 106 to provide training image data, i.e., training data. The client devices can provide an interface that allows users to interact with the client devices. The client devices can also output information to the user through this interface. Although... Figure 1 Only six client devices are described, but those skilled in the art will understand that this disclosure can support any number of client devices.

[0041] Client devices 101, 102, 103, 104, 105, and / or 106 may include various types of computer devices, such as portable handheld devices, general-purpose computers (such as personal computers and laptops), workstation computers, wearable devices, smart screen devices, self-service terminal devices, service robots, gaming systems, thin clients, various messaging devices, sensors, or other sensing devices. These computer devices can run various types and versions of software applications and operating systems, such as Microsoft Windows, Apple iOS, UNIX-like operating systems, Linux or Linux-like operating systems (such as Google Chrome OS); or include various mobile operating systems, such as Microsoft Windows Mobile OS, iOS, Windows Phone, and Android. Portable handheld devices may include cellular phones, smartphones, tablets, personal digital assistants (PDAs), etc. Wearable devices may include head-mounted displays (such as smart glasses) and other devices. Gaming systems may include various handheld gaming devices, internet-enabled gaming devices, etc. Client devices are capable of executing various applications, such as various internet-related applications, communication applications (such as email applications), short message service (SMS) applications, and can use various communication protocols.

[0042] Network 110 can be any type of network well known to those skilled in the art, and can use any of a variety of available protocols (including but not limited to TCP / IP, SNA, IPX, etc.) to support data communication. By way of example only, one or more networks 110 can be a local area network (LAN), an Ethernet-based network, a token ring network, a wide area network (WAN), the Internet, a virtual network, a virtual private network (VPN), an intranet, an extranet, a blockchain network, a public switched telephone network (PSTN), an infrared network, a wireless network (e.g., Bluetooth, WIFI), and / or any combination of these and / or other networks.

[0043] Server 120 may include one or more general-purpose computers, special-purpose server computers (e.g., PC (personal computer) servers, UNIX servers, mid-range servers), blade servers, mainframe computers, server clusters, or any other suitable arrangement and / or combination. Server 120 may include one or more virtual machines running a virtual operating system, or other computing architectures involving virtualization (e.g., one or more flexible pools of logical storage devices that can be virtualized to maintain virtual storage devices for servers). In various embodiments, server 120 may run one or more services or software applications that provide the functionality described below.

[0044] The computing unit in server 120 can run one or more operating systems, including any of the aforementioned operating systems and any commercially available server operating system. Server 120 can also run any of a variety of additional server applications and / or middleware applications, including HTTP servers, FTP servers, CGI servers, JAVA servers, database servers, etc.

[0045] In some implementations, server 120 may include one or more applications to analyze and merge data feeds and / or event updates received from users of client devices 101, 102, 103, 104, 105 and / or 106. Server 120 may also include one or more applications to display data feeds and / or real-time events via one or more display devices of client devices 101, 102, 103, 104, 105 and / or 106.

[0046] In some implementations, server 120 can be a server for a distributed system or a server integrated with blockchain. Server 120 can also be a cloud server, or an intelligent cloud computing server or intelligent cloud host with artificial intelligence technology. A cloud server is a host product in the cloud computing service system, designed to address the shortcomings of traditional physical hosts and Virtual Private Server (VPS) services, such as high management difficulty and weak business scalability.

[0047] System 100 may also include one or more databases 130. In some embodiments, these databases may be used to store data and other information. For example, one or more of the databases 130 may be used to store information such as audio files and video files. Databases 130 may reside in various locations. For example, a database used by server 120 may be local to server 120, or it may be located away from server 120 and may communicate with server 120 via a network-based or dedicated connection. Databases 130 may be of different types. In some embodiments, the database used by server 120 may be, for example, a relational database. One or more of these databases may store, update, and retrieve data from and from the databases in response to commands.

[0048] In some embodiments, one or more of the databases 130 may also be used by an application to store application data. The databases used by the application may be of different types, such as key-value stores, object stores, or regular stores supported by a file system.

[0049] Figure 1The system 100 can be configured and operated in various ways to enable the application of the various methods and apparatus described in this disclosure.

[0050] Figure 2 A schematic diagram of a detection frame for target detection according to an embodiment of the present disclosure is shown.

[0051] like Figure 2 As shown, image 200 and detection boxes 210, 220, and 230 displayed on image 200 are schematically illustrated. Detection boxes, also commonly referred to as bounding boxes, contain the target within a rectangle-like structure.

[0052] In the example, image 200 can be training data used to train an object detection model. According to one example, detection boxes 210, 220, and 230 can represent manually labeled data, in which case image 200 can represent training data with manually labeled 210, 220, and 230. According to another example, detection boxes 210, 220, and 230 can represent pseudo-labels generated by the methods of embodiments of this disclosure, in which case image 200 can represent training data with pseudo-labels 210, 220, and 230.

[0053] In the example, target category indication regions 215, 225, and 235 can also be displayed near detection boxes 210, 220, and 230. Target category indication regions 215, 225, and 235 can be used to indicate the target category and confidence level of detection boxes 210, 220, and 230, respectively. The target category can refer to the class to which the target contained in the detection box belongs. The confidence level can refer to the probability of correctly detecting the target. For example, target category indication region 215 can indicate that the target category of detection box 210 is "person" and the probability of correctly detecting the target is "0.85". Similarly, target category indication regions 225 and 235 can indicate that the target category of detection boxes 220 and 230 is "dog" and the probabilities of correctly detecting the target are "0.72" and "0.55", respectively.

[0054] In the example, the position coordinates of the four vertices of detection boxes 210, 220, and 230 are available. Figure 2 The position coordinates of four vertices A, B, C and D are shown in the example of detection box 230, as shown as (x1,y1), (x2,y1), (x1,y2) and (x2,y2).

[0055] Those skilled in the art will understand that, for purposes of illustration, Figure 2 This is just an example. In practical applications, the target categories of the detection boxes can be various, depending on the preset detection targets of the object detection model, such as motor vehicles, bicycles, and pedestrians on the road.

[0056] Figure 3 A flowchart of a pseudo-label generation method 300 according to an embodiment of the present disclosure is shown.

[0057] In this disclosure, the term "pseudo-label" can be used in contrast to "human-labeled" labels. In examples, such as... Figure 2 The detection boxes 210, 220, and 230 shown can represent pseudo tags generated by the pseudo tag generation method 300 of this disclosure embodiment. In this case, as combined with Figure 2 The image 200 shown can represent image data with pseudo-labels 210, 220 and 230.

[0058] like Figure 3 As shown, the pseudo-label generation method 300 according to an embodiment of the present disclosure may include steps S302, S304 and S306.

[0059] In step S302, at least one first detection box is obtained by performing a first target detection on the unlabeled image data, and at least one second detection box is obtained by performing a second target detection on the unlabeled image data.

[0060] In the example, the first object detection can be performed by a first object detection model, while the second object detection can be performed by a second object detection model. The first and second object detection models can have different model structures, such as different backbone structures.

[0061] In the example, the first object detection model can be a first teacher model in a semi-supervised training method, which may have initial model parameters obtained from a corresponding first student model. The second object detection model can be a second teacher model different from the first teacher model, which may have initial model parameters obtained from a corresponding second student model. Therefore, object detection can be performed on unlabeled image data via the first teacher model and the second teacher model respectively, to generate at least one first detection box and at least one second detection box.

[0062] In the example, step S302 may further include obtaining at least one third detection box by performing a third object detection on the unlabeled image data. That is, more than two different object detections may be performed in step S302.

[0063] Unlike traditional methods for generating pseudo-labels, the pseudo-label generation method 300 according to embodiments of this disclosure reduces or avoids missed target detections by performing at least two different target detections.

[0064] In step S304, an independent detection box is determined among the detection boxes including the at least one first detection box and the at least one second detection box. This independent detection box does not correspond to the other detection boxes in the detection boxes including the at least one first detection box and the at least one second detection box.

[0065] In this disclosure, a lack of correspondence between two detection boxes may mean that the two detection boxes do not contain the same target. Conversely, a correspondence between two detection boxes may mean that the two detection boxes contain the same target.

[0066] In other words, an independent detection box can refer to one or more first detection boxes that do not correspond to any of the at least one first detection box. Alternatively, an independent detection box can also refer to one or more second detection boxes that do not correspond to any of the at least one second detection box.

[0067] Since there cannot be a direct correspondence between two first detection frames or two second detection frames, when an independent detection frame exists within the at least one first detection frame, this independent detection frame does not correspond to any of the other detection frames in the at least one first detection frame or the at least one second detection frame. Alternatively, when an independent detection frame exists within the at least one second detection frame, this independent detection frame does not correspond to any of the other detection frames in the at least one first detection frame or the at least one second detection frame.

[0068] In the example, whether two bounding boxes correspond can be determined by their degree of overlap. For instance, the degree of overlap can be characterized by IOU (Intersection-over-Union). IOU can be calculated based on the position coordinates of each bounding box. (The last sentence appears to be incomplete and possibly refers to a different example.) Figure 2 As shown, when obtaining a detection box, for example, through object detection, the position coordinates of the four vertices of the detection box (e.g., Figure 2 The position coordinates of the four vertices A, B, C, and D shown can also be obtained. Therefore, IOU can be calculated using the position coordinates. Considering that the method of calculating IOU is known in this field, its details will not be elaborated here. Generally speaking, it can be considered that when the IOU value of two detection boxes is greater than 0.5, the degree of overlap is high, that is, it is very likely that the same target is bounded; while when the IOU value is less than or equal to 0.5, the degree of overlap is low, that is, it is unlikely that the same target is bounded.

[0069] In the example, whether two bounding boxes correspond can be further determined by the target categories indicated by the two bounding boxes. However, in real-world applications, the IOU value of the two bounding boxes might be greater than 0.5 because the two targets are too close in location (e.g., a person sitting on a chair; because the person and the chair are very close, the IOU value of the bounding box for the person and the bounding box for the chair is greater than 0.5). This makes it possible to determine the correspondence solely based on the IOU value. Therefore, the target categories indicated by the two bounding boxes can be used for further determination. That is, if the two bounding boxes indicate different target categories, even if their IOU value is greater than 0.5, it means that the two bounding boxes do not contain the same target.

[0070] Step S304 for determining independent detection boxes can be understood as determining missed detections in the first and second target detections through cross-validation. For example, if an independent detection box exists among the at least one first detection box, it means that there is no corresponding detection box among the at least one second detection box obtained through the second target detection, i.e., the second target detection has missed detections. Conversely, if an independent detection box exists among the at least one second detection box, it means that there is no corresponding detection box among the at least one first detection box obtained through the first target detection, i.e., the first target detection has missed detections. Alternatively, if an independent detection box exists among the at least one first detection box, and an independent detection box also exists among the at least one second detection box, it indicates that both the first and second target detections have missed detections.

[0071] In step S306, the independent detection box is determined as a pseudo-label of the unlabeled image data.

[0072] This step can recall missed detections, thereby generating more accurate pseudo-labels for the unlabeled image data.

[0073] In the pseudo-label generation method 300 according to the embodiments of the present disclosure, by performing at least two different target detections on the same image data, it is convenient to find potential targets that may be missed; at the same time, by determining the independent detection boxes among the detection boxes obtained by the at least two different target detections, the missed targets can be found and recalled, thereby facilitating the generation of more accurate pseudo-labels, and thus providing higher quality training data to improve training effect.

[0074] Figures 4A to 4C A schematic diagram illustrating an example of an independent detection frame according to an embodiment of this disclosure is shown.

[0075] like Figure 4AAs shown, four detection boxes 411-1, 411-2, 411-3, and 411-4 were obtained by performing a first target detection on image 410. Detection box 411-1 contains target A, detection box 411-2 contains target B, detection box 411-3 contains target C, and detection box 411-4 contains target D. That is, four targets A, B, C, and D in image 410 were detected by the first target detection.

[0076] Furthermore, three detection boxes 412-1, 412-3, and 412-4 were obtained by performing a second target detection on image 410. Detection box 412-1 contains target A, detection box 412-3 contains target C, and detection box 412-4 contains target D. That is, three targets A, C, and D in image 410 were detected by the second target detection.

[0077] Therefore, it can be determined that detection box 411-2 out of the total seven detection boxes is an independent detection box, and this detection box 411-2 has no corresponding relationship with the other detection boxes. In other words, target B was detected by the first target detection, but not by the second target detection, so target B is a missed target.

[0078] In this case, the detection box 411-2, which is an independent detection box, can be determined as the label of image 410 to achieve recall of missed targets.

[0079] Figure 4B It shows the relationship with Figure 4A A similar example. Three detection boxes 421-1, 421-3, and 421-4 were obtained by performing a first object detection on image 420. Detection box 421-1 contains object A, detection box 421-3 contains object C, and detection box 421-4 contains object D. That is, three objects A, C, and D in image 420 were detected via the first object detection.

[0080] Furthermore, four detection boxes 422-1, 422-2, 422-3, and 422-4 were obtained by performing a second object detection on image 420. Detection box 422-1 contains object A, detection box 422-2 contains object B, detection box 422-3 contains object C, and detection box 422-4 contains object D. That is, four objects A, B, C, and D in image 420 were detected through the second object detection.

[0081] Therefore, it can be determined that detection box 422-2 out of the total seven detection boxes is an independent detection box, and this detection box 422-2 has no corresponding relationship with the other detection boxes. In other words, target B was detected by the second target detection, but not by the first target detection, so target B is a missed target.

[0082] In this case, the detection box 422-2, which is an independent detection box, can be determined as the label of image 420 to achieve recall of missed targets.

[0083] Figure 4A and Figure 4B The examples show cases where either the first or second target detection method misses a target. Figure 4C This illustrates a situation where both the first and second target detection methods miss detections.

[0084] like Figure 4C As shown, four detection boxes 431-1, 431-2, 431-3, and 431-4 were obtained by performing a first target detection on image 430. Detection box 431-1 contains target A, detection box 431-2 contains target B, detection box 431-3 contains target C, and detection box 431-4 contains target D. That is, four targets A, B, C, and D in image 430 were detected by the first target detection.

[0085] Furthermore, four detection boxes 432-1, 432-3, 432-4, and 432-5 were obtained by performing a second object detection on image 430. Detection box 432-1 contains object A, detection box 432-3 contains object C, detection box 432-4 contains object D, and detection box 432-5 contains object E. That is, four objects A, C, D, and E in image 430 were detected through the second object detection.

[0086] Therefore, it can be determined that detection boxes 431-2 and 432-5 out of a total of eight detection boxes are independent detection boxes. Detection box 431-2 has no correspondence with the other detection boxes, and detection box 432-5 also has no correspondence with the other detection boxes. In other words, target B is detected by the first target detection, but not by the second target detection, so target B is a missed target. At the same time, target E is detected by the second target detection, but not by the first target detection, so target E is also a missed target. That is, both the first and second target detection methods have missed detections.

[0087] In this case, detection boxes 431-2 and 432-5, which are independent detection boxes, can be identified as labels for image 430 to achieve recall of missed targets.

[0088] Those skilled in the art will understand that, for purposes of illustration, Figures 4A to 4C This is for illustrative purposes only. The number of detection boxes and the number of individual detection boxes determined can vary depending on the specific circumstances and are not limited to this. Figures 4A to 4C As shown.

[0089] Figure 5A flowchart of step 500, which involves determining an independent detection frame according to an embodiment of this disclosure, is shown.

[0090] Here, the independent detection box determination step 500 can correspond to the combination... Figure 3 In step S304, at least one first detection box and at least one second detection box are determined as an independent detection box, which has no corresponding relationship with the remaining detection boxes. Such an independent detection box can be like... Figures 4A to 4C As shown in the independent detection boxes 411-2, 422-2, 431-2 and 432-5.

[0091] like Figure 5 As shown, the independent detection frame determination step 500 may include steps S502 and S504. In step S502, the degree of overlap between each first detection frame in the at least one first detection frame and each second detection frame in the at least one second detection frame can be calculated. In step S504, an independent detection frame can be determined based on the degree of overlap.

[0092] Since the at least one first detection box and the at least one second detection box are obtained through first target detection and second target detection, it is not necessary to directly determine whether each detection box detects the correct target in order to determine the correctness of the detection. Instead, the correctness of the detection can be determined indirectly through pairwise comparison or cross-validation between the first detection box and the second detection box.

[0093] In this way, it is easy and accurate to find out if there are any missed detections in the detection box, that is, to identify independent detection boxes that do not correspond to any other detection boxes.

[0094] In the example, step S502 can be performed by traversing the at least one first detection box and the at least one second detection box. That is, each first detection box in the at least one first detection box and each second detection box in the at least one second detection box can be traversed to determine whether there is an independent detection box that does not correspond to any of the other detection boxes.

[0095] For example, suppose there are 10 first detection boxes and 10 second detection boxes. The overlap between the first of the 10 first detection boxes and each of the 10 second detection boxes can be calculated individually (i.e., calculated 10 times). Similarly, after calculating the overlap between each of the 10 first detection boxes and each of the 10 second detection boxes (i.e., calculated 100 times), pairwise comparisons or cross-validations between the at least one first detection box and the at least one second detection box can be performed. Therefore, it can be determined whether there exists an independent detection box that does not correspond to any of the other detection boxes.

[0096] In the example, calculating the degree of overlap can include calculating the Intersection over Union (IOU). As mentioned earlier, IOU can be a parameter used to characterize the degree of overlap. Generally, it can be considered that the degree of overlap is high when the IOU value of two detection boxes is greater than 0.5, and low when the IOU value is less than or equal to 0.5. Therefore, according to one example, the IOU threshold can be set to 0.5 to determine the degree of overlap.

[0097] like Figure 5 As shown, step S504 may include any one of the first case S504-1 to the fourth case S504-4.

[0098] According to some embodiments, the first case S504-1 may include: in response to the fact that the degree of overlap between a first detection frame in the at least one first detection frame and each second detection frame in the at least one second detection frame is less than or equal to a predetermined threshold, the first detection frame is determined as an independent detection frame.

[0099] Therefore, it is easy and accurate to find the targets that have been missed.

[0100] In the example, the first case S504-1 can correspond to the combination as follows. Figure 4A The example shown illustrates this. By calculating the Intersection over Union (IOU) between each of the four first detection boxes 411-1, 411-2, 411-3, and 411-4 and each of the three second detection boxes 412-1, 412-3, and 412-4, it can be determined that the overlap between the first detection box 411-2 and each of the three second detection boxes 412-1, 412-3, and 412-4 is less than a predetermined threshold, for example, 0.5. Therefore, the first detection box 411-2 can be identified as an independent detection box.

[0101] According to some embodiments, the second case S504-2 may include: in response to the fact that the degree of overlap between a second detection frame in the at least one second detection frame and each first detection frame in the at least one first detection frame is less than or equal to a predetermined threshold, the second detection frame is determined as an independent detection frame.

[0102] Therefore, it is easy and accurate to find the targets that have been missed.

[0103] In the example, the second case S504-2 can correspond to the combination as follows. Figure 4BThe example shown illustrates this. By calculating the Intersection over Union (IOU) between each of the three first detection boxes 421-1, 421-3, and 421-4 and each of the four second detection boxes 422-1, 422-2, 422-3, and 422-4, it can be determined that the overlap between the second detection box 422-2 and each of the three first detection boxes 421-1, 421-3, and 421-4 is less than a predetermined threshold, for example, 0.5. Therefore, the second detection box 422-2 can be identified as an independent detection box.

[0104] According to some embodiments, the third case S504-3 can be a situation where the first case S504-1 and the second case S504-2 occur simultaneously.

[0105] In the example, the third case S504-2 can correspond to the combination as follows. Figure 4C The example shown illustrates this. By calculating the Intersection over Union (IOU) between each of the four first detection boxes 431-1, 431-2, 431-3, and 431-4 and each of the four second detection boxes 432-1, 432-3, 432-4, and 432-5, it can be determined that the overlap between the first detection box 431-2 and each of the four second detection boxes 432-1, 432-3, 432-4, and 432-5 is less than a predetermined threshold, for example, 0.5. Simultaneously, it can be determined that the overlap between the second detection box 432-5 and each of the four first detection boxes 431-1, 431-2, 431-3, and 431-4 is also less than a predetermined threshold, for example, 0.5. Therefore, the first detection box 431-2 and the second detection box 432-5 can be determined as independent detection boxes.

[0106] According to some embodiments, the fourth case S504-4 may include: in response to the fact that the overlap between a first detection box in the at least one first detection box and a second detection box in the at least one second detection box is greater than a predetermined threshold, and the target category indicated by the first detection box is different from the target category indicated by the second detection box, both the first detection box and the second detection box are determined as independent detection boxes.

[0107] The fourth scenario, S504-4, can involve the following special case: when two different targets are too close in position, the IOU value of their corresponding two detection boxes may also exceed a predetermined threshold. If the relationship between these two detection boxes is determined solely by their degree of overlap, an error will occur. Therefore, the relationship can be further determined by the target category indicated by these two detection boxes.

[0108] In this way, when there are many targets in the image and / or the targets are close in location, potential missed detections can be identified more accurately.

[0109] The following will combine Figure 6 An example describing the fourth case S504-4.

[0110] Figure 6 A schematic diagram illustrating an example scenario of the independent detection frame determination step according to an embodiment of this disclosure is shown.

[0111] like Figure 6 As shown, a first detection box 610 is obtained by performing a first object detection on image 600, and a second detection box 620 is obtained by performing a second object detection on image 600. Assuming that in image 600, both people and chairs are expected detection targets, the target category indicated by the first detection box 610 is "people", and the target category indicated by the second detection box 620 is "chairs".

[0112] When the first detection box 610 and the second detection box 620 are obtained, it can be determined whether there are independent detection boxes among these detection boxes. To do this, the degree of overlap between the first detection box 610 and the second detection box 620 can be calculated, for example, by calculating the Intersection over Union (IOU).

[0113] Because the person and the chair are very close together, the Intersection over Union (IOU) between the first detection box 610 and the second detection box 620 may be greater than, for example, 0.5. In this case, since the target category indicated by the first detection box 610 is different from the target category indicated by the second detection box 620, it can be determined that the first detection box 610 and the second detection box 620 are both independent detection boxes.

[0114] In other words, even if the Intersection over Union (IOU) value of two detection boxes is greater than a predetermined threshold, it is possible that these two detection boxes do not contain the same target. Therefore, a more accurate determination can be made by further examining the target category indicated by these two detection boxes.

[0115] The above combination Figures 4A to 4C , Figure 5 and Figure 6 Examples of independent detection box determination steps according to embodiments of this disclosure are described in detail. Independent detection boxes can represent targets that were missed, while for correctly detected targets, i.e., pairs of detection boxes with corresponding relationships, more accurate pseudo-labels can be generated by fusing their location information (furthermore, their confidence in the target category).

[0116] According to some embodiments, the pseudo-label generation method of this disclosure may further include the following steps.

[0117] First, we can identify paired detection boxes that correspond to each other among the other detection boxes. A paired detection box is defined as one in which the two detection boxes overlap by a predetermined threshold and indicate the same target category.

[0118] Then, the position coordinates of the paired detection boxes can be normalized to obtain normalized detection boxes. For example, normalization can include averaging. In addition to directly averaging the position coordinates of the detection boxes, a weighted average based on confidence levels can be performed to obtain more accurate results.

[0119] Then, the normalized detection boxes and independent detection boxes can be identified as pseudo-labels for the unlabeled image data.

[0120] Here, combined with, for example Figure 4A The example shown illustrates the above process. By calculating the Intersection over Union (IOU) between detection boxes, the first detection box 411-2 can be determined as an independent detection box. Accordingly, the corresponding pairs of detection boxes in the remaining detection boxes can be determined as follows: detection boxes 411-1 and 412-1, detection boxes 411-3 and 412-3, and detection boxes 411-4 and 412-4. This is because the overlap between the two detection boxes in these pairs is greater than a predetermined threshold (e.g., 0.5), and the two detection boxes indicate the same target category.

[0121] Therefore, the position coordinates of detection boxes 411-1 and 412-1 can be normalized. Similarly, the position coordinates of detection boxes 411-3 and 412-3 can be normalized, and the position coordinates of detection boxes 411-4 and 412-4 can also be normalized. This yields three normalized detection boxes. Accordingly, these three normalized detection boxes, along with detection box 411-2 (which is an independent detection box), can be used as pseudo-labels for the image.

[0122] In this way, during the process of generating pseudo-labels through object detection, more accurate pseudo-labels can be generated by further fusing correctly detected bounding boxes.

[0123] In the following text, we will combine Figure 7 and Figure 8 This invention describes a method for training an object detection model using pseudo-labels generated according to embodiments of the present disclosure.

[0124] Figure 7 A flowchart of a target detection model training method 700 according to an embodiment of the present disclosure is shown.

[0125] According to embodiments of this disclosure, the object detection model training method 700 is based on a semi-supervised training method, that is, using a combination of teacher and student models to train the model. Unlike traditional semi-supervised methods, the method according to embodiments of this disclosure improves the accuracy of the teacher model's predictions on training data by using a combination of multiple teacher and student models, thereby enhancing the overall training effect.

[0126] like Figure 7 As shown, the object detection model training method 700 may include steps S702, S704 and S706.

[0127] In step S702, image training data with pseudo-labels is obtained.

[0128] Here, the pseudo-tags are generated according to the pseudo-tag generation method of this disclosure (e.g., in conjunction with...). Figure 3 The pseudo-label generation method 300 is used to generate the labels. As previously described, the pseudo-label generation method includes obtaining at least one first detection box by performing a first object detection on the unlabeled image data, and obtaining at least one second detection box by performing a second object detection on the unlabeled image data. Here, the first object detection is performed via a pre-trained first teacher model, and the second object detection is performed via a pre-trained second teacher model.

[0129] In the example, the first teacher model can be the model that is expected to be obtained through training, while the second teacher model can cooperate with the first teacher model during training to generate more accurate pseudo-labels. For example, the first and second teacher models can have different model backbone structures. The first teacher model can be, for example, a small model with the desired fast object detection speed, while the second teacher model can be a large model with a complex structure to help generate more accurate pseudo-labels together with the first teacher model.

[0130] Since the pseudo-label generation method according to the present disclosure is used in step S702, it can ensure that the occurrence of target missed detection is reduced or avoided, thereby improving the quality of pseudo-labels to make fuller use of unlabeled training data, thereby improving the overall training effect.

[0131] In step S704, based on the first image training data with pseudo-labels and the second image training data with artificial labels, a first student model corresponding to the first teacher model is trained to obtain the first student model update parameters, and a second student model corresponding to the second teacher model is trained to obtain the second student model update parameters.

[0132] In step S706, the first teacher model is updated based on the first student model update parameters and the first current model parameters of the first teacher model, and the second teacher model is updated based on the second student model update parameters and the second current model parameters of the second teacher model.

[0133] In the example, the update process in step S706 can be performed using the EMA (Exponential Moving Average) strategy. Executing updates using the EMA strategy achieves a more gradual update effect, reducing the possibility of drastic parameter changes.

[0134] For example, taking multi-round training as an example, the first and second teacher models can be updated according to the following formula 1:

[0135] θ t_new = (1-α)*θ t_old +α*θ s (Formula 1)

[0136] Where, θ t_old This represents the model parameters of the teacher model after the previous training round, which is also the current model parameters of the teacher model in this training round (i.e., the first current model parameters of the first teacher model and the second current model parameters of the second teacher model in step S706); θ s This represents the updated model parameters of the student model after this round of training (i.e., the updated parameters of the first student model and the updated parameters of the second student model in step S704); θ t_new This represents the updated model parameters of the teacher model after this round of training (i.e., the model parameters obtained after the update operation in step S706); α represents a preset weight parameter, for example, the value can be between 0.01 and 0.05.

[0137] In the example, training can be completed when a preset number of rounds or a predetermined metric has been reached.

[0138] According to some embodiments, in step S702, the pre-trained first teacher model may have first initial model parameters, and the pre-trained second teacher model may have second initial model parameters. The first initial model parameters may be obtained by training the first student model based on second image training data with artificial labels, and the second initial model parameters may be obtained by training the second student model based on the second image training data with artificial labels.

[0139] In the example, pre-training of the first student model and the second student model can be performed before the first round of training to obtain the first initial model parameters of the first student model and the second initial model parameters of the second student model. The first initial model parameters of the first student model can be directly assigned to the first teacher model, and the second initial model parameters of the second student model can be directly assigned to the second teacher model.

[0140] In this way, initial model parameters can be assigned to the first teacher model and the second teacher model, so that initial pseudo-labels for unlabeled training data can be obtained through the first teacher model and the second teacher model.

[0141] In the target detection model training method according to embodiments of this disclosure, the quality of training data can be improved by employing a pseudo-label generation method that can reduce or avoid missed detections, thereby enhancing the overall training effect. With each iteration of training, the model parameters of the first teacher model and the second teacher model are continuously optimized, thereby continuously optimizing the pseudo-labels generated by the first teacher model and the second teacher model. This allows the overall training process to be continuously strengthened in a virtuous cycle, ultimately improving the overall training effect.

[0142] Figure 8 A schematic diagram of a target detection model training method according to an embodiment of the present disclosure is shown.

[0143] The total training data used for training can consist of labeled training data 820 (i.e., with human labels) and unlabeled training data 830.

[0144] For the unlabeled training data 830, such as Figure 8 As shown, the pseudo-tag generation method according to the embodiments of this disclosure (such as in combination with the first teacher model 801 and the second teacher model 811) can be used as an aid. Figure 3 The pseudo-label generation method 300 is used to generate pseudo-labels. That is, the first object detection in the method can be performed via the first teacher model 801, and the second object detection in the method can be performed via the second teacher model 811. Thus, pseudo-labels can be generated and provided to the unlabeled training data 830.

[0145] In the example, the initial model parameters of the first teacher model 801 can be obtained by training the first student model 802 using labeled training data 820. The initial model parameters of the second teacher model 811 can be obtained by training the second student model 812 using labeled training data 820.

[0146] like Figure 8 As shown, unlabeled training data 830 with pseudo-labels and labeled training data 820 can be used together as the entire training data and input into the first student model 802 and the second student model 812 for training, respectively. Thus, the first student model 802 can acquire a first update parameter through training, and the second student model 812 can acquire a second update parameter through training.

[0147] like Figure 8As shown, the first teacher model 801 can be updated using a first update parameter, and the second teacher model 811 can be updated using a second update parameter. Specifically, the first teacher model 801 can be updated based on the first update parameter and the current model parameters of the first teacher model 801, and the second teacher model 811 can be updated based on the second update parameter and the current model parameters of the second teacher model 811. In the example, this update process can be performed based on an update strategy such as EMA.

[0148] Therefore, with each round of training iteration, the model parameters of the first teacher model 801 and the second teacher model 811 will be continuously optimized, thereby continuously optimizing the pseudo-labels generated by the first teacher model and the second teacher model. This will continuously strengthen the overall training process in a virtuous cycle and ultimately improve the overall training effect.

[0149] According to another aspect of this disclosure, a pseudo-label generation apparatus is also provided.

[0150] Figure 9 A structural block diagram of a pseudo-label generation apparatus 900 according to an embodiment of the present disclosure is shown.

[0151] like Figure 9 As shown, the pseudo-label generation device 900 includes a target detection module 902, a first determination module 904, and a first processing module 906.

[0152] The target detection module 902 is configured to obtain at least one first detection box by performing a first target detection on unlabeled image data, and to obtain at least one second detection box by performing a second target detection on the unlabeled image data.

[0153] The first determining module 904 is configured to determine an independent detection box in a detection box that includes the at least one first detection box and the at least one second detection box, wherein the independent detection box does not correspond to any other detection boxes in the detection box that includes the at least one first detection box and the at least one second detection box.

[0154] The first processing module 906 is configured to identify independent detection boxes as pseudo-labels for unlabeled image data.

[0155] The operations performed by the target detection module 902, the first determination module 904, and the first processing module 906 of the fake tag generation device 900 can be combined with, for example... Figure 3 The steps S302, S304 and S306 of the pseudo-tag generation method 300 correspond to each other, so the details of each aspect will not be repeated here.

[0156] According to some embodiments, the fake tag generation device 900 and the first determining module 904 may further include further sub-modules, which will combine Figure 10 Provide a detailed description.

[0157] Figure 10 A structural block diagram of a pseudo-label generation apparatus 1000 according to another embodiment of the present disclosure is shown.

[0158] like Figure 10 As shown, the pseudo-label generation device 1000 includes a target detection module 1020, a first determination module 1040, and a first processing module 1060.

[0159] According to some embodiments, the target detection module 1020, the first determination module 1040, and the first processing module 1060 may be combined with the above. Figure 9 The target detection module 902, the first determination module 904, and the first processing module 906 correspond to each other.

[0160] According to some embodiments, the first determining module 1040 may include: a calculation module 1042 configured to calculate the degree of overlap between each first detection box in the at least one first detection box and each second detection box in the at least one second detection box; and a judgment module 1044 configured to determine an independent detection box based on the degree of overlap.

[0161] According to some embodiments, the judgment module 1044 may include: a first sub-judgment unit 1044-1, configured to determine the first detection box as an independent detection box in response to the fact that the degree of overlap between a first detection box in the at least one first detection box and each second detection box in the at least one second detection box is less than or equal to a predetermined threshold.

[0162] According to some embodiments, the judgment module 1044 may include: a second sub-judgment unit 1044-2, configured to determine the second detection frame as an independent detection frame in response to the fact that the degree of overlap between a second detection frame in the at least one second detection frame and each first detection frame in the at least one first detection frame is less than or equal to a predetermined threshold.

[0163] According to some embodiments, the determination module 1044 may include: a third sub-determination unit 1044-3, configured to determine the first detection box and the second detection box as independent detection boxes in response to the fact that the degree of overlap between a first detection box in the at least one first detection box and a second detection box in the at least one second detection box is greater than a predetermined threshold, and the target category indicated by the first detection box is different from the target category indicated by the second detection box.

[0164] According to some embodiments, the pseudo-label generation device 1000 may further include: a second determining module 1070, configured to determine pairs of detection boxes that have a corresponding relationship in the remaining detection boxes, wherein the overlap between the two detection boxes in the pairs of detection boxes is greater than a predetermined threshold and the target categories indicated by the two detection boxes are the same; a normalization module 1080, configured to normalize the position coordinates of the pairs of detection boxes to obtain normalized detection boxes; and a second processing module 1090, configured to determine the normalized detection boxes and independent detection boxes as pseudo-labels for the unlabeled image data.

[0165] According to another aspect of this disclosure, a target detection model training apparatus is also provided.

[0166] Figure 11 A structural block diagram of a target detection model training apparatus 1100 according to an embodiment of the present disclosure is shown.

[0167] like Figure 11 As shown, the target detection model training device 1100 includes a training data acquisition module 1120, a student model training module 1140, and a teacher model update module 1160.

[0168] Training data acquisition module 1120 is configured to acquire first image training data with pseudo-labels, wherein the pseudo-labels are generated according to the pseudo-label generation apparatus of embodiments of the present disclosure (e.g., as in combination with...). Figure 9 The pseudo-label generation device 900, such as Figure 10 The pseudo-label generation device 1000 generates the pseudo-label, wherein a first target detection is performed via a pre-trained first teacher model, and a second target detection is performed via a pre-trained second teacher model.

[0169] The student model training module 1140 is configured to train a first student model corresponding to a first teacher model to obtain the first student model update parameters based on first image training data with pseudo-labels and second image training data with artificial labels, and to train a second student model corresponding to a second teacher model to obtain the second student model update parameters.

[0170] The teacher model update module 1160 is configured to update the first teacher model based on the first student model update parameters and the first current model parameters of the first teacher model, and to update the second teacher model based on the second student model update parameters and the second current model parameters of the second teacher model.

[0171] According to some embodiments, the pre-trained first teacher model can have first initial model parameters, and the pre-trained second teacher model can have second initial model parameters. The first initial model parameters can be obtained by training the first student model based on second image training data with artificial labels, and the second initial model parameters can be obtained by training the second student model based on second image training data with artificial labels.

[0172] The operations performed by the training data acquisition module 1120, the student model training module 1140, and the teacher model update module 1160 of the object detection model training device 1100 can be combined with, for example... Figure 7 Steps S702, S704, and S706 of the target detection model training method 700 correspond to each other, so the details of each aspect will not be repeated here.

[0173] According to another aspect of this disclosure, an electronic device is also provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the method as described in the embodiments of this disclosure.

[0174] According to another aspect of this disclosure, a non-transitory computer-readable storage medium storing computer instructions is also provided, wherein the computer instructions are used to cause a computer to perform the methods described in the embodiments of this disclosure.

[0175] According to another aspect of this disclosure, a computer program product is also provided, including a computer program, wherein the computer program, when executed by a processor, implements the method described in the embodiments of this disclosure.

[0176] refer to Figure 12 The present invention describes a structural block diagram of an electronic device 1200 that can serve as a server or client of the present disclosure, which is an example of a hardware device that can be applied to various aspects of the present disclosure. The electronic device is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0177] like Figure 12As shown, the electronic device 1200 includes a computing unit 1201, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 1202 or a computer program loaded from a storage unit 1208 into a random access memory (RAM) 1203. The RAM 1203 may also store various programs and data required for the operation of the electronic device 1200. The computing unit 1201, ROM 1202, and RAM 1203 are interconnected via a bus 1204. An input / output (I / O) interface 1205 is also connected to the bus 1204.

[0178] Multiple components in electronic device 1200 are connected to I / O interface 1205, including: input unit 1206, output unit 1207, storage unit 1208, and communication unit 1209. Input unit 1206 can be any type of device capable of inputting information to electronic device 1200. Input unit 1206 can receive input digital or character information and generate key signal inputs related to user settings and / or function control of the electronic device, and may include, but is not limited to, a mouse, keyboard, touchscreen, trackpad, trackball, joystick, microphone, and / or remote control. Output unit 1207 can be any type of device capable of presenting information, and may include, but is not limited to, a monitor, speaker, video / audio output terminal, vibrator, and / or printer. Storage unit 1208 may include, but is not limited to, a hard disk and an optical disk. The communication unit 1209 allows the electronic device 1200 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks, and may include, but is not limited to, modems, network cards, infrared communication devices, wireless communication transceivers and / or chipsets, such as Bluetooth™ devices, 802.11 devices, WiFi devices, WiMax devices, cellular communication devices and / or the like.

[0179] The computing unit 1201 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 1201 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 1201 performs the various methods and processes described above, such as the label generation method and the object detection model training method. For example, in some embodiments, the label generation method and the object detection model training method can be implemented as computer software programs tangibly contained in a machine-readable medium, such as storage unit 1208. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 1200 via ROM 1202 and / or communication unit 1209. When the computer program is loaded into RAM 1203 and executed by the computing unit 1201, one or more steps of the label generation method and the object detection model training method described above can be performed. Alternatively, in other embodiments, the computing unit 1201 may be configured in any other suitable manner (e.g., by means of firmware) to perform the label generation method and the object detection model training method.

[0180] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0181] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0182] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0183] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0184] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), the Internet, and blockchain networks.

[0185] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0186] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0187] The collection, storage, use, processing, transmission, provision, and disclosure of images involved in this technical solution comply with relevant laws and regulations and do not violate public order and good morals.

[0188] While embodiments or examples of this disclosure have been described with reference to the accompanying drawings, it should be understood that the methods, systems, and devices described above are merely exemplary embodiments or examples, and the scope of the invention is not limited by these embodiments or examples, but only by the granted claims and their equivalents. Various elements in the embodiments or examples may be omitted or replaced by their equivalents. Furthermore, the steps may be performed in a different order than that described in this disclosure. Further, various elements in the embodiments or examples may be combined in various ways. Importantly, as the technology evolves, many elements described herein can be replaced by equivalents that appear after this disclosure.

Claims

1. A method for training an object detection model, comprising: Obtain the first image training data with the first pseudo-label and the second pseudo-label, where: Obtaining the first pseudo-tag includes: At least one first detection box corresponding to at least one target is obtained by performing a first target detection on unlabeled first image training data, and at least one second detection box corresponding to at least one target is obtained by performing a second target detection on the unlabeled first image training data, wherein the first target detection is performed via a pre-trained first teacher model, and the second target detection is performed via a pre-trained second teacher model. Identify an independent detection box within a detection box that includes at least one first detection box and at least one second detection box, wherein the independent detection box does not contain the same target as any other detection box in the detection box that includes at least one first detection box and at least one second detection box; and The independent detection box in the detection box including the at least one first detection box and the at least one second detection box is determined as the first pseudo-label of the unlabeled first image training data, wherein the first pseudo-label indicates the target missed in the first target detection and the second target detection; Obtaining the second pseudo-tag includes: The second pseudo-label is determined based on the remaining detection boxes, wherein the second pseudo-label indicates the correctly detected target in the first target detection and the second target detection; Based on the first image training data and the second image training data with artificial labels, a first student model corresponding to the first teacher model is trained to obtain the first student model update parameters, and a second student model corresponding to the second teacher model is trained to obtain the second student model update parameters; and The first teacher model is updated based on the first student model update parameters and the first current model parameters of the first teacher model, and the second teacher model is updated based on the second student model update parameters and the second current model parameters of the second teacher model.

2. The method according to claim 1, wherein, The determination of an independent detection box within a detection box that includes the at least one first detection box and the at least one second detection box includes: Calculate the degree of overlap between each first detection frame in the at least one first detection frame and each second detection frame in the at least one second detection frame; and The independent detection frames are determined based on the degree of overlap.

3. The method according to claim 2, wherein, The independent detection frames are determined based on the degree of overlap as follows: In response to the fact that the overlap between a first detection box in the at least one first detection box and each second detection box in the at least one second detection box is less than or equal to a predetermined threshold, the first detection box is determined as the independent detection box.

4. The method according to claim 2 or 3, wherein, The independent detection frames are determined based on the degree of overlap as follows: In response to the fact that the degree of overlap between a second detection frame in the at least one second detection frame and each first detection frame in the at least one first detection frame is less than or equal to a predetermined threshold, the second detection frame is determined as the independent detection frame.

5. The method according to claim 2, wherein, The independent detection frames are determined based on the degree of overlap as follows: In response to a first detection box in the at least one first detection box and a second detection box in the at least one second detection box having an overlap greater than a predetermined threshold, and the target category indicated by the first detection box being different from the target category indicated by the second detection box, both the first detection box and the second detection box are determined as independent detection boxes.

6. The method according to claim 2, wherein determining the second pseudo-label based on the remaining detection boxes comprises: Identify pairs of detection boxes that have a corresponding relationship in the remaining detection boxes, wherein the overlap between the two detection boxes in the pair of detection boxes is greater than a predetermined threshold and the two detection boxes indicate the same target category; The position coordinates of the paired detection boxes are normalized to obtain normalized detection boxes; and The normalized detection box is determined as the second pseudo-label of the unlabeled first image training data.

7. The method according to claim 1, wherein, The pre-trained first teacher model has a first initial model parameter, and the pre-trained second teacher model has a second initial model parameter, wherein the first initial model parameter is obtained by training the first student model based on the second image training data with artificial labels, and the second initial model parameter is obtained by training the second student model based on the second image training data with artificial labels.

8. A target detection model training device, comprising: The training data acquisition module is configured to acquire first image training data with a first pseudo-label and a second pseudo-label, wherein: The first pseudo-tag is obtained by a first pseudo-tag generation device, wherein the first pseudo-tag generation device includes: The object detection module is configured to obtain at least one first detection box corresponding to at least one object by performing first object detection on unlabeled first image training data, and to obtain at least one second detection box corresponding to at least one object by performing second object detection on the unlabeled first image training data, wherein the first object detection is performed via a pre-trained first teacher model, and the second object detection is performed via a pre-trained second teacher model. A first determining module is configured to determine an independent detection box within a detection box comprising the at least one first detection box and the at least one second detection box, wherein the independent detection box does not contain the same target as the other detection boxes in the detection box comprising the at least one first detection box and the at least one second detection box; and A first processing module is configured to determine the independent detection box in the detection box including the at least one first detection box and the at least one second detection box as the first pseudo label of the unlabeled first image training data; The second pseudo-label is obtained by a second pseudo-label generation device, which is configured to determine the second pseudo-label based on the remaining detection frames, wherein the second pseudo-label indicates a correctly detected target in the first target detection and the second target detection; The student model training module is configured to train a first student model corresponding to the first teacher model to obtain the first student model update parameters of the first student model based on the first image training data with the first pseudo-label and the second pseudo-label and the second image training data with the artificial label, and to train a second student model corresponding to the second teacher model to obtain the second student model update parameters of the second student model. The teacher model update module is configured to update the first teacher model based on the first student model update parameters and the first current model parameters of the first teacher model, and to update the second teacher model based on the second student model update parameters and the second current model parameters of the second teacher model.

9. The apparatus according to claim 8, wherein, The first determining module includes: The calculation module is configured to calculate the degree of overlap between each first detection box in the at least one first detection box and each second detection box in the at least one second detection box; and The judgment module is configured to determine the independent detection frame based on the degree of overlap.

10. The apparatus according to claim 9, wherein, The judgment module includes: The first sub-judgment unit is configured to determine the first detection box as the independent detection box in response to the fact that the degree of overlap between a first detection box in the at least one first detection box and each second detection box in the at least one second detection box is less than or equal to a predetermined threshold.

11. The apparatus according to claim 9 or 10, wherein, The judgment module includes: The second sub-judgment unit is configured to determine the second detection frame as the independent detection frame in response to the fact that the degree of overlap between one of the second detection frames in the at least one second detection frame and each of the first detection frames in the at least one first detection frame is less than or equal to a predetermined threshold.

12. The apparatus according to claim 9, wherein, The judgment module includes: The third sub-judgment unit is configured to determine both the first detection box and the second detection box as independent detection boxes in response to the fact that the degree of overlap between a first detection box in the at least one first detection box and a second detection box in the at least one second detection box is greater than a predetermined threshold, and the target category indicated by the first detection box is different from the target category indicated by the second detection box.

13. The apparatus according to claim 9, wherein the second pseudo-tag generation apparatus comprises: The second determining module is configured to determine pairs of detection boxes that have a corresponding relationship in the remaining detection boxes, wherein the overlap between the two detection boxes in the pair of detection boxes is greater than a predetermined threshold and the two detection boxes indicate the same target category; A normalization module is configured to normalize the position coordinates of the paired detection boxes to obtain normalized detection boxes; and The second processing module is configured to determine the normalized detection box as the second pseudo-label of the unlabeled first image training data.

14. The apparatus according to claim 8, wherein, The pre-trained first teacher model has a first initial model parameter, and the pre-trained second teacher model has a second initial model parameter, wherein the first initial model parameter is obtained by training the first student model based on the second image training data with artificial labels, and the second initial model parameter is obtained by training the second student model based on the second image training data with artificial labels.

15. An electronic device comprising: At least one processor; as well as A memory that is communicatively connected to the at least one processor; The memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enable the at least one processor to perform the method according to any one of claims 1-7.

16. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-7.

17. A computer program product comprising a computer program, wherein, The computer program, when executed by a processor, implements the method according to any one of claims 1-7.

Images