Training Method and Device for Object Detection Model, Object Detection Method and Device

Abstract

The present disclosure provides a training method and apparatus for an object detection model, an object detection method and apparatus, relating to the field of artificial intelligence technologies, and particularly to technologies such as deep learning, image processing, and computer vision. The specific implementation solution is as follows: The training method includes: inputting a first image into a first model for object detection to obtain a first prediction result, and inputting a second image into a second model for object detection to obtain a second prediction result; performing an inverse transformation process corresponding to a first transformation process on the first prediction result to obtain a first detection result, and performing an inverse transformation process corresponding to a second transformation process on the second prediction result to obtain a second detection result; calculating a regression loss according to the positions of the detection frames in the first detection result and the positions of the detection frames in the second detection result; calculating a classification loss according to the second prediction result and the regression target of the second model; and training the second model according to the regression loss and the classification loss to obtain an object detection model.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technologies, and in particular to technologies such as deep learning, image processing, and computer vision. Background Art

[0002] With the development of artificial intelligence technologies, deep learning models have been widely applied. For example, deep learning models can be applied to multiple tasks in the field of computer vision, such as object recognition, face recognition, object detection, and semantic segmentation.

[0003] Among them, the task of object detection is to find all objects of interest in an image and determine the category and location of the objects. Since the training methods of existing deep learning models do not consider the relevance between classification tasks and regression tasks, the results of object detection by the trained deep learning models are not accurate. Summary of the Invention

[0004] The present disclosure provides a training method and device for an object detection model, an object detection method and device, an electronic device, a storage medium, and a computer program product.

[0005] According to one aspect of the present disclosure, a training method for an object detection model is provided, including: inputting a first image into a first model for object detection to obtain a first prediction result, and inputting a second image into a second model for object detection to obtain a second prediction result; the first image is obtained by performing a first transformation process on an unlabeled sample image, the second image is obtained by performing a second transformation process on the sample image, and the first transformation process and the second transformation process are different data augmentation processes; performing an inverse transformation process corresponding to the first transformation process on the first prediction result to obtain a first detection result, and performing an inverse transformation process corresponding to the second transformation process on the second prediction result to obtain a second detection result; calculating a regression loss according to the positions of the detection frames in the first detection result and the positions of the detection frames in the second detection result; calculating a classification loss according to the second prediction result and the regression target of the second model; the regression target is determined according to the class probability distribution in the first prediction result and the intersection over union between the detection frames in the first detection result and the detection frames in the second detection result; training the second model according to the regression loss and the classification loss to obtain an object detection model.

[0006] According to another aspect of the present disclosure, an object detection method is provided, including: acquiring an image to be processed; inputting the image to be processed into an object detection model for object detection to obtain an object detection result; wherein, the object detection model is trained based on the above training method.

[0007] According to another aspect of the present disclosure, there is provided a training device for a target detection model, including: a prediction module configured to input a first image into a first model for target detection to obtain a first prediction result, and input a second image into a second model for target detection to obtain a second prediction result; the first image is obtained by performing a first transformation process on an unlabeled sample image, the second image is obtained by performing a second transformation process on the sample image, and the first transformation process and the second transformation process are different data augmentation processes; a processing module configured to perform an inverse transformation process corresponding to the first transformation process on the first prediction result to obtain a first detection result, and perform an inverse transformation process corresponding to the second transformation process on the second prediction result to obtain a second detection result; a first calculation module configured to calculate a regression loss according to the positions of the detection frames in the first detection result and the positions of the detection frames in the second detection result; a second calculation module configured to calculate a classification loss according to the second prediction result and the regression target of the second model; the regression target is determined according to the class probability distribution in the first prediction result and the intersection over union between the detection frames in the first detection result and the detection frames in the second detection result; a training module configured to train the second model according to the regression loss and the classification loss to obtain a target detection model.

[0008] According to another aspect of the present disclosure, there is provided a target detection device, including: an acquisition module configured to acquire an image to be processed; a detection module configured to input the image to be processed into a target detection model for target detection to obtain a target detection result; wherein, the target detection model is trained based on the above training device.

[0009] According to another aspect of the present disclosure, there is provided an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein, the memory stores instructions executable by the at least one processor, and when the instructions are executed by the at least one processor, the at least one processor is enabled to execute the above training method or the above target detection method.

[0010] According to another aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause the computer to execute the above training method or the above target detection method.

[0011] According to another aspect of the present disclosure, there is provided a computer program product, including a computer program, where the computer program, when executed by a processor, implements the above training method or the above target detection method.

[0012] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it used to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood from the following description. Description of the Drawings

[0013] The drawings are used to better understand the present solution and do not constitute a limitation to the present disclosure. Among them:

[0014] Figure 1 is a flowchart of a method for training an object detection model according to an embodiment of the present disclosure;

[0015] Figure 2 is a schematic diagram of the training process of an object detection model according to an embodiment of the present disclosure;

[0016] Figure 3 is a flowchart of step S105 according to an embodiment of the present disclosure;

[0017] Figure 4 is a schematic diagram of an apparatus for training an object detection model according to an embodiment of the present disclosure;

[0018] Figure 5 is a flowchart of an object detection method according to an embodiment of the present disclosure;

[0019] Figure 6 is a schematic diagram of an object detection apparatus according to an embodiment of the present disclosure;

[0020] Figure 7 is a block diagram of an electronic device for implementing the method of the embodiment of the present disclosure. Detailed Embodiments

[0021] The following describes exemplary embodiments of the present disclosure with reference to the accompanying drawings. Various details of the embodiments of the present disclosure are included to facilitate understanding, and they should be considered merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, descriptions of well-known functions and structures are omitted for clarity and conciseness in the following description.

[0022] It should be noted that the terms "first", "second", etc. in the description, claims and the above-mentioned drawings of the present disclosure are used to distinguish similar objects, and do not necessarily describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present disclosure described here can be implemented in an order different from those illustrated or described here. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device comprising a series of steps or units does not necessarily limit to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0023] According to an embodiment of the present disclosure, an embodiment of a method for training a target detection model is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. And although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order from that here.

[0024] Figure 1 is a flowchart of a method for training a target detection model according to an embodiment of the present disclosure, as Figure 1 shown, the training method includes the following steps S101 to step S105:

[0025] Step S101: Input the first image into the first model for target detection to obtain a first prediction result, and input the second image into the second model for target detection to obtain a second prediction result. Wherein, the first image is obtained by performing a first transformation process on an unlabeled sample image, the second image is obtained by performing a second transformation process on the sample image, and the first transformation process and the second transformation process are different data augmentation processes.

[0026] Wherein, the sample image can be an image collected by a vehicle-mounted camera, and the target included in the image can be an obstacle. The sample image can also be an image collected by a medical device, and the target included in the image can be a certain part. The sample image can also be other types of images, which are not specifically limited.

[0027] The first transformation process is usually a relatively small-degree transformation process on the unlabeled sample image, and can also be called weak data augmentation processing. Specifically, it can include transformation processes such as scaling, translation, and changing the color space. The second transformation process is usually a relatively large-degree transformation process on the unlabeled sample image, and can also be called strong data augmentation processing. Specifically, it can include transformation processes such as rotation, flipping, shearing, adding noise, and mixing different data.

[0028] Input the first image into the first model for object detection to obtain a first prediction result, which specifically includes a detection box for describing the position of the object in the first image and the class probability distribution of the object. Input the second image into the second model for object detection to obtain a second prediction result, which specifically includes a detection box for describing the position of the object in the second image and the class probability distribution of the object. Among them, the detection box can also be called the object box, and its shape is usually rectangular. It can be represented by the coordinates (x1, y1) of the upper left corner and the coordinates (x2, y2) of the lower right corner of the detection box, or by the center coordinates (xc, yc) and the width and height (w, h) of the detection box.

[0029] In some scenarios, the first model can also be called the teacher model, and the second model can also be called the student model. In specific implementation, both the first model and the second model can adopt the DETR (Detection Transformer) model based on the end-to-end framework.

[0030] Step S102: Perform an inverse transformation corresponding to the first transformation process on the first prediction result to obtain a first detection result, and perform an inverse transformation corresponding to the second transformation process on the second prediction result to obtain a second detection result.

[0031] By performing corresponding inverse transformation processes on the first prediction result and the second prediction result, the first prediction result output by the first model can be marked at the correct position of the second prediction result output by the second model.

[0032] Step S103: Calculate the regression loss according to the position of the detection box in the first detection result and the position of the detection box in the second detection result. In some scenarios, the regression loss can also be called the consistency loss. In specific implementation, the position of the detection box in the first detection result and the position of the detection box in the second detection result can be substituted into the regression loss function to calculate the regression loss. Among them, the regression loss function can specifically be the mean square error function, the mean absolute error function, etc.

[0033] Step S104: Calculate the classification loss according to the second prediction result and the regression target of the second model. Among them, the regression target is determined according to the class probability distribution in the first prediction result and the intersection over union between the detection box in the first detection result and the detection box in the second detection result.

[0034] Among them, the intersection over union (IOU) is used to characterize the overlapping degree between the detection boxes in the first detection result and the detection boxes in the second detection result. The higher the IOU of the two detection boxes, the greater the probability that the two detection boxes correspond to the same target.

[0035] Specifically, the classification loss is calculated according to the class probability distribution of the target in the second prediction result and the regression target of the second model. Specifically, the two can be substituted into the classification loss function to calculate the classification loss. Among them, the classification loss function can specifically be a cross-entropy loss function, a negative log-likelihood loss function, etc.

[0036] In a specific example, the following classification loss function is used to calculate the classification loss L cls .

[0037] L cls = -q × log(p) + α × (1 - q) γ log(1 - p))

[0038] Among them, q is the regression target of the second model, p is the class probability distribution of the target in the second prediction result, and α and γ are hyperparameters. Assume that the class probability distribution in the first prediction result is cls_distribution, and the intersection over union between the detection boxes in the first detection result and the detection boxes in the second detection result is iou_score. q is specifically determined according to iou_score and cls_distribution.

[0039] Step S105: Train the second model according to the regression loss and the classification loss to obtain an object detection model. Specifically, optimize and update the parameters of the second model according to the regression loss and the classification loss to obtain a trained object detection model.

[0040] During the above model training process, since the sample images are unlabeled, the training of the second model is an unsupervised training method. However, during the actual training process of the model, labeled sample images can also be used to train the second model in a supervised manner. Combining the unsupervised method and the supervised method to jointly train the second model can obtain a more accurate object detection model. In a specific example, a part of the data, such as 5% of the data, can be extracted from the CoCo dataset (a large-scale dataset) as labeled data, and the remaining data as unlabeled data. Use part of the labeled data and part of the unlabeled data to perform semi-supervised training on the model to obtain an object detection model.

[0041] In the embodiments of the present disclosure, during the model training process, the regression target of the second model is determined according to the class probability distribution in the first prediction result and the intersection over union (IoU) between the detection box in the first detection result and the detection box in the second detection result, and the second model is trained using the classification loss determined according to the regression target of the second model. This can not only transmit more information predicted by the first model to the second model, but also improve the consistency between the classification task and the regression task, that is, a classification result with a high confidence also has a high-quality localization accuracy. Furthermore, the accuracy of detecting a target by the target detection model trained using this training method can be improved.

[0042] In an alternative embodiment, the regression target is determined according to the product of the class probability distribution in the first prediction result and the IoU. In a specific implementation, the product can be directly used as the regression target of the second model. Specifically, in the above example, q = iou_score * cls_distribution. Alternatively, the product can be modified, and the modified value can be used as the regression target of the second model. Specifically, the modification can be performed by multiplying the product by a certain coefficient, etc., where the coefficient can be set according to the actual situation.

[0043] In an alternative embodiment, the first image is obtained by performing a first transformation process on the sample image using a first transformation matrix. The above step S102 specifically includes: performing an inverse transformation process corresponding to the first transformation process on the first prediction result using the inverse matrix of the first transformation matrix to obtain a first detection result.

[0044] In a specific implementation, different first transformation processes correspond to different first transformation matrices. Applying different first transformation matrices to the sample image can achieve different first transformation processes. Correspondingly, applying the inverse matrix of the first transformation matrix to the first prediction result can obtain a first detection result corresponding to the sample image.

[0045] In an alternative embodiment, the second image is obtained by performing a second transformation process on the sample image using a second transformation matrix. The above step S102 specifically includes: performing an inverse transformation process corresponding to the second transformation process on the second prediction result using the inverse matrix of the second transformation matrix to obtain a second detection result.

[0046] In a specific implementation, different second transformation processes correspond to different second transformation matrices. Applying different second transformation matrices to the sample image can achieve different second transformation processes. Correspondingly, applying the inverse matrix of the second transformation matrix to the second prediction result can obtain a second detection result corresponding to the sample image.

[0047] In an alternative embodiment, before performing the inverse transformation process on the first prediction result, it includes: filtering the first prediction result based on confidence by means of bipartite graph matching. Specifically, the first prediction results with confidence not meeting the preset conditions are filtered out, and the first prediction results with confidence meeting the preset conditions are retained, which can improve the accuracy of the first prediction result.

[0048] The following gives a specific example for the above training process. As Figure 2 shown, the unlabeled sample image is subjected to weak data augmentation using the first transformation matrix, and then the obtained first image is input into the teacher model for object detection, and the first prediction result is output. After confidence filtering, an accurate first prediction result is obtained. The first prediction result is processed using the inverse matrix of the first transformation matrix to obtain the first detection result. The unlabeled sample image is subjected to strong data augmentation using the second transformation matrix, and then the obtained second image is input into the student model for object detection, and the second prediction result is output. The second prediction result is processed using the inverse matrix of the second transformation matrix to obtain the second detection result. The regression loss is calculated based on the first detection result and the second detection result, and the intersection over union between the detection boxes in the first detection result and the detection boxes in the second detection result is calculated, and the regression target of the student model is determined according to the class probability distribution in the first prediction result and the intersection over union. The classification loss is calculated based on the second prediction result and the regression target. Finally, backpropagation calculation is performed according to the regression loss and the classification loss, and the parameters of the student model are optimized according to the gradient values obtained from the backpropagation calculation until the convergence condition is met.

[0049] In an alternative embodiment, the trained second model is directly used as the object detection model.

[0050] In another alternative embodiment, as Figure 3 shown, the above step S105 specifically includes:

[0051] Step S301: Train the second model according to the regression loss and the classification loss.

[0052] Step S302: Transfer the parameters of the trained second model to the first model to obtain the object detection model. Specifically, the parameters of the second model can be transferred to the first model by means of exponential moving average.

[0053] In this embodiment, the parameters of the trained second model are transferred to the first model, and the first model after parameter transfer is used as the object detection model.

[0054] According to an embodiment of the present disclosure, there is also provided an embodiment of a training device for an object detection model, wherein, Figure 4Schematic diagram of a training device for an object detection model according to an embodiment of the present disclosure. The training device includes a prediction module 401, a processing module 402, a first calculation module 403, a second calculation module 404, and a training module 405. The prediction module 401 is configured to input a first image into a first model for object detection to obtain a first prediction result, and input a second image into a second model for object detection to obtain a second prediction result. Wherein, the first image is obtained by performing a first transformation process on an unlabeled sample image, the second image is obtained by performing a second transformation process on the sample image, and the first transformation process and the second transformation process are different data augmentation processes. The processing module 402 is configured to perform an inverse transformation process corresponding to the first transformation process on the first prediction result to obtain a first detection result, and perform an inverse transformation process corresponding to the second transformation process on the second prediction result to obtain a second detection result. The first calculation module 403 is configured to calculate a regression loss according to the positions of the detection frames in the first detection result and the positions of the detection frames in the second detection result. The second calculation module 404 is configured to calculate a classification loss according to the second prediction result and the regression target of the second model. Wherein, the regression target is determined according to the class probability distribution in the first prediction result and the intersection over union between the detection frames in the first detection result and the detection frames in the second detection result. The training module 405 is configured to train the second model according to the regression loss and the classification loss to obtain an object detection model.

[0055] It should be noted that the above prediction module 401, processing module 402, first calculation module 403, second calculation module 404, and training module 405 correspond to steps S101 to S105 in the above embodiment. The examples and application scenarios implemented by the five modules and the corresponding steps are the same, but are not limited to the content disclosed in the above embodiment.

[0056] In an alternative embodiment, the regression target is determined according to the product of the class probability distribution in the first prediction result and the intersection over union.

[0057] In an alternative embodiment, the first image is obtained by performing a first transformation process on the sample image using a first transformation matrix; the processing module is specifically configured to perform an inverse transformation process corresponding to the first transformation process on the first prediction result using the inverse matrix of the first transformation matrix to obtain a first detection result.

[0058] In an alternative embodiment, the second image is obtained by performing a second transformation process on the sample image using a second transformation matrix; specifically, the processing module is configured to perform an inverse transformation process corresponding to the second transformation process on the second prediction result using the inverse matrix of the second transformation matrix to obtain a second detection result.

[0059] In an alternative embodiment, the training device further includes a filtering module configured to filter the first prediction result based on confidence in a bipartite graph matching manner.

[0060] In an alternative embodiment, the training module is specifically configured to train the second model according to the regression loss and the classification loss, and transfer the parameters of the trained second model to the first model to obtain a target detection model.

[0061] According to an embodiment of the present disclosure, an embodiment of a target detection method is also provided. It should be noted that the steps illustrated in the flowchart of the accompanying drawings may be executed in a computer system such as a set of computer-executable instructions, and although the logical order is illustrated in the flowchart, in some cases, the steps shown or described may be executed in a different order than here.

[0062] Figure 5 is a flowchart of a target detection method according to an embodiment of the present disclosure, as Figure 5 shown, the target detection method includes the following steps S501 to step S502:

[0063] Step S501, obtain an image to be processed.

[0064] Step S502, input the image to be processed into a target detection model for target detection to obtain a target detection result. Wherein, the target detection model is trained according to the training method of the above embodiment.

[0065] Wherein, the image to be processed may be an image collected by a vehicle-mounted camera, and the target included in the image may be an obstacle. The image to be processed may also be an image collected by a medical device, and the target included in the image may be a certain part. The image to be processed may also be other types of images, which are not specifically limited.

[0066] In the embodiment of the present disclosure, since the training method of the above embodiment can transfer more information predicted by the first model to the second model, and can also improve the consistency of the classification task and the regression task, therefore, using the target detection model trained by this training method to perform target detection on the image to be processed can obtain more accurate detection results, that is, a classification result with high confidence also has high-quality positioning accuracy.

[0067] According to an embodiment of the present disclosure, an embodiment of an object detection device is further provided, wherein Figure 6 is a schematic diagram of an object detection device according to an embodiment of the present disclosure. The object detection device includes an acquisition module 601 and a detection module 602. The acquisition module 601 is configured to acquire an image to be processed. The detection module 602 is configured to input the image to be processed into an object detection model for object detection to obtain an object detection result; wherein, the object detection model is trained by the training device according to the above embodiment.

[0068] It should be noted that the above acquisition module 601 and detection module 602 correspond to steps S501 to S502 in the above embodiment. The examples and application scenarios implemented by the two modules are the same as those of the corresponding steps, but are not limited to the content disclosed in the above embodiment.

[0069] In the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision, and disclosure of user personal information and other processes all comply with the provisions of relevant laws and regulations and do not violate public order and good customs.

[0070] According to an embodiment of the present disclosure, the present disclosure further provides an electronic device, a readable storage medium, and a computer program product.

[0071] Figure 7 FIG. shows a schematic block diagram of an exemplary electronic device 700 that can be used to implement the embodiments of the present disclosure. The electronic device is intended to represent various forms of digital computers, such as, for example, a laptop computer, a desktop computer, a workbench, a personal digital assistant, a server, a blade server, a mainframe computer, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as, for example, a personal digital processor, a cellular phone, a smart phone, a wearable device, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0072] As Figure 7 shown, the device 700 includes a computing unit 701, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 702 or a computer program loaded from a storage unit 708 into a random access memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the device 700 can also be stored. The computing unit 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0073] Multiple components in device 700 are connected to I / O interface 705, including: input unit 706, such as a keyboard, mouse, etc.; output unit 707, such as various types of displays, speakers, etc.; storage unit 708, such as a disk, optical disc, etc.; and communication unit 709, such as a network card, modem, wireless communication transceiver, etc. Communication unit 709 allows device 700 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0074] Computing unit 701 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of computing unit 701 include but are not limited to a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Computing unit 701 executes the various methods and processes described above, such as the training method or the object detection method of the object detection model. For example, in some embodiments, the training method or the object detection method of the object detection model can be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as storage unit 708. In some embodiments, part or all of the computer program can be loaded and / or installed onto device 700 via ROM 702 and / or communication unit 709. When the computer program is loaded into RAM 703 and executed by computing unit 701, one or more steps of the training method or the object detection method of the object detection model described above can be executed. Alternatively, in other embodiments, computing unit 701 can be configured to execute the training method or the object detection method of the object detection model in any other suitable way (e.g., by means of firmware).

[0075] The various embodiments of the systems and techniques described above in this article can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special or general-purpose programmable processor, and can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit the data and instructions to the storage system, the at least one input device, and the at least one output device.

[0076] The program code for implementing the methods of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing devices, such that when the program codes are executed by the processor or controller, the functions / operations specified in the flowchart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, executed partially on the machine and partially on a remote machine as an independent software package, or executed entirely on a remote machine or server.

[0077] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0078] In order to provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball) through which the user can provide input to the computer. Other kinds of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0079] The systems and techniques described herein can be implemented in a computing system including backend components (e.g., as a data server), or a computing system including middleware components (e.g., an application server), or a computing system including frontend components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described herein), or a computing system including any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected with each other by digital data communication in any form or medium (e.g., a communication network). Examples of communication networks include: local area network (LAN), wide area network (WAN), and the Internet.

[0080] A computer system can include a client and a server. The client and the server are generally far from each other and typically interact through a communication network. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, a server of a distributed system, or a server incorporating blockchain.

[0081] It should be understood that various forms of the processes shown above can be used, with steps reordered, added, or deleted. For example, the steps recited in this disclosure can be executed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this is not limited herein.

[0082] The above specific embodiments do not constitute a limitation on the protection scope of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure shall be included within the protection scope of this disclosure.

Claims

1. A training method for an object detection model, comprising: Inputting a first image into a first model for object detection to obtain a first prediction result, and inputting a second image into a second model for object detection to obtain a second prediction result; the first image is obtained by performing a first transformation process on an unlabeled sample image, the second image is obtained by performing a second transformation process on the sample image, the first transformation process and the second transformation process are different data augmentation processes. Specifically, the first transformation process is a transformation process with a relatively small degree on the unlabeled sample image, and the second transformation process is a transformation process with a relatively large degree on the unlabeled sample image; Performing an inverse transformation process corresponding to the first transformation process on the first prediction result to obtain a first detection result, and performing an inverse transformation process corresponding to the second transformation process on the second prediction result to obtain a second detection result; Calculating a regression loss according to the positions of the detection boxes in the first detection result and the positions of the detection boxes in the second detection result; Calculating a classification loss according to the second prediction result and the regression target of the second model; The regression target is determined according to the class probability distribution in the first prediction result and the intersection over union between the detection boxes in the first detection result and the detection boxes in the second detection result; Training the second model according to the regression loss and the classification loss to obtain an object detection model.

2. The training method according to claim 1, wherein, The regression target is determined according to the product of the class probability distribution in the first prediction result and the intersection over union.

3. The training method according to claim 1, wherein The first image is obtained by performing a first transformation process on the sample image using a first transformation matrix; The performing an inverse transformation process corresponding to the first transformation process on the first prediction result to obtain a first detection result includes: performing an inverse transformation process corresponding to the first transformation process on the first prediction result using the inverse matrix of the first transformation matrix to obtain a first detection result.

4. The training method according to claim 1, wherein, The second image is obtained by performing a second transformation process on the sample image using a second transformation matrix; The performing an inverse transformation process corresponding to the second transformation process on the second prediction result to obtain a second detection result includes: performing an inverse transformation process corresponding to the second transformation process on the second prediction result using the inverse matrix of the second transformation matrix to obtain a second detection result.

5. The training method according to claim 1, before performing the inverse transformation process on the first prediction result, includes: Filtering the first prediction result according to the confidence level based on the bipartite graph matching method.

6. The training method according to any one of claims 1-5, wherein, The training the second model according to the regression loss and the classification loss to obtain an object detection model includes: Training the second model according to the regression loss and the classification loss; Transferring the parameters of the trained second model to the first model to obtain an object detection model.

7. An object detection method, comprising: Obtaining an image to be processed; Inputting the image to be processed into an object detection model for object detection to obtain an object detection result; Wherein, the object detection model is trained according to the training method described in any one of claims 1-6.

8. A training device for an object detection model, comprising: A prediction module, configured to input a first image into a first model for object detection to obtain a first prediction result, and input a second image into a second model for object detection to obtain a second prediction result; the first image is obtained by performing a first transformation process on an unlabeled sample image, the second image is obtained by performing a second transformation process on the sample image, the first transformation process and the second transformation process are different data augmentation processes. Specifically, the first transformation process is a transformation process with a relatively small degree on the unlabeled sample image, and the second transformation process is a transformation process with a relatively large degree on the unlabeled sample image; A processing module, configured to perform an inverse transformation process corresponding to the first transformation process on the first prediction result to obtain a first detection result, and perform an inverse transformation process corresponding to the second transformation process on the second prediction result to obtain a second detection result; A first calculation module, configured to calculate a regression loss according to the positions of the detection frames in the first detection result and the positions of the detection frames in the second detection result; A second calculation module, configured to calculate a classification loss according to the second prediction result and the regression target of the second model; The regression target is determined according to the class probability distribution in the first prediction result and the intersection over union between the detection frames in the first detection result and the detection frames in the second detection result; A training module, configured to train the second model according to the regression loss and the classification loss to obtain an object detection model.

9. The training device according to claim 8, wherein, The regression target is determined according to the product of the class probability distribution in the first prediction result and the intersection over union.

10. The training device according to claim 8, wherein, The first image is obtained by performing a first transformation process on the sample image by using a first transformation matrix; Specifically, the processing module is configured to perform an inverse transformation process corresponding to the first transformation process on the first prediction result by using the inverse matrix of the first transformation matrix to obtain a first detection result.

11. The training device according to claim 8, wherein, The second image is obtained by performing a second transformation process on the sample image by using a second transformation matrix; Specifically, the processing module is configured to perform an inverse transformation process corresponding to the second transformation process on the second prediction result by using the inverse matrix of the second transformation matrix to obtain a second detection result.

12. The training device according to claim 8, further comprising a filtering module, configured to filter the first prediction result based on confidence by means of bipartite graph matching.

13. The training device according to any one of claims 8-12, wherein, Specifically, the training module is configured to train the second model according to the regression loss and the classification loss, and transfer the parameters of the trained second model to the first model to obtain an object detection model.

14. An object detection device, comprising: An acquisition module, configured to acquire an image to be processed; A detection module, configured to input the image to be processed into an object detection model for object detection to obtain an object detection result; Wherein, the object detection model is trained based on the training device according to any one of claims 8-13.

15. An electronic device, comprising: At least one processor; and a memory communicatively connected to the at least one processor; wherein, the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the training method according to any one of claims 1-6 or the object detection method according to claim 7.

16. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to execute the training method according to any one of claims 1-6 or the object detection method according to claim 7.

17. A computer program product, comprising a computer program which, when executed by a processor, implements the training method according to any one of claims 1-6 or the object detection method according to claim 7.

Images