Training method and device of target detection model, equipment and storage medium

Abstract

The application discloses a target detection model training method and device, equipment and a storage medium, and relates to the technical field of artificial intelligence. The method comprises the following steps: obtaining a difficult image set based on an image subset by a first-stage target detection model, wherein the difficult image refers to an image mis-detected by the first-stage target detection model; sampling a positive sample image set and a difficult image sample set from the image subset and the difficult image set; obtaining a fusion image sample set based on the positive sample image and the difficult image sample, wherein the fusion image sample comprises a target corresponding to the positive sample image and a non-target corresponding to the difficult image sample; and training the first-stage target detection model based on the positive sample image set, the difficult image sample set and the fusion image sample set. The application can be applied to the scenes of artificial intelligence, intelligent transportation and auxiliary driving, and can improve the detection success rate of the target detection model and the robustness to difficult image interference.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the field of artificial intelligence, and particularly relate to a training method and device of a target detection model, a device and a storage medium. BACKGROUND

[0002] With the development of artificial intelligence technology, its research and application in the field of face (i.e. face) recognition are also increasing.

[0003] The related technology trains a face detection model that can be used for face detection through face images with label information. However, as the application scenarios of face recognition and analysis gradually popularize, the input images in different application scenarios are increasingly complex, thereby causing the face detection model to have a mis-detection situation, such as detecting a non-face image as a face image, and the detection success rate of the face detection model is not high enough. SUMMARY

[0004] Embodiments of the present application provide a training method, device and storage medium of a target detection model, which can improve the detection success rate of the target detection model and the robustness to interference of difficult images (such as mis-detection images). The technical solution can include the following contents.

[0005] According to an aspect of an embodiment of the present application, a training method of a target detection model is provided, and the method comprises:

[0006] obtaining a training data set, wherein the training data set comprises a sub-set of images in at least one scene;

[0007] detecting each image in the training data set by a first-stage target detection model to obtain a difficult image set, wherein the first-stage target detection model is trained based on the training data set, and a difficult image in the difficult image set is an image that is detected by the first-stage target detection model as including a target but actually does not include the target;

[0008] sampling a positive sample image set in the at least one scene and a difficult image sample set in the difficult image set from the sub-set of images in the at least one scene and the difficult image set, wherein a positive sample image in the positive sample image set includes the target;

[0009] obtaining a fusion image sample set based on the positive sample image in the positive sample image set in the at least one scene and the difficult image sample in the difficult image sample set, wherein a fusion image sample in the fusion image sample set includes a target corresponding to the positive sample image and a non-target corresponding to the difficult image sample;

[0010] The first-stage target detection model is trained based on the positive sample image set under the at least one scene, the difficult image sample set and the fusion image sample set, to obtain a second-stage target detection model.

[0011] According to an aspect of an embodiment of the present application, a device for training a target detection model is provided, and the device comprises:

[0012] a data set obtaining module configured to obtain a training data set, wherein the training data set comprises image sub-sets under at least one scene;

[0013] a difficult image obtaining module configured to detect each image in the training data set by using a first-stage target detection model to obtain a difficult image set, wherein the first-stage target detection model is trained based on the training data set, and a difficult image in the difficult image set is an image that is detected by the first-stage target detection model as including a target but actually does not include the target;

[0014] a sample image sampling module configured to sample a positive sample image set under the at least one scene and a difficult image sample set under the difficult image set from the image sub-sets under the at least one scene and the difficult image set, wherein a positive sample image in the positive sample image set includes the target;

[0015] a fusion image obtaining module configured to obtain a fusion image sample set based on a positive sample image in the positive sample image set under the at least one scene and a difficult image sample in the difficult image sample set, wherein a fusion image sample in the fusion image sample set includes a target corresponding to the positive sample image and a non-target corresponding to the difficult image sample;

[0016] a detection model training module configured to train the first-stage target detection model based on the positive sample image set under the at least one scene, the difficult image sample set and the fusion image sample set, to obtain a second-stage target detection model.

[0017] According to an aspect of an embodiment of the present application, a computer device is provided, which comprises a processor and a memory, wherein the memory stores a computer program, the computer program is loaded and executed by the processor to implement the above-mentioned target detection model training method.

[0018] According to an aspect of an embodiment of the present application, a computer readable storage medium is provided, which stores a computer program, the computer program is loaded and executed by a processor to implement the above-mentioned target detection model training method.

[0019] According to an aspect of the embodiments of the present application, a computer program product or computer program is provided, which comprises computer instructions stored in a computer readable storage medium. A processor of a computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions, so that the computer device executes the training method of the target detection model.

[0020] The technical solutions provided by the embodiments of the present application have at least the following beneficial effects.

[0021] By training the first-stage target detection model based on the positive sample image, the difficult image sample and the fusion image sample, the identification capability of the target detection model for the difficult sample image is improved, so that the detection accuracy of the target detection model is improved, and the detection success rate of the target detection model is further improved. At the same time, the fusion image sample can realize the comparability between the target (such as the face) and the non-target (such as the non-face), and the target detection model is trained based on the fusion image sample, so that the robustness of the target detection model to the interference of the non-target to the target is improved.

[0022] In addition, the target detection model is trained in the second stage based on the positive sample image and the fusion image sample obtained based on the positive sample image, so that the balanced constraint of the positive sample and the negative sample is realized, the problem of insufficient model training and low training precision caused by too many negative samples is avoided, and the training efficiency and the training accuracy of the target detection model are improved. BRIEF DESCRIPTION OF DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0024] Figure 1 is a schematic diagram of the scheme implementation environment provided by an embodiment of the present application;

[0025] Figure 2 is a flowchart of the training method of the target detection model provided by an embodiment of the present application;

[0026] Figure 3 is a schematic diagram of the difficult image provided by an embodiment of the present application;

[0027] Figure 4is a flowchart of a first stage training method of a target detection model provided by an embodiment of the present application;

[0028] Figure 5 is a flowchart of a sampling method of a sample image provided by an embodiment of the present application;

[0029] Figure 6 is a schematic diagram of an acquisition method of a fusion image sample provided by an embodiment of the present application;

[0030] Figure 7 is a schematic diagram of a target grid with comparability provided by an embodiment of the present application;

[0031] Figure 8 is a block diagram of a training device of a target detection model provided by an embodiment of the present application;

[0032] Figure 9 is a structural schematic diagram of a computer device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0033] In order to make the purpose, technical scheme and advantages of the present application clearer, the embodiments of the present application will be further described in detail below with reference to the drawings.

[0034] Artificial intelligence (AI) is to use digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology of computer science, which tries to understand the essence of intelligence and produce a new intelligent machine that can react in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines, so that machines have the functions of perception, reasoning and decision-making.

[0035] Artificial intelligence technology is a comprehensive discipline, involving a wide range of fields, both hardware and software technologies. Artificial intelligence basic technologies generally include technologies such as sensors, special artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction system, mechatronics, etc. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning, etc.

[0036] Computer Vision (CV) is a science that studies how to make machines "see". More specifically, it refers to using cameras and computers to replace human eyes to identify and measure targets, and further perform image processing to make computer processing more suitable for human eye observation or image transmission to instruments for detection. As a scientific discipline, computer vision researches related theories and technologies, and attempts to establish artificial intelligence systems that can obtain information from images or multidimensional data. Computer vision technology generally includes image processing, image recognition, image semantic understanding, image retrieval, OCR (Optical Character Recognition), video processing, video semantic understanding, video content / behavior recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, and map construction technologies.

[0037] Machine Learning (ML) is a multi-disciplinary subject that involves probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory, and other disciplines. It is a specialized research on how computers simulate or implement human learning behavior to acquire new knowledge or skills, and reorganize existing knowledge structure to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental approach to making computers intelligent, and its applications are widespread in various fields of artificial intelligence. Machine learning and deep learning generally include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and teaching learning technologies.

[0038] The technical solutions provided by the embodiments of the present application relate to computer vision technology and machine learning technology of artificial intelligence. The computer vision technology is used to obtain a feature map corresponding to a sample image, and a detection result corresponding to the sample image is obtained based on the feature map corresponding to the sample image. Then, the machine learning technology is used to perform two-stage training on a target detection model based on the detection result corresponding to the sample image and label information corresponding to the sample image, so as to obtain a trained target detection model.

[0039] The execution subject of each step of the method provided by the embodiments of the present application can be a computer device, which refers to an electronic device with data calculation, processing, and storage capabilities. The computer device can be a terminal such as a PC (Personal Computer), a tablet computer, a smart phone, a wearable device, a smart robot, a vehicle-mounted terminal, or a server. The server can be a standalone physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server providing cloud computing services.

[0040] The technical scheme provided in the embodiments of the present application is applicable to any scene requiring target detection, such as a target detection scene (for example, face detection), an image analysis scene (for example, expression analysis, posture judgment, quality analysis, three-dimensional face reconstruction, three-dimensional animation, etc.), a smart traffic scene, an auxiliary driving scene, a data statistics scene, and the like. The technical scheme provided in the embodiments of the present application can improve the detection success rate and robustness of the target detection model.

[0041] Reference is made to Figure 1 which shows a schematic diagram of a scheme implementation environment provided in an embodiment of the present application. The scheme implementation environment can include a model training device 10 and a model using device 20.

[0042] The model training device 10 can be an electronic device such as a PC, a computer, a tablet computer, a smart robot, a vehicle-mounted terminal, or the like, or other electronic devices with strong computing capability, or a server. The model training device 10 is configured to train a target detection model 30.

[0043] In the embodiments of the present application, the target detection model 30 is a neural network model that can be used for target detection and target recognition tasks. For example, the target detection model 30 can be used to detect a target in an image, a video, a photo, or the like, to obtain a detection result for the target, which can be used to indicate the position, size, type, and the like of the target. The target is not limited in the embodiments of the present application, and can be, for example, a person (or a part of a person, such as a face, a head, a torso, or the like), an animal (or a part of an animal), a traffic element, an object (such as a car, a seat, a tree, or the like), or the like. For example, the target detection model 30 can be used to detect the position and size of a face in an input image.

[0044] Optionally, the model training device 10 can train the target detection model 30 in a machine learning manner, so that the target detection model 30 has better detection performance. Optionally, the embodiments of the present application can include a two-stage training process, a first stage is configured to preliminarily train the target detection model 30, and a second stage is configured to perform targeted training on the target detection model 30, to improve the detection success rate and robustness of the target detection model.

[0045] The trained target detection model 30 described above can be deployed in the model using device 20 for use, to provide a target detection service. The model using device 20 can be a terminal device such as a mobile phone, a computer, a smart television, a multimedia playing device, a wearable device, a vehicle-mounted terminal, a smart robot, or the like, or a server, and the embodiments of the present application are not limited thereto.

[0046] In some embodiments, the target detection model 30 can include a backbone network and a detection head network. The backbone network is configured to extract spatial features of an input image to obtain a feature map of the input image, which retains spatial structure information of the image (e.g., target and non-target). The detection head network is configured to map the feature map of the input image to obtain a detection result of the input image, which can be a vector including coordinate, size, detection confidence, and detection object category, etc. For example, the input image can be divided into a plurality of grids, such as S*S grids, S being the size of the grid, and a plurality of bounding boxes can be set for each grid, which can be denoted as B bounding boxes, B being a positive integer. The detection result can refer to the detection result corresponding to each bounding box (i.e., S*S*B bounding boxes), which can be used to indicate the predicted coordinate and size of the bounding box, the detection confidence of whether the target exists in the bounding box, and the predicted category of the bounding box.

[0047] Optionally, the backbone network and the detection head network can be constructed based on any one of a CNN (Convolutional Neural Network), an R-CNN (Region-CNN), a Faster R-CNN, an SSD (Single Shot MultiBox Detector), and a YOLO (You Only Look Once). The CNN can include convolution calculation, non-linear activation function calculation, and pooling calculation.

[0048] For example, referring to Figure 1 , the two-stage training process of the target detection model can include the following three parts:

[0049] Target detection model initialization: Obtain training data, which includes at least one image subset in each scene, and sample sample images in each scene from each image subset. The target detection model 30 is trained in the first stage by the obtained training samples in sequence, and the target detection model 30 in the first stage is obtained.

[0050] Negative sample data mining: Based on the training data, the difficult image set is obtained by the target detection model 30 in the first stage, and the difficult image in the difficult image set is the image that is detected by the target detection model in the first stage as including a target but actually does not include a target. Then, the positive sample images and the difficult sample images are sampled from each image subset and the difficult image set.

[0051] Fine-tuning training of the target detection model: the positive sample images and the difficult sample images are fused to obtain fused image samples. Finally, based on the positive sample images, the difficult image samples and the fused image samples, the first-stage target detection model is trained in the second stage to obtain the second-stage target detection model, i.e., the trained target detection model. The second-stage target detection model can be used for target detection.

[0052] The specific training method of the target detection model will be described in detail below.

[0053] Please refer to Figure 2 which shows a flowchart of the training method of the target detection model provided in an embodiment of the present application. The execution subject of each step of the method can be the model training device introduced above. The method can include the following steps (201-205).

[0054] Step 201: Obtain a training data set, which includes at least one image sub-set in a scene.

[0055] Each image in the training data set can be used as a candidate sample image for sampling of sample images.

[0056] The source scene of the images in the training data set is not limited in the embodiments of the present application, such as traffic roads, shopping malls, stations, tourist attractions, office areas, schools, parks, etc. Each image sub-set can include images including targets or images including non-targets, i.e., images in the scene can be included in the image sub-set corresponding to the scene. For example, taking a face as a target, the image sub-set can include images including faces and images including non-faces.

[0057] By collecting training data from at least one scene and training the target detection model based on the training data, the universality of the target detection model can be improved.

[0058] Step 202: Detect each image in the training data set by the first-stage target detection model to obtain a difficult image set; wherein the first-stage target detection model is trained based on the training data set, and the difficult image in the difficult image set is an image detected by the first-stage target detection model as including a target but actually not including a target.

[0059] The first-stage target detection model refers to the target detection model after preliminary training. For the first-stage target detection model, there are also misdetected images, i.e., the difficult images in the embodiments of the present application. For example, the difficult image can be an image that is easily misdetected by the first-stage target detection model and is a non-target. For example, refer to Figure 3, the first-stage target detection model can incorrectly detect the traffic sign 301 and the face of the dog 302 as a face of a person, and the two images can be determined as difficult images. Figure 3

[0060] In one example, referring to Figure 4 and Figure 5 , the training process of the first-stage target detection model can be as follows: the sample image sampling model 401 determines a target image sub-set according to the sampling weights corresponding to the n scenes respectively, and then randomly samples the sample images corresponding to the sampling quantities from the target image sub-set according to the sampling quantities corresponding to the target image sub-set, and stops the sampling of the sample images in the case that the n scenes all have corresponding sample images, so as to ensure sufficient sampling. Optionally, the sampling weights corresponding to the n scenes respectively and the sampling quantities corresponding to the n scenes respectively can be set and adjusted according to empirical values. For example, the sampling weights corresponding to the n scenes respectively can be set to be consistent, and the sampling quantities corresponding to the n scenes respectively can be set to be consistent.

[0061] The sample image enhancement module 402 performs data enhancement on each sample image to obtain an enhanced sample image. Optionally, the data enhancement manner is not limited in the embodiments of the present application, such as but not limited to: left-right flipping, random black noise addition, brightness adjustment, tone adjustment, random cropping, random erasing, and the like. The embodiments of the present application can make the target detection model learn different forms of the same image by performing different forms of data enhancement on the sample image, which is beneficial to improve the robustness of the target detection model.

[0062] The target detection model 403 detects the data-enhanced sample image to obtain a corresponding detection result. Exemplarily, the data-enhanced sample image is divided into S*S grids, and the target detection model 403 outputs, for each grid, B predicted detection boxes, and predicted coordinates and predicted sizes, detection confidence and predicted categories corresponding to the predicted detection boxes. Wherein, the coordinates of the detection box can be represented by the upper left corner coordinates (x, y), and the size of the detection box can be represented by the width and height (w, h).

[0063] The training loss calculation module 404 calculates the training loss based on the detection result and the label information. Exemplarily, the training loss of the target detection model 403 can be represented as follows:

[0064] L total =L coord +L noobj +L class ;

[0065] Wherein, L coord ​L is a first training loss used to indicate a prediction accuracy of the target detection model on a position and a size of a target, noobj L is a second training loss used to indicate a prediction accuracy of the target detection model on whether a target is included in a sample image, class L is a third training loss used to indicate a prediction accuracy of the target detection model on a category corresponding to a detection box.

[0066] Optionally, the first training loss can be represented as follows:

[0067]

[0068] wherein S 2 is a total number of grids corresponding to the sample image, S is a size of the grid, B is a number of detection boxes corresponding to each grid, x ij is a predicted horizontal coordinate of the jth detection box corresponding to the ith grid, is a real horizontal coordinate of the jth detection box corresponding to the ith grid, y ij is a predicted vertical coordinate of the jth detection box corresponding to the ith grid, is a real vertical coordinate of the jth detection box corresponding to the ith grid, w ij is a predicted width of the jth detection box corresponding to the ith grid, is a real width of the jth detection box corresponding to the ith grid, h ij is a predicted height of the jth detection box corresponding to the ith grid, is a real height of the jth detection box corresponding to the ith grid.

[0069] The second training loss can be represented as follows:

[0070]

[0071] wherein, represents outputting 1 if a target is included in the jth detection box corresponding to the ith grid, and otherwise outputting 0, represents outputting 0 if a target is included in the jth detection box corresponding to the ith grid, and otherwise outputting 1, C i represents a detection confidence that a target is included in the ith grid, represents real label information on whether a target is included in the ith grid.

[0072] The third training loss can be represented as follows:

[0073]

[0074] wherein, represents outputting 1 if the target is included in the ith grid, otherwise outputting 0, c represents a category of the detected target (such as a face), class represents all categories corresponding to the sample image, p i (c) represents a predicted category corresponding to the ith grid, represents real label information of the category corresponding to the ith grid.

[0075] If the target detection model does not meet the training termination condition, the model is trained by the model training module 405, the target detection model 403 is trained based on the training loss, and the sample image is continuously obtained by the sample image sampling module 401. Optionally, the sample image can be cyclically sampled, or the sampling can be completed at one time, and the embodiments of the present application do not limit this. Wherein, the target detection model 403 can be trained by using gradient descent, such as stochastic gradient descent, stochastic gradient descent with momentum, Adam algorithm, Adagard algorithm, etc.

[0076] If the target detection model meets the training termination condition, the training of the target detection model 403 is stopped, and the first-stage target detection model is obtained. Wherein, the training termination condition can be that the number of iterations meets the set value, or the training loss of the target detection model 403 is less than the set value, or the training loss of the target detection model 403 tends to be stable, and the embodiments of the present application do not limit this.

[0077] Step 203, sample a positive sample image set under at least one scene and a difficult image sample set under the difficult image set from the image sub-set under at least one scene and the difficult image set, wherein the positive sample image in the positive sample image set includes a target.

[0078] In the embodiments of the present application, the positive sample image can refer to an image that does not include a non-target. For example, taking the target as a face, each positive sample image is a normal face image, and each difficult image sample is a non-face image that is easy to detect errors.

[0079] In one example, step 203 can include the following contents: obtaining the sampling weight corresponding to the image sub-set under at least one scene and the difficult image set respectively; in each sampling process, determining the target set from the image sub-set under at least one scene and the difficult image set based on the sampling weight corresponding to the image sub-set under at least one scene and the difficult image set respectively; sampling in the target set according to the sampling number corresponding to the target set, to obtain the sample set corresponding to the target set.

[0080] The difficult images and the images including non-targets can be removed from the at least one image subset to obtain an adjusted image subset, and the difficult images are taken as an additional training data branch, and the sampling weight corresponding to the at least one scene and the sampling weight corresponding to the difficult image set are set. For example, the sampling weight corresponding to the difficult image set is set to be consistent with the sampling weight corresponding to the at least one scene.

[0081] For example, based on the above embodiment, on the basis of the n adjusted image subsets, the difficult image set is added to obtain n+1 image subsets. Then, according to the sampling weight, the target set is determined from the n+1 image subsets, and then the target set is randomly sampled to obtain sample images corresponding to the sampling quantity. If the sampling quantity is consistent, denoted as b, then there are (n+1)*b data quantities. Among them, n*b is the data quantity of the positive sample images, and b is the data quantity of the difficult image samples.

[0082] In step 204, based on the positive sample images in the positive sample image set under the at least one scene and the difficult image samples in the difficult image sample set, a fusion image sample set is obtained; wherein the fusion image samples in the fusion image sample set include the targets corresponding to the positive sample images and the non-targets corresponding to the difficult image samples.

[0083] In the embodiment of the present application, the data quantity of the fusion image samples is consistent with the data quantity of the positive sample images, so as to balance the constraint of the positive samples and the negative samples. At the same time, the fusion image samples can make the positive sample images and the difficult image samples comparable, that is, the non-targets are taken as the interference factors of the targets, so as to improve the robustness of the target detection model.

[0084] Optionally, for each positive sample image in the positive sample image set under the at least one scene, a target difficult image sample can be randomly determined from the difficult image sample set to obtain a target difficult image sample corresponding to the positive sample image; then the positive sample image and the target difficult image sample corresponding to the positive sample image are fused to obtain a fusion image sample corresponding to the positive sample image; wherein the fusion image sample set includes the fusion image samples corresponding to each positive sample image in the positive sample image set respectively.

[0085] In one example, the fusion image sample can refer to an image obtained by fusing a target in the positive sample image into a difficult sample image. The specific fusion process can be as follows: performing data augmentation on the positive sample image to obtain a data-augmented positive sample image; cutting at least one target cut image from the data-augmented positive sample image, the target cut image including the target; obtaining a first target cut image from the at least one target cut image; and fusing the first target cut image into the target difficult image sample corresponding to the positive sample image to obtain a fusion image sample corresponding to the positive sample image.

[0086] For example, referring to Figure 6 , in order, from the positive sample image set, obtaining a positive sample image 601, performing data augmentation on the positive sample image 601 to obtain an augmented positive sample image 601, randomly sampling a difficult image sample from the difficult image sample set as a target difficult image sample 602 corresponding to the augmented positive sample image 601, cutting a target cut image 603 and a target cut image 604 from the augmented positive sample image 601, randomly selecting a target cut image from the target cut image 603 and the target cut image 604 as a first target cut image, and finally fusing the first target cut image into the target difficult image sample 602 to obtain a fusion image sample 605 corresponding to the positive sample image 601.

[0087] Optionally, the embodiments of the present application do not limit the size and fusion position of the first target cut image, which can be adaptively set and adjusted according to experience values.

[0088] In another example, the fusion image sample can refer to an image obtained by splicing a region including a non-target in the difficult image with a region including a target in the positive sample image. The specific fusion process can be as follows: performing data augmentation on the positive sample image to obtain a data-augmented positive sample image; cutting at least one target cut image from the data-augmented positive sample image, the target cut image including the target; cutting at least one non-target cut image from the target difficult image sample corresponding to the positive sample image, the non-target cut image including the non-target; obtaining a first target cut image from the at least one target cut image, and obtaining a first non-target cut image from the at least one non-target cut image; and splicing the first target cut image and the first non-target cut image to obtain a fusion image sample corresponding to the positive sample image.

[0089] For example, in combination with Figure 6The non-target intercepted image 606 in the target difficult image sample 602 is intercepted, and since there is only one non-target intercepted image, the non-target intercepted image 606 can be directly determined as the first non-target intercepted image 606. The first non-target intercepted image 606 and the first target intercepted image 604 are spliced to obtain a fusion image sample 607.

[0090] Optionally, in the acquisition of the fusion image sample, the positive sample image can be subjected to data enhancement, or can not be subjected to data enhancement, the difficult image sample can be subjected to data enhancement, or can not be subjected to data enhancement.

[0091] In step 205, based on the positive sample image set, the difficult image sample set and the fusion image sample set in at least one scene, the target detection model in the first stage is trained to obtain the target detection model in the second stage.

[0092] The target detection model in the second stage is the trained target detection model, and the target detection model in the second stage has the ability to correctly detect difficult images and images with interference factors of non-targets in difficult images. Optionally, based on the above embodiment, the data amount of the sample image in the second stage can be denoted as (2*n+1)*b, which includes n*b positive sample images, n*b fusion image samples and b difficult image samples.

[0093] In one example, the acquisition process of the target detection model in the second stage can be as follows:

[0094] 1. For the target sample images in the positive sample image set, the difficult image sample set and the fusion image sample set in at least one scene, the detection results corresponding to the target sample images are acquired by the target detection model in the first stage; wherein the detection results include the prediction bounding box, the detection confidence and the prediction class corresponding to each detection frame of the target sample image.

[0095] The target sample image can be any sample image in the positive sample image set, the difficult image sample set and the fusion image sample set. It should be noted that the detection result in the first stage training process is acquired by the target detection model (not trained), and the detection result in the second stage training process is acquired by the target detection model in the first stage.

[0096] 2. Based on the prediction bounding box and the detection frame corresponding to the target sample image, a first training loss is determined, which is used to indicate the prediction accuracy of the target detection model in the first stage for the position and size of the target.

[0097] The calculation function of the first training loss in the second stage training process is the same as the calculation function of the first training loss in the first stage training process, and the first training loss in the second stage training process can be obtained based on each prediction bounding box and each detection bounding box corresponding to the target sample image by using the calculation function.

[0098] 3. Determine a second training loss based on each detection confidence corresponding to the target sample image and label information of the target in the target sample image, the second training loss being used to indicate the prediction accuracy of the target detection model in the first stage for whether the target sample image includes the target.

[0099] Optionally, the calculation function of the second training loss in the second stage training process is different from the calculation function of the second training loss in the first stage training process in a weight parameter. For example, the second training loss includes a first sub-loss corresponding to the detection confidence of the target and a second sub-loss corresponding to the detection confidence of the non-target; the weight parameter of the second sub-loss corresponding to the target detection model after the training is greater than the weight parameter of the second sub-loss corresponding to the target detection model in the first stage.

[0100] For example, the calculation function of the second training loss in the second stage training process can be expressed as follows:

[0101]

[0102] wherein β is an adjustment weight, is the first sub-loss, is the second sub-loss.

[0103] In an exemplary embodiment, in the first stage training process, the weight parameter ratio between the second sub-loss and the first sub-loss is 1:1, and in the two-stage training process, the weight parameter of the first sub-loss can be kept unchanged and the weight parameter of the second sub-loss can be enhanced so as to achieve the weight parameter ratio between the second sub-loss and the first sub-loss of 1.5:1. Alternatively, the weight parameter ratio between the second sub-loss and the first sub-loss can be directly set to 1.5:1. By increasing the weight of the second sub-loss in the second stage training process, the optimization efficiency of the target detection model can be improved.

[0104] 4. Determine a third training loss based on each prediction category corresponding to the target sample image and the real category corresponding to each detection bounding box, the third training loss being used to indicate the prediction accuracy of the target detection model in the first stage for the category corresponding to the detection bounding box.

[0105] The calculation function of the third training loss in the second stage training process is the same as the calculation function of the third training loss in the first stage training process. Based on the respective predicted categories of the target sample image and the respective real categories of the respective detection boxes, the third training loss in the second stage training process can be obtained.

[0106] It should be noted that for the fusion image sample in the fusion image sample set, in each grid corresponding to the fusion image sample, there is a target grid that simultaneously includes the pixels corresponding to the target and the pixels corresponding to the non-target; wherein each grid corresponds to a plurality of detection boxes.

[0107] The target grid can make the target and the non-target comparable, that is, the non-target is regarded as a disturbance factor of the target, thereby improving the robustness of the target detection model to the disturbance of the non-target to the target.

[0108] For example, referring to Figure 7 For the detection box 702 corresponding to the fusion image sample 701, there is a target and a non-target. The real label data corresponding to the detection box 702 is 1. If the target detection model in the second stage can output a predicted category of 1 and a predicted bounding box that can be automatically adjusted to a detection result including the target for the detection box 702, the target detection model in the second stage has the ability to correctly detect the non-target.

[0109] 5. Based on the respective first training loss, second training loss and third training loss of each sample image in the positive sample image set, difficult sample image set and fusion image sample set under at least one scene, the target detection model in the first stage is trained to obtain the target detection model in the second stage.

[0110] Optionally, the same training method as the first stage training process can be used to iteratively train the target detection model in the first stage based on the first training loss, the second training loss and the third training loss to obtain the target detection model in the second stage.

[0111] In one feasible example, the difficult image samples can be classified in detail, and for different classified difficult image samples, the corresponding detection head network is trained by using the technical solution provided in the embodiments of the present application to improve the detection accuracy of the target detection model for a certain type of difficult image.

[0112] In summary, the technical scheme provided by the embodiments of the present application trains the target detection model in the first stage based on the positive sample image, the difficult image sample and the fusion image sample. The target detection model is trained based on the difficult image sample, which can improve the recognition ability of the target detection model for difficult sample images, thereby improving the detection accuracy of the target detection model, and further improving the detection success rate of the target detection model. At the same time, the fusion image sample can realize the comparability between the target (such as the face) and the non-target (such as the non-face). The target detection model is trained based on the fusion image sample, which can improve the robustness of the target detection model to the interference of the non-target to the target.

[0113] In addition, the target detection model is trained in the second stage based on the positive sample image and the fusion image sample obtained based on the positive sample image, which realizes the balanced constraint of the positive sample and the negative sample, avoids the problem of insufficient model training and low training accuracy caused by too many negative samples, and thereby improves the training efficiency and training accuracy of the target detection model.

[0114] In addition, by increasing the weight of the second sub-loss in the second stage training process, the optimization efficiency of the target detection model can be improved.

[0115] In addition, since the embodiments of the present application only optimize the training process of the target detection model, without increasing the parameter amount of the target detection model or setting an additional interception model for the non-target, the target detection model trained by the embodiments of the present application is compatible with the target detection model that has been deployed online, thereby reducing the cost of model optimization and improving the practicality of the model optimization method.

[0116] In one exemplary embodiment, the target detection model is taken as a face detection model for face (such as human face) detection, and the training method of the target detection model provided by the embodiments of the present application is introduced, which can specifically include the following contents.

[0117] First stage training:

[0118] Obtain a plurality of image subsets in different scenes, such as face images and non-face images in a traffic scene, face images and non-face images in a vehicle, and face images and non-face images in a station. The face image can refer to an image including a face, and the non-face image can refer to an image including a target object other than a face or not including a face.

[0119] According to the sampling weight, the image subsets in each scene are independently sampled, and for each image subset, uniform random sampling is performed to obtain a first-stage sample image set. In this way, the sampling is independent of the number of images in each scene, thereby improving the rationality of the sampling.

[0120] Data augmentation is performed on each sample image in the first-stage sample image set to obtain a first-stage enhanced sample image set.

[0121] The face detection model is iteratively trained through the first-stage enhanced sample image set to obtain a first-stage face detection model.

[0122] Second-stage training:

[0123] Each image in the image subsets in multiple scenes is detected through the first-stage face detection model to obtain a difficult image set. The difficult image refers to a non-face image that is misdetected as a face by the first-stage face detection model.

[0124] The difficult images and non-face images are removed from the multiple image subsets to obtain multiple adjusted image subsets.

[0125] According to the sampling weight, the adjusted image subsets and the difficult image set are independently sampled, and for each set, uniform random sampling is performed to obtain a second-stage preliminary sample image set. The second-stage preliminary sample image set includes a positive sample set corresponding to multiple scenes and a negative sample set corresponding to the difficult image set.

[0126] Based on the positive sample set and the negative sample set, a fusion sample set with the same data volume as the positive sample set is obtained. The fusion samples in the fusion sample set include faces in the positive samples and non-faces in the negative samples.

[0127] Based on the fusion sample set, the positive sample set and the negative sample set, the first-stage face detection model is trained to obtain a second-stage face detection model.

[0128] In summary, the technical scheme provided by the embodiments of the present application can improve the recognition ability of the face detection model for negative samples by training the first-stage face detection model based on negative samples, thereby improving the detection accuracy of the face detection model and further improving the detection success rate of the face detection model. At the same time, the fusion sample can realize the comparability between the face and the non-face, and the first-stage face detection model can be trained based on the fusion sample to improve the robustness of the face detection model to non-face interference.

[0129] The following is an embodiment of the device of the present application, which can be used to execute the method embodiments of the present application. For details not disclosed in the device embodiments of the present application, please refer to the method embodiments of the present application.

[0130] Please refer to Figure 8 which shows a block diagram of a training device of a target detection model according to an embodiment of the present application. The device can be used to implement the training method of the target detection model described above. The device 800 can include a data set acquisition module 801, a difficult image acquisition module 802, a sample image sampling module 803, a fusion image acquisition module 804, and a detection model training module 805.

[0131] The data set acquisition module 801 is configured to acquire a training data set, wherein the training data set includes at least one image sub-set under a scene.

[0132] The difficult image acquisition module 802 is configured to detect each image in the training data set by a first-stage target detection model to obtain a difficult image set, wherein the first-stage target detection model is trained based on the training data set, and the difficult image in the difficult image set is an image that is detected by the first-stage target detection model as including a target but actually does not include the target.

[0133] The sample image sampling module 803 is configured to sample a positive sample image set under the at least one scene and a difficult image sample set under the difficult image set from the image sub-set under the at least one scene and the difficult image set, wherein the positive sample image in the positive sample image set includes the target.

[0134] The fusion image acquisition module 804 is configured to obtain a fusion image sample set based on the positive sample image in the positive sample image set under the at least one scene and the difficult image sample in the difficult image sample set, wherein the fusion image sample in the fusion image sample set includes the target corresponding to the positive sample image and the non-target corresponding to the difficult image sample.

[0135] The detection model training module 805 is configured to train the first-stage target detection model based on the positive sample image set under the at least one scene, the difficult image sample set, and the fusion image sample set to obtain a second-stage target detection model.

[0136] In an exemplary embodiment, the fusion image acquisition module 804 is configured to:

[0137] For each positive sample image in the positive sample image set under the at least one scene, a target difficult image sample is determined from the difficult image sample set randomly, to obtain a target difficult image sample corresponding to the positive sample image;

[0138] The positive sample image and the target difficult image sample corresponding to the positive sample image are fused to obtain a fusion image sample corresponding to the positive sample image; wherein the fusion image sample set includes the fusion image sample corresponding to each positive sample image in the positive sample image set.

[0139] In an example embodiment, the fusion image acquisition module 804 is further configured to:

[0140] The positive sample image is data enhanced to obtain a data-enhanced positive sample image;

[0141] At least one target cropped image is cropped from the data-enhanced positive sample image, and the target cropped image includes the target;

[0142] A first target cropped image in the at least one target cropped image is acquired;

[0143] The first target cropped image is fused into the target difficult image sample corresponding to the positive sample image to obtain a fusion image sample corresponding to the positive sample image.

[0144] In an example embodiment, the fusion image acquisition module 804 is further configured to:

[0145] The positive sample image is data enhanced to obtain a data-enhanced positive sample image;

[0146] At least one target cropped image is cropped from the data-enhanced positive sample image, and the target cropped image includes the target;

[0147] At least one non-target cropped image is cropped from the target difficult image sample corresponding to the positive sample image, and the non-target cropped image includes the non-target;

[0148] A first target cropped image in the at least one target cropped image is acquired, and a first non-target cropped image in the at least one non-target cropped image is acquired;

[0149] The first target cropped image and the first non-target cropped image are spliced to obtain a fusion image sample corresponding to the positive sample image.

[0150] In an example embodiment, the sample image sampling module 803 is configured to:

[0151] obtain sampling weights corresponding to the image sub-set under the at least one scene and the difficult image set respectively;

[0152] In each round of sampling process, based on the sampling weights corresponding to the image sub-set under the at least one scene and the difficult image set respectively, determine a target set from the image sub-set under the at least one scene and the difficult image set;

[0153] According to the sampling number corresponding to the target set, sample in the target set to obtain a sample set corresponding to the target set.

[0154] In one example embodiment, the detection model training module 805 is configured to:

[0155] For a target sample image in the positive sample image set under the at least one scene, the difficult image sample set and the fusion image sample set, obtain a detection result corresponding to the target sample image by the first-stage target detection model; wherein the detection result includes a prediction bounding box corresponding to each detection box of the target sample image, a detection confidence and a prediction class;

[0156] Based on each prediction bounding box corresponding to the target sample image and the detection box, determine a first training loss, which is used to indicate the prediction accuracy of the first-stage target detection model for the position and size of the target;

[0157] Based on each detection confidence corresponding to the target sample image and the label information of the target in the target sample image, determine a second training loss, which is used to indicate the prediction accuracy of the first-stage target detection model for whether the target sample image includes the target;

[0158] Based on each prediction class corresponding to the target sample image and the real class corresponding to each detection box, determine a third training loss, which is used to indicate the prediction accuracy of the first-stage target detection model for the class corresponding to the detection box;

[0159] Based on the first training loss, the second training loss and the third training loss corresponding to each sample image in the positive sample image set under the at least one scene, the difficult image sample set and the fusion image sample set respectively, train the first-stage target detection model to obtain the second-stage target detection model.

[0160] In an example embodiment, in each of the grids corresponding to the fusion image samples in the set of fusion image samples, there is a target grid that includes both the pixels corresponding to the target and the pixels corresponding to the non-target; and each grid corresponds to a plurality of the detection boxes.

[0161] In an example embodiment, the second training loss includes a first sub-loss and a second sub-loss, the first sub-loss corresponds to the detection confidence of the target, and the second sub-loss corresponds to the detection confidence of the non-target; the weight parameter of the second sub-loss corresponding to the target detection model after the training is greater than the weight parameter of the second sub-loss corresponding to the target detection model of the first stage.

[0162] In summary, the technical scheme provided by the embodiments of the present application trains the target detection model of the first stage based on the positive sample image, the difficult image sample, and the fusion image sample. Training the target detection model of the first stage based on the difficult image sample can improve the recognition ability of the target detection model for difficult sample images, thereby improving the detection accuracy of the target detection model and further improving the detection success rate of the target detection model. At the same time, the fusion image sample can realize the comparability between the target (such as the face) and the non-target (such as the non-face), and training the target detection model of the first stage based on the fusion image sample can improve the robustness of the target detection model to the interference of the non-target to the target.

[0163] In addition, since the target detection model is trained in the second stage based on the positive sample image and the fusion image sample obtained based on the positive sample image, the balance constraint of the positive sample and the negative sample is realized, the problem of insufficient model training and low training accuracy caused by too many negative samples is avoided, and the training efficiency and the training accuracy of the target detection model are improved.

[0164] It should be noted that the device provided in the above embodiments is only used as an example to illustrate the division of the above functional modules in realizing its functions. In actual applications, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the above described functions. In addition, the device and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process is described in detail in the method embodiments, which will not be repeated here.

[0165] Please refer to Figure 9 which shows a structural schematic diagram of a computer device provided by an embodiment of the present application. The computer device can be any electronic device with data calculation, processing and storage functions, and the computer device can be realized as Figure 1The model training device 10 and / or the model using device 20 in the illustrated scheme implementation environment. Specifically, it can include the following.

[0166] The computer device 900 includes a central processing unit (such as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), and an FPGA (Field Programmable Gate Array), etc.) 901, a system memory 904 including a RAM (Random-Access Memory) 902 and a ROM (Read-Only Memory) 903, and a system bus 905 connecting the system memory 904 and the central processing unit 901. The computer device 900 also includes a basic input / output system (I / O system) 906 that helps transfer information between various devices within the server, and a mass storage device 907 for storing an operating system 913, application programs 914, and other program modules 915.

[0167] In some embodiments, the basic input / output system 906 includes a display 908 for displaying information and an input device 909 such as a mouse, a keyboard, or the like for inputting information by a user. The display 908 and the input device 909 are both connected to the central processing unit 901 through an input / output controller 910 connected to the system bus 905. The basic input / output system 906 can also include the input / output controller 910 for receiving and processing input from a keyboard, a mouse, or an electronic stylus, and the like. Similarly, the input / output controller 910 also provides output to a display screen, a printer, or other types of output devices.

[0168] The mass storage device 907 is connected to the central processing unit 901 through a mass storage controller (not shown) connected to the system bus 905. The mass storage device 907 and its associated computer readable medium provide non-volatile storage for the computer device 900. That is, the mass storage device 907 can include a computer readable medium (not shown) such as a hard disk or a CD-ROM (Compact Disc Read-Only Memory) drive.

[0169] Without loss of generality, the computer readable medium can include computer storage media and communication media. Computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer storage media includes RAM, ROM, EPROM (Erasable Programmable Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), flash memory or other solid state memory technology, CD-ROM, DVD (Digital Video Disc), or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices. It should be understood by those skilled in the art that computer storage media does not limit to the above-mentioned several types. The system memory 904 and the mass storage device 907 mentioned above can be collectively referred to as memory.

[0170] According to the embodiments of the present application, the computer device 900 can also run on a remote computer connected to the network through a network such as the Internet. That is, the computer device 900 can be connected to the network 912 through the network interface unit 911 connected to the system bus 905, or can be connected to other types of network or remote computer system (not shown) using the network interface unit 911.

[0171] The memory further includes a computer program stored in the memory and configured to be executed by one or more processors to implement the training method of the target detection model.

[0172] In one exemplary embodiment, a computer readable storage medium is also provided, and the storage medium stores a computer program which, when executed by a processor, implements the training method of the target detection model.

[0173] Optionally, the computer readable storage medium can include ROM (Read-Only Memory), RAM (Random-Access Memory), SSD (Solid State Drives) or optical disc, etc. Among them, the random access memory can include ReRAM (Resistance Random Access Memory) and DRAM (Dynamic Random Access Memory).

[0174] In an example embodiment, a computer program product or computer program is also provided, which includes computer instructions stored in a computer readable storage medium. A processor of a computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions, so that the computer device executes the training method of the target detection model described above.

[0175] It should be noted that the information (including but not limited to object device information, object personal information, etc.), data (including but not limited to data for analysis, stored data, displayed data, etc.) and signals involved in the present application are all authorized by the object or fully authorized by all parties, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards of relevant countries and regions. For example, the images and the like involved in the present application are obtained under sufficient authorization.

[0176] It should be understood that "multiple" referred to in the present text refers to two or more. "And / or" describes the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B can represent: A exists alone, A and B exist together, and B exists alone. The character " / " generally represents that the associated objects before and after it are in an "or" relationship. In addition, the step numbers described in the present text only exemplarily show a possible execution order between steps, and in some other embodiments, the above steps can also be executed in a non-numbered order, such as two steps with different numbers are executed at the same time, or two steps with different numbers are executed in an order opposite to the illustration, and the embodiments of the present application do not limit this.

[0177] The above only describes example embodiments of the present application, and is not intended to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A method for training a target detection model, the method comprising: The method comprises: obtaining a training data set, the training data set comprising a sub-set of images under at least one scene; detecting each image in the training data set by a first-stage target detection model to obtain a difficult image set, wherein the first-stage target detection model is trained based on the training data set, and a difficult image in the difficult image set is an image detected by the first-stage target detection model as including a target but actually not including the target; sampling a positive sample image set under the at least one scene and a difficult image sample set from the difficult image set, wherein a positive sample image in the positive sample image set includes the target; obtaining a fusion image sample set based on the positive sample image in the positive sample image set under the at least one scene and the difficult image sample in the difficult image sample set, wherein a fusion image sample in the fusion image sample set includes a target corresponding to the positive sample image and a non-target corresponding to the difficult image sample; obtaining a detection result corresponding to a target sample image in the positive sample image set under the at least one scene, the difficult image sample set and the fusion image sample set by the first-stage target detection model, wherein the detection result includes a prediction bounding box corresponding to each detection box of the target sample image, a detection confidence and a prediction class; determining a first training loss based on each prediction bounding box of the target sample image and the detection box, wherein the first training loss is used to indicate the prediction accuracy of the first-stage target detection model for the position and size of the target; determining a second training loss based on each detection confidence of the target sample image and label information of the target in the target sample image, wherein the second training loss is used to indicate the prediction accuracy of the first-stage target detection model for whether the target sample image includes the target; determining a third training loss based on each prediction class of the target sample image and a real class corresponding to the detection box, wherein the third training loss is used to indicate the prediction accuracy of the first-stage target detection model for the class corresponding to the detection box; training the first-stage target detection model based on the first training loss, the second training loss and the third training loss corresponding to each sample image in the positive sample image set under the at least one scene, the difficult image sample set and the fusion image sample set to obtain a second-stage target detection model.

2. The method of claim 1, wherein, The method comprises: obtaining a training data set, the training data set comprising a sub-set of images under at least one scene; detecting each image in the training data set by a first-stage target detection model to obtain a difficult image set, wherein the first-stage target detection model is trained based on the training data set, and a difficult image in the difficult image set is an image detected by the first-stage target detection model as including a target but actually not including the target; sampling a positive sample image set under the at least one scene and a difficult image sample set from the difficult image set, wherein a positive sample image in the positive sample image set includes the target; obtaining a fusion image sample set based on the positive sample image in the positive sample image set under the at least one scene and the difficult image sample in the difficult image sample set, wherein a fusion image sample in the fusion image sample set includes a target corresponding to the positive sample image and a non-target corresponding to the difficult image sample; obtaining a detection result corresponding to a target sample image in the positive sample image set under the at least one scene, the difficult image sample set and the fusion image sample set by the first-stage target detection model, wherein the detection result includes a prediction bounding box corresponding to each detection box of the target sample image, a detection confidence and a prediction class; determining a first training loss based on each prediction bounding box of the target sample image and the detection box, wherein the first training loss is used to indicate the prediction accuracy of the first-stage target detection model for the position and size of the target; determining a second training loss based on each detection confidence of the target sample image and label information of the target in the target sample image, wherein the second training loss is used to indicate the prediction accuracy of the first-stage target detection model for whether the target sample image includes the target; determining a third training loss based on each prediction class of the target sample image and a real class corresponding to the detection box, wherein the third training loss is used to indicate the prediction accuracy of the first-stage target detection model for the class corresponding to the detection box; training the first-stage target detection model based on the first training loss, the second training loss and the third training loss corresponding to each sample image in the positive sample image set under the at least one scene, the difficult image sample set and the fusion image sample set to obtain a second-stage target detection model. The method comprises: obtaining a training data set, the training data set comprising a sub-set of images under at least one scene; detecting each image in the training data set by a first-stage target detection model to obtain a difficult image set, wherein the first-stage target detection model is trained based on the training data set, and a difficult image in the difficult image set is an image detected by the first-stage target detection model as including a target but actually not including the target; sampling a positive sample image set under the at least one scene and a difficult image sample set from the difficult image set, wherein a positive sample image in the positive sample image set includes the target; obtaining a fusion image sample set based on the positive sample image in the positive sample image set under the at least one scene and the difficult image sample in the difficult image sample set, wherein a fusion image sample in the fusion image sample set includes a target corresponding to the positive sample image and a non-target corresponding to the difficult image sample; obtaining a detection result corresponding to a target sample image in the positive sample image set under the at least one scene, the difficult image sample set and the fusion image sample set by the first-stage target detection model, wherein the detection result includes a prediction bounding box corresponding to each detection box of the target sample image, a detection confidence and a prediction class; determining a first training loss based on each prediction bounding box of the target sample image and the detection box, wherein the first training loss is used to indicate the prediction accuracy of the first-stage target detection model for the position and size of the target; determining a second training loss based on each detection confidence of the target sample image and label information of the target in the target sample image, wherein the second training loss is used to indicate the prediction accuracy of the first-stage target detection model for whether the target sample image includes the target; determining a third training loss based on each prediction class of the target sample image and a real class corresponding to the detection box, wherein the third training loss is used to indicate the prediction accuracy of the first-stage target detection model for the class corresponding to the detection box; training the first-stage target detection model based on the first training loss, the second training loss and the third training loss corresponding to each sample image in the positive sample image set under the at least one scene, the difficult image sample set and the fusion image sample set to obtain a second-stage target detection model. For each positive sample image in the positive sample image set under the at least one scene, a target difficult image sample is determined from the difficult image sample set at random, obtaining a target difficult image sample corresponding to the positive sample image; The positive sample image and the target difficult image sample corresponding to the positive sample image are fused to obtain a fusion image sample corresponding to the positive sample image; The fusion image sample set comprises a fusion image sample corresponding to each positive sample image in the positive sample image set.

3. The method of claim 2, wherein, The fusion image sample corresponding to the positive sample image is obtained by fusing the positive sample image and the target difficult image sample corresponding to the positive sample image, comprising: The positive sample image is data enhanced to obtain a data enhanced positive sample image; At least one target cropped image is cropped from the data enhanced positive sample image, and the target is included in the target cropped image; A first target cropped image is obtained from the at least one target cropped image; The first target cropped image is fused into the target difficult image sample corresponding to the positive sample image to obtain the fusion image sample corresponding to the positive sample image.

4. The method of claim 2, wherein, The fusion image sample corresponding to the positive sample image is obtained by fusing the positive sample image and the target difficult image sample corresponding to the positive sample image, comprising: The positive sample image is data enhanced to obtain a data enhanced positive sample image; At least one target cropped image is cropped from the data enhanced positive sample image, and the target is included in the target cropped image; At least one non-target cropped image is cropped from the target difficult image sample corresponding to the positive sample image, and the non-target is included in the non-target cropped image; A first target cropped image is obtained from the at least one target cropped image, and a first non-target cropped image is obtained from the at least one non-target cropped image; The first target cropped image and the first non-target cropped image are spliced to obtain the fusion image sample corresponding to the positive sample image.

5. The method of claim 1, wherein, The positive sample image set under the at least one scene and the difficult image sample set under the difficult image set are sampled from the image sub-set under the at least one scene and the difficult image set, comprising: Sampling weights corresponding to the image sub-set under the at least one scene and the difficult image set are obtained; In each round of sampling process, based on the sampling weights corresponding to the image sub-set under the at least one scene and the difficult image set, a target set is determined from the image sub-set under the at least one scene and the difficult image set; According to the sampling number corresponding to the target set, sampling is performed in the target set to obtain a sample set corresponding to the target set.

6. The method of claim 1, wherein, In each grid corresponding to the fusion image sample in the fusion image sample set, there is a target grid that simultaneously includes the pixels corresponding to the target and the pixels corresponding to the non-target; each grid corresponds to a plurality of detection boxes.

7. The method of claim 6, wherein, The second training loss comprises a first sub-loss and a second sub-loss, the first sub-loss corresponds to a detection confidence of the target, and the second sub-loss corresponds to a detection confidence of the non-target; A weight parameter of the second sub-loss corresponding to the target detection model of the second stage is greater than a weight parameter of the second sub-loss corresponding to the target detection model of the first stage. 8.A device for training a target detection model, characterized in that, The device comprises: a data set acquisition module configured to acquire a training data set, the training data set comprising a sub-set of images in at least one scene; a difficult image acquisition module configured to detect each image in the training data set by a target detection model of a first stage to obtain a difficult image set, wherein the target detection model of the first stage is trained based on the training data set, and a difficult image in the difficult image set is an image that is detected by the target detection model of the first stage as including a target but actually not including the target; a sample image sampling module configured to sample a positive sample image set in the at least one scene and a difficult image sample set in the difficult image set from the sub-set of images in the at least one scene and the difficult image set, wherein a positive sample image in the positive sample image set includes the target; a fusion image acquisition module configured to obtain a fusion image sample set based on a positive sample image in the positive sample image set in the at least one scene and a difficult image sample in the difficult image sample set, wherein a fusion image sample in the fusion image sample set includes a target corresponding to the positive sample image and a non-target corresponding to the difficult image sample. The detection model training module is configured to, for a target sample image in the set of positive sample images, the set of difficult image samples, and the set of fused image samples under the at least one scene, obtain a detection result corresponding to the target sample image by the target detection model of the first stage; the detection result includes a predicted bounding box corresponding to each detection box of the target sample image, a detection confidence, and a predicted class; based on each predicted bounding box of the target sample image and the detection boxes, a first training loss is determined, the first training loss indicating a prediction accuracy of the target detection model of the first stage for a position and a size of the target; based on each detection confidence of the target sample image and label information of the target in the target sample image, a second training loss is determined, the second training loss indicating a prediction accuracy of the target detection model of the first stage for whether the target sample image includes the target; based on each predicted class of the target sample image and a real class corresponding to each detection box, a third training loss is determined, the third training loss indicating a prediction accuracy of the target detection model of the first stage for a class corresponding to the detection box; based on the first training loss, the second training loss, and the third training loss corresponding to each sample image in the set of positive sample images, the set of difficult image samples, and the set of fused image samples under the at least one scene, the target detection model of the first stage is trained to obtain a target detection model of the second stage.

9. A computer device, comprising: The computer device includes a processor and a memory, and the memory stores a computer program, which is loaded and executed by the processor to implement the target detection model training method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, which is loaded and executed by the processor to implement the target detection model training method according to any one of claims 1 to 7.

11. A computer program product, characterised in that, The computer program product includes computer instructions, which are executed by the processor to implement the target detection model training method according to any one of claims 1 to 7.

Images