A method and apparatus for training a model for object detection

Abstract

This disclosure relates to the field of artificial intelligence technology, and provides a training method, apparatus, computer device, and computer-readable storage medium for a model used in object detection. The method can identify different sample types by utilizing sample images in each group of training image samples, the corresponding real target object images, and the corresponding real category labels of the real target object images. This allows for setting different loss function weight values ​​based on different sample types, fully utilizing the potential of training image samples of sample types that have a significant impact on model training performance, and reducing the impact of training image samples of sample types that have no or poor impact on model training performance on the training of object detection and classification / regression models. This makes the training process of object detection and classification / regression models more comprehensive, improving the training efficiency and effectiveness of object detection and classification / regression models.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of artificial intelligence, and in particular, to a training method and device for a model for object detection. BACKGROUND

[0002] With the development of technology, using computer vision in the retail industry has become one of the most important technology trends. Physical retailers use computer vision algorithms in combination with store cameras to understand their customers' customer types and behavioral information. At present, although existing algorithms can recognize faces and determine human features such as gender or age range, retailers need more to track customers' movements in the store by using a graph detection model in computer vision technology, so that the navigation route, walking pattern, and store attention time can be analyzed according to the customer's movement route.

[0003] There are some common problems in the current graph detection model. The training samples of the current graph detection model are given by the Selective Search algorithm, and these training samples contain "background" and "other objects", while the object detection task of the graph detection model is to detect "people" in the picture. In actual detection tasks, the popularity of offline shopping malls has declined significantly, and there will be many actual images with only one or two people. Although the Selective Search algorithm generates about 1000-2000 person training sample images, the training sample images are not generated specifically for the "person" category; in this case, the data set (i.e. the training sample image) is easy to have the number of positive samples less than the number of negative samples. In this way, it will lead to the graph detection model trained based on the above data set has poor generalization ability, thereby leading to the decline of the precision of the graph detection model and the recall rate of the positive sample; in this case, this kind of graph detection model needs to oversample the positive sample or undersample the negative sample, etc., thereby leading to a relatively complex and difficult training process. SUMMARY

[0004] Therefore, the embodiments of the present disclosure provide a training method and device for a model for object detection, a computer device, and a computer readable storage medium, to solve the problem that the data set of the training sample images generated by the Selective Search algorithm in the prior art is easy to have the number of positive samples less than the number of negative samples, which will lead to the graph detection model trained based on the above data set has poor generalization ability, thereby leading to the decline of the precision of the graph detection model and the recall rate of the positive sample, and further leading to a relatively complex and difficult training process.

[0005] In a first aspect, the embodiments of the present disclosure provide a training method for a model for object detection, comprising:

[0006] Obtain a training image sample set; wherein, the training image sample set includes several training image samples, and each set of training image samples includes a sample image, a real target object image corresponding to the sample image, and a real category label corresponding to the real target object image;

[0007] For each training round of each set of training image samples, the sample images in the training image samples are input into a preset target detection model to obtain several candidate target image boxes corresponding to the sample images; the several candidate target image boxes corresponding to the sample images are input into a preset classification and regression model to obtain the predicted target object image and the predicted category label corresponding to the predicted target object image; based on the predicted target object image and the predicted category label corresponding to the predicted target object image, as well as the real target object image and the real category label corresponding to the sample image, the loss function value of the training image sample in this round is determined;

[0008] Based on the loss function value of each group of training image samples in each of the N training rounds, determine the sample type of each group of training image samples;

[0009] The target detection model and the classification regression model are trained using the training image sample set, the sample type of each training image sample in the training image sample set, and the loss function weight value corresponding to the preset sample type, to obtain the trained target detection model and classification regression model.

[0010] A second aspect of this disclosure provides a training apparatus for a model of object detection, the apparatus comprising:

[0011] A set acquisition unit is used to acquire a training image sample set; wherein, the training image sample set includes several training image samples, and each set of training image samples includes a sample image, a real target object image corresponding to the sample image, and a real category label corresponding to the real target object image;

[0012] The numerical determination unit is used for each round of training for each set of training image samples. It inputs the sample images from the training image samples into a preset target detection model to obtain several candidate target image boxes corresponding to the sample images; inputs the several candidate target image boxes corresponding to the sample images into a preset classification and regression model to obtain the predicted target object image and the predicted category label corresponding to the predicted target object image; and determines the loss function value of the training image sample for this round based on the predicted target object image and the predicted category label corresponding to the predicted target object image, as well as the real target object image and the real category label corresponding to the real target object image.

[0013] The type determination unit is used to determine the sample type of each group of training image samples based on the loss function value of each group of training image samples in each of the N rounds of training.

[0014] The model training unit is used to train the object detection model and the classification regression model using the training image sample set, the sample type of each training image sample in the training image sample set, and the loss function weight value corresponding to the preset sample type, so as to obtain the trained object detection model and classification regression model.

[0015] A third aspect of this disclosure provides a computer device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method described above.

[0016] A fourth aspect of this disclosure provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the above-described method.

[0017] The beneficial effects of this disclosure embodiment compared with the prior art are as follows: This disclosure embodiment can first obtain a training image sample set; wherein, the training image sample set includes several training image samples, each group of training image samples includes a sample image, a real target object image corresponding to the sample image, and a real category label corresponding to the real target object image. Then, for each round of training for each group of training image samples, the sample image in the training image sample can be input into a preset target detection model to obtain several candidate target image boxes corresponding to the sample image; the several candidate target image boxes corresponding to the sample image can be input into a preset classification regression model to obtain the predicted target object image corresponding to the sample image and the predicted category label corresponding to the predicted target object image; based on the predicted target object image corresponding to the sample image and the predicted category label corresponding to the predicted target object image, and the real target object image corresponding to the sample image and the real category label corresponding to the real target object image, the loss function value of the training image sample in this round is determined. Next, based on the loss function value of each group of training image samples in each round of training in the N rounds of training, the sample type of each group of training image samples can be determined. Finally, the target detection model and the classification regression model can be trained using the training image sample set, the sample type of each training image sample in the training image sample set, and the loss function weight value corresponding to the preset sample type, to obtain the trained target detection model and classification regression model.As can be seen, in this embodiment, the loss function value of each training image sample in each training image sample set, along with the corresponding real target object image and the corresponding real category label, is first determined. Then, based on the loss function value of each training image sample in each training round, the training image samples are classified to determine the sample type of each training image sample set. This allows for a thorough global analysis and mining of the training image samples. By classifying the loss function values ​​of the training image samples during the first stage of training (i.e., N rounds of training), different sample types (e.g., simple samples, ordinary samples, and difficult samples) are identified. Since different sample types of training image samples have different degrees of influence on the training effect of the object detection model and the classification / regression model (e.g., affecting the generalization ability of the object detection model and the classification / regression model), different loss function weight values ​​can be set according to different sample types. This allows for the utilization of the training image sample set, the sample type of each training image sample in the training image sample set, and the pre-... By defining the loss function weights corresponding to different sample types, during the training of the object detection model and the classification regression model, the loss function weights of training image samples of sample types that do not affect the model training effect can be reduced, while the loss function weights of training image samples of sample types that have a greater impact on the model training effect can be increased. This fully utilizes the potential of training image samples of sample types that have a greater impact on the model training effect, and reduces the impact of training image samples of sample types that have no impact or a poor impact on the model training effect on the training of the object detection model and the classification regression model. It can be seen that this embodiment can effectively improve the distribution of training weights (i.e., loss function weights) of training image samples of different sample types, thereby making the training process of the object detection model and the classification regression model more thorough, improving the training efficiency and effect of the object detection model and the classification regression model, and thus improving the performance of the object detection model and the classification regression model, and further improving the detection effect of the object detection model and the classification regression model in actual business scenarios (such as improving the object detection accuracy of the object detection model and the classification regression model). Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of this disclosure, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a schematic diagram illustrating an application scenario of an embodiment of this disclosure;

[0020] Figure 2 This is a flowchart of a training method for a model for object detection provided in an embodiment of this disclosure;

[0021] Figure 3 This is a block diagram of a training apparatus for a model for object detection provided in an embodiment of this disclosure;

[0022] Figure 4 This is a schematic diagram of a computer device provided in an embodiment of this disclosure. Detailed Implementation

[0023] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, so as to provide a thorough understanding of the embodiments of this disclosure. However, those skilled in the art will understand that this disclosure may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this disclosure with unnecessary detail.

[0024] The following will describe in detail, with reference to the accompanying drawings, a training method and apparatus for a model for object detection according to embodiments of the present disclosure.

[0025] In existing technologies, traditional graph detection models suffer from several common problems. The training samples for these models are provided by the Selective Search algorithm, which include both "background" and "other objects," while the target detection task of graph detection models is to detect "people" within images. In real-world detection tasks, the popularity of offline shopping malls often declines significantly, resulting in many images containing only one or two people. Although the Selective Search algorithm generates approximately 1000-2000 training sample images, these images are not specifically generated for the "people" category. In such datasets (i.e., training sample images), it's easy for the number of positive samples to be less than the number of negative samples. This leads to poor generalization ability in graph detection models trained on such datasets, resulting in decreased accuracy and recall of positive samples. In such cases, these graph detection models require oversampling of positive samples or undersampling of negative samples, making the training process relatively complex and difficult.

[0026] To address the aforementioned problems, this invention provides a training method for a model used in object detection. In this method, the loss function value for each training image sample in each training image set is determined using the sample images, the corresponding real target object images, and the corresponding real category labels. Then, based on the loss function value in each training image set, the training image samples are classified to determine their sample type. This allows for a thorough global analysis and mining of the training image samples. By classifying the loss function values ​​of the training image samples during the first training stage (i.e., N rounds of training), different sample types (e.g., simple samples, ordinary samples, and difficult samples) are identified. Since different sample types have varying degrees of influence on the training effect of the object detection model and the classification / regression model (e.g., affecting the generalization ability of the object detection model and the classification / regression model), different loss function weights can be set according to different sample types to optimize the training image sample set. The sample types and corresponding loss function weights for each training image sample are used to train the object detection model and classification regression model. This reduces the loss function weights of training image samples of sample types that do not affect the model's training effect, while increasing the loss function weights of training image samples of sample types that significantly impact the model's training effect. This fully utilizes the potential of training image samples of sample types that have a significant impact on the model's training effect, and reduces the impact of training image samples of sample types that have no or poor impact on the training effect of the object detection model and classification regression model. Therefore, this embodiment can effectively improve the distribution of training weights (i.e., loss function weights) of training image samples of different sample types, thereby making the training process of the object detection model and classification regression model more thorough. This improves the training efficiency and effect of the object detection model and classification regression model, thus enhancing their performance and further improving their detection effect in actual business scenarios (e.g., improving the object detection accuracy of the object detection model and classification regression model).

[0027] For example, embodiments of the present invention can be applied to, for example... Figure 1 The application scenario shown can include terminal device 1 and server 2.

[0028] Terminal device 1 can be hardware or software. When terminal device 1 is hardware, it can be various electronic devices with a display screen that support communication with server 2, including but not limited to smartphones, tablets, laptops, and desktop computers; when terminal device 1 is software, it can be installed in the aforementioned electronic devices. Terminal device 1 can be implemented as multiple software programs or software modules, or as a single software program or software module, and this disclosure does not limit this. Furthermore, various applications can be installed on terminal device 1, such as data processing applications, instant messaging tools, social platform software, search applications, shopping applications, etc.

[0029] Server 2 can be a server that provides various services, such as a backend server that receives requests sent by terminal devices with which it has established communication connections. This backend server can receive and analyze the requests sent by the terminal devices and generate processing results. Server 2 can be a single server, a server cluster consisting of several servers, or a cloud computing service center; this disclosure does not limit this.

[0030] It should be noted that server 2 can be either hardware or software. When server 2 is hardware, it can be various electronic devices that provide various services to terminal device 1. When server 2 is software, it can be multiple software programs or software modules that provide various services to terminal device 1, or it can be a single software program or software module that provides various services to terminal device 1. This disclosure does not impose any limitations on this aspect.

[0031] Terminal device 1 and server 2 can communicate via a network. The network can be a wired network using coaxial cable, twisted pair, or fiber optic connection, or a wireless network that enables interconnection of various communication devices without wiring, such as Bluetooth, Near Field Communication (NFC), or Infrared. This disclosure does not limit the scope of the embodiments.

[0032] Specifically, the user can input a training image sample set through terminal device 1; terminal device 1 then sends the training image sample set to server 2. Server 2 stores target detection models and classification regression models to be trained. Server 2 can first train each round of training for each set of training image samples by inputting the sample images from the training image samples into a preset target detection model to obtain several candidate target image boxes corresponding to the sample images. Then, it inputs these candidate target image boxes into a preset classification regression model to obtain the predicted target object image and the predicted category label corresponding to the predicted target object image. Based on the predicted target object image and the predicted category label corresponding to the sample image, as well as the real target object image and the real category label corresponding to the sample image, it determines the loss function value for the current round of training for each training image sample. Then, Server 2 can determine the sample type of each set of training image samples based on the loss function values ​​for each of the N rounds of training for each set of training image samples. Next, Server 2 can use the training image sample set, the sample type of each training image sample in the training image sample set, and the preset loss function weight values ​​corresponding to the sample type to train the target detection model and the classification regression model, obtaining the trained target detection model and classification regression model.In this way, since this application can first use the sample images in each group of training image samples, the corresponding real target object images, and the corresponding real category labels of the real target object images to determine the loss function value of each group of training image samples in each round of training, and then classify each group of training image samples according to the loss function value of each group of training image samples in each round of training to determine the sample type of the group of training image samples; in this way, the training image samples are fully analyzed and mined globally. By classifying the loss function values ​​of the training image samples in the first stage of training (i.e., N rounds of training), different sample types (such as simple samples, ordinary samples, and difficult samples) are identified. Since the degree of influence of different sample types of training image samples on the training effect of the object detection model and the classification and regression model (such as the influence on the generalization ability of the object detection model and the classification and regression model) is different, different loss function weight values ​​can be set according to different sample types, so as to utilize the training image sample set, the sample type of each group of training image samples in the training image sample set, and the loss function weight value of each group of training image samples in the training image sample set. By pre-setting loss function weight values ​​corresponding to different sample types, the training process of the object detection model and classification regression model can reduce the loss function weight values ​​of training image samples of sample types that do not affect the model training effect, and increase the loss function weight values ​​of training image samples of sample types that have a greater impact on the model training effect. This fully utilizes the potential of training image samples of sample types that have a greater impact on the model training effect, and reduces the impact of training image samples of sample types that have no or poor impact on the model training effect on the training of the object detection model and classification regression model. It can be seen that this embodiment can effectively improve the distribution of training weights (i.e., loss function weight values) of training image samples of different sample types, thereby making the training process of the object detection model and classification regression model more thorough, improving the training efficiency and effect of the object detection model and classification regression model, and thus improving the performance of the object detection model and classification regression model, and further improving the detection effect of the object detection model and classification regression model in actual business scenarios (such as improving the object detection accuracy of the object detection model and classification regression model).

[0033] It should be noted that the specific types, quantities, and combinations of terminal device 1, server 2, and network can be adjusted according to the actual needs of the application scenario, and this disclosure embodiment does not impose any restrictions on this.

[0034] It should be noted that the above application scenarios are shown only for the purpose of understanding this disclosure, and the implementation of this disclosure is not limited in any way. On the contrary, the implementation of this disclosure can be applied to any applicable scenario.

[0035] Figure 2This is a flowchart of a training method for a model for object detection provided in an embodiment of this disclosure. Figure 2 A training method for a model used in object detection can be derived from... Figure 1 The terminal device or server executes the command. For example... Figure 2 As shown, the training method for this model used for object detection includes:

[0036] S201: Obtain the training image sample set.

[0037] The training image sample set includes several training image samples. Each training image sample set includes a sample image, a real target object image corresponding to the sample image, and a real category label corresponding to the real target object image.

[0038] In this embodiment, the training image sample set may include several training image samples. Each set of training recommended samples includes a sample image, a corresponding real target object image, and a corresponding real category label. The sample image can be understood as the image to be recognized, the corresponding real target object image can be understood as the image region in the sample image that contains only the target object, and the corresponding real category label can be understood as the actual recognition result for the real target object image.

[0039] For example, assuming the target corresponds to a "human face," the real target image corresponding to the sample image can be the image regions corresponding to all faces in the sample image; that is, image patches containing only human faces are cropped from the sample image. The real category label corresponding to the real target image can be understood as the user information corresponding to the human face in the real target image. Similarly, assuming the target corresponds to an "animal," the real target image corresponding to the sample image can be the image regions corresponding to all animals in the sample image; that is, image patches containing only animals are cropped from the sample image. The real category label corresponding to the real target image can be understood as the species type of the animal in the real target image.

[0040] S202: For each round of training for each set of training image samples, input the sample images from the training image samples into a preset target detection model to obtain several candidate target image boxes corresponding to the sample images; input the several candidate target image boxes corresponding to the sample images into a preset classification and regression model to obtain the predicted target object image corresponding to the sample images and the predicted category label corresponding to the predicted target object image; determine the loss function value of the training image samples for this round based on the predicted target object image corresponding to the sample images and the predicted category label corresponding to the predicted target object image, as well as the real target object image corresponding to the sample images and the real category label corresponding to the real target object image.

[0041] After obtaining the training image sample set, each group of training image samples in the set can be used to train the preset object detection model and classification regression model for N rounds, obtaining the loss function value of each group of training image samples in each round of training. It can be understood that since a set of training image samples is used to train the preset object detection model and classification regression model for N rounds, N loss function values ​​corresponding to that set of training image samples can be obtained. If the training image sample set includes Z groups of training image samples, then Z*N loss function values ​​can be obtained. It should be noted that N can be a positive integer greater than 1, and N rounds can be a preset number of rounds based on actual needs, or it can be the number of training rounds that causes slight overfitting in the object detection model and classification regression model (i.e., the loss function value of the training image sample set decreases, but the loss function value of the validation set begins to increase). For example, assuming that the object detection model and classification regression model are trained using the training image sample set, and slight overfitting occurs in the object detection model and classification regression model after the 100th round of training, then N can be set to 100. It should be noted that, in one implementation, the object detection model can be a convolutional neural network, such as the CNN series of networks, specifically Fast R-CNN (Fast Region-CNN) or SSD (Single Shot MultiBox Detector, which uses a single deep neural network to detect objects in an image); the classification and regression model can be a self-attention network, such as the ResNet series.

[0042] Specifically, for each training round of training for each group of training image samples in the training image sample set, the sample image in the training image sample is input into a preset target detection model to obtain several candidate target image boxes corresponding to the sample image; that is, the sample image is input into the target detection model, and the target detection model performs operations such as convolution / pooling on the sample image to extract the image features of the sample image, and calculates the image features of the sample image through bounding-box regression to obtain several candidate target image boxes corresponding to the sample image.

[0043] Then, several candidate target image boxes corresponding to the sample image can be input into a preset classification and regression model to obtain the predicted target object image corresponding to the sample image and the predicted category label corresponding to the predicted target object image. It can be understood that the classification and regression model can perform classification and boundary-box regression; that is, firstly, it determines the final predicted target object image through boundary-box regression, and secondly, it classifies the predicted target object image from the previous step to obtain the predicted category label.

[0044] Next, the loss function value for the current round of training image samples can be determined based on the predicted target object image and the predicted category label corresponding to the sample image, as well as the real target object image and the real category label corresponding to the sample image. For example, in one implementation, the loss functions of the object detection model and the classification regression model are multi-class cross-entropy loss function and logistic regression loss function, respectively. The loss function values ​​of the predicted target object image and the predicted category label corresponding to the sample image, as well as the real target object image and the real category label corresponding to the sample image, can be calculated using the loss function values ​​of the sample image and the predicted category label corresponding to the sample image, and the real category label corresponding to the sample image. These loss function values ​​are then used as the loss function values ​​for the current round of training of the training image samples.

[0045] S203: Determine the sample type of each group of training image samples based on the loss function value of each group of training image samples in each of the N rounds of training.

[0046] Since different types of training image samples have varying degrees of impact on the training performance of object detection and classification / regression models (e.g., affecting their generalization ability), it is necessary to classify the training image samples in the training image sample set. This allows for different training methods for the object detection and classification / regression models based on different sample types. In this embodiment, the loss function value is used as an indicator to adaptively and efficiently classify the training image samples, avoiding cumbersome data analysis and business knowledge intervention. This method is efficient, widely adaptable, and can be well adapted to other scenarios.

[0047] Specifically, in this embodiment, for each group of training image samples in the training image sample set, the sample type of that group of training image samples can be determined based on the loss function values ​​of that group of training image samples in each round of training. In other words, in this embodiment, after performing a global and thorough analysis and mining of the group of training image samples, the loss function values ​​of that group of training image samples in each round of training can be obtained, and the sample type of that group of training image samples can be determined by classifying them based on the loss function values ​​of that group of training image samples in each round of training.

[0048] It should be noted that the training image samples can be divided into three types: first sample type (e.g., ordinary samples), second sample type (e.g., simple samples), and third sample type (e.g., difficult samples).

[0049] S204: Using the training image sample set, the sample type of each training image sample in the training image sample set, and the loss function weight value corresponding to the preset sample type, train the object detection model and the classification regression model to obtain the trained object detection model and classification regression model.

[0050] In this embodiment, different loss function weight values ​​can be pre-set for training image samples of different sample types during the training process. This allows the potential of the training image samples to be fully utilized, enabling the object detection model and classification / regression model to be trained more thoroughly, improving model performance, and further enhancing the object detection and classification effects of the object detection and classification / regression models in real-world business scenarios. Furthermore, it effectively identifies a large number of useless, simple samples, thereby reducing some...

[0051] The simple sample training weights that do not affect the model training effect, that is, during the training of the object detection model and the classification regression model, can reduce the loss function weight values ​​of training image samples of sample types that do not affect the model training effect, and increase the loss function weight values ​​of training image samples of sample types that have a greater impact on the model training effect. This is to fully utilize the potential of training image samples of sample types that have a greater impact on the model training effect, and reduce the number of samples that have no impact or a poor impact on the model training effect.

[0052] The impact of different types of training image samples on the training of object detection models and classification / regression models can be studied to effectively improve the training weight distribution of simple and difficult samples, thereby improving the efficiency and effectiveness of model training.

[0053] In this embodiment, since the third sample type (such as hard samples) has a significant impact on the model training effect, the loss function weight value of the training image samples of the third sample type can be increased.

[0054] This means that the weight value of the loss function corresponding to training image samples of the third sample type is greater than the weight value of the loss function corresponding to training image samples of the first sample type. For example, the weight value of the loss function for training image samples of the third sample type is between 2 and 5 times that of training image samples of the first sample type. Because the second sample type (e.g., simple samples) has a relatively small impact on model training performance, for example, it has no benefit for training object detection models or classification / regression models, and...

[0055] This can lead to overfitting of the model and reduced online generalization ability. Therefore, the loss function weights for the 5 training image samples of the second sample type can be reduced. In other words, the training samples of the second sample type can be used instead of the second sample type.

[0056] The weight value of the loss function corresponding to the image sample is less than the weight value of the loss function corresponding to the training image sample of the first sample type. For example, the weight value of the loss function for the training image sample of the first sample type is between 2 and 5 times the weight value of the loss function for the training image sample of the second sample type.

[0057] In this implementation, after determining the sample type of each group of training image samples in the training image sample set, the target detection model and classification regression model can be trained using the training image sample set, the sample type of each group of training image samples in the training image sample set, and the preset loss function weight values ​​corresponding to the sample types, to obtain the trained target detection model and classification regression model. That is, the target detection model and classification regression model are trained using the training image samples in the training image sample set. During the process of adjusting the model parameters of the target detection model and classification regression model based on the loss function values, the loss function values ​​need to be calculated based on the sample type of each group of training image samples in the training image sample set, the preset loss function weight values ​​corresponding to the sample types, and the preset loss function. Training continues until the loss function values ​​of the target detection model and classification regression model meet preset conditions, or the number of training iterations reaches a preset number, to obtain the trained target detection model and classification regression model.

[0058] Next, an example will be given to illustrate how, in S204, the loss function value is calculated based on the sample type of each group of training image samples in the training image sample set, the weight value of the loss function corresponding to the preset sample type, and the preset loss function. For example, the training image sample set includes four training image samples: the first training image sample (including sample image s1), the second training image sample (including sample image s2), the third training image sample (including sample image s3), and the fourth training image sample (including sample image s4); assuming that the sample types of the first training text sample and the second training text sample are hard samples (i.e., the third sample class)... The sample types of the third and fourth training text samples are ordinary samples (i.e., the first sample type). The real target object images corresponding to s1, s2, s3, and s4, and the real category labels corresponding to the real target object images, are l1, l2, l3, and l4, respectively. The loss function weight of the third sample type is w times that of the first sample type. The neural network (i.e., the object detection model and the classification and regression model) is f(), and the loss function is loss(). Then, in this gradient calculation, the loss function value of the object detection model and the classification and regression model after the weights are redistributed is Loss(). 平均 The calculation formula is: Loss 平均 =w*(loss(l1, f(s1)+loss(l2, f(s2))+loss(l3, f(s3)+loss(l4, f(s4)). Then, proceed with the model training as normal.

[0059] As can be seen, the embodiments of this disclosure can first obtain a training image sample set; wherein, the training image sample set includes several training image samples, each group of training image samples includes a sample image, a real target object image corresponding to the sample image, and a real category label corresponding to the real target object image. Then, for each round of training for each group of training image samples, the sample images in the training image samples are input into a preset target detection model to obtain several candidate target image boxes corresponding to the sample images; the several candidate target image boxes corresponding to the sample images are input into a preset classification regression model to obtain the predicted target object image corresponding to the sample image and the predicted category label corresponding to the predicted target object image; based on the predicted target object image corresponding to the sample image and the predicted category label corresponding to the predicted target object image, and the real target object image corresponding to the sample image and the real category label corresponding to the real target object image, the loss function value of the training image sample in this round is determined. Next, based on the loss function value of each group of training image samples in each round of training in the N rounds of training, the sample type of each group of training image samples can be determined. Finally, the target detection model and the classification regression model can be trained using the training image sample set, the sample type of each training image sample in the training image sample set, and the loss function weight value corresponding to the preset sample type, to obtain the trained target detection model and classification regression model.As can be seen, in this embodiment, the loss function value of each training image sample in each training image sample set, along with the corresponding real target object image and the corresponding real category label, is first determined. Then, based on the loss function value of each training image sample in each training round, the training image samples are classified to determine the sample type of each training image sample set. This allows for a thorough global analysis and mining of the training image samples. By classifying the loss function values ​​of the training image samples during the first stage of training (i.e., N rounds of training), different sample types (e.g., simple samples, ordinary samples, and difficult samples) are identified. Since different sample types of training image samples have different degrees of influence on the training effect of the object detection model and the classification / regression model (e.g., affecting the generalization ability of the object detection model and the classification / regression model), different loss function weight values ​​can be set according to different sample types. This allows for the utilization of the training image sample set, the sample type of each training image sample in the training image sample set, and the pre-... By defining the loss function weights corresponding to different sample types, during the training of the object detection model and the classification regression model, the loss function weights of training image samples of sample types that do not affect the model training effect can be reduced, while the loss function weights of training image samples of sample types that have a greater impact on the model training effect can be increased. This fully utilizes the potential of training image samples of sample types that have a greater impact on the model training effect, and reduces the impact of training image samples of sample types that have no impact or a poor impact on the model training effect on the training of the object detection model and the classification regression model. It can be seen that this embodiment can effectively improve the distribution of training weights (i.e., loss function weights) of training image samples of different sample types, thereby making the training process of the object detection model and the classification regression model more thorough, improving the training efficiency and effect of the object detection model and the classification regression model, and thus improving the performance of the object detection model and the classification regression model, and further improving the detection effect of the object detection model and the classification regression model in actual business scenarios (such as improving the object detection accuracy of the object detection model and the classification regression model).

[0060] In some embodiments, step S203, "determining the sample type of each group of training image samples based on the loss function values ​​of each group of training image samples in each of the N rounds of training," may include the following steps:

[0061] S203a: For each group of training image samples in the training image sample set, determine the degree of loss reduction of the training image samples throughout the training process based on the loss function value of the training image samples in each round of training.

[0062] In one implementation of this embodiment, the target detection model and the classification regression model can be trained M times using the training image sample set. That is, each round of training will use all the training image samples in the training image sample set to train the target detection model and the classification regression model once.

[0063] First, based on the loss function values ​​of the training image samples in each training round, determine the average loss function value for the first M rounds of training and the average loss function value for the next X rounds of training, where M and X are both positive integers and both M and X are less than N. This can be understood as first determining the average loss function value of the training image samples in the first M rounds of training based on their loss function values, and then determining the average loss function value of the training image samples in the next X rounds of training based on their loss function values ​​in each of the next X rounds of training. For example, the average loss function value of the training image samples in the first 10% of rounds can be calculated based on all the loss function values ​​of the training image samples in the first 10% of rounds, and the average loss function value of the training image samples in the next 10% of rounds can be calculated based on all the loss function values ​​of the training image samples in the next 10% of rounds.

[0064] Then, the overall training loss reduction rate of the training image samples can be determined based on the average loss function values ​​of the first M training rounds and the loss function values ​​of the subsequent X training rounds. That is, the overall training loss reduction rate of the training image samples can be determined based on the average loss function values ​​of the training image samples in the first M training rounds and the average loss function values ​​of the training image samples in the subsequent X training rounds. In one implementation, the overall training loss reduction rate of the training image samples can be calculated using the following formula: c = (ab) / a; where c is the overall training loss reduction rate of the training image samples, a is the average loss function value of the training image samples in the first M training rounds, and b is the average loss function value of the training image samples in the subsequent X training rounds.

[0065] S203b: Determine the sample type of each group of training image samples based on the overall training loss reduction rate of each group of training image samples in the training image sample set.

[0066] After determining the overall training loss descent rate of each training image sample in the training image sample set, the overall training loss descent rates of each training image sample in the training image sample set can be sorted from high to low to obtain a sorting result. Then, the sample type of the training image samples in the first n positions (e.g., in the first 30%) in the sorting result can be determined as the first sample type (e.g., normal samples); the sample type of the training image samples in the last n positions (e.g., in the last 30%) in the sorting result can be determined as the second sample type (e.g., easy samples); and the sample type of the training image samples in the sorting result that are not in the first n positions or the last n positions (e.g., after the first 30% and before the last 30%) can be determined as the third sample type (e.g., hard samples).

[0067] For example, suppose the training image sample set includes 100 training image samples. The training loss descent rates of each training image sample in the set are sorted from highest to lowest to obtain the sorting result. The training image samples ranked from the first to the thirtieth position can be classified as ordinary samples, the training image samples ranked from the thirty-first to the seventieth position can be classified as difficult samples, and the training image samples ranked from the seventy-first to the hundredth position can be classified as easy samples.

[0068] All of the above-mentioned optional technical solutions can be combined in any way to form optional embodiments of this disclosure, and will not be described in detail here.

[0069] The following are embodiments of the apparatus disclosed herein, which can be used to execute embodiments of the method disclosed herein. For details not disclosed in the apparatus embodiments of this disclosure, please refer to the embodiments of the method disclosed herein.

[0070] Figure 3 This is a schematic diagram of a training apparatus for a model used for object detection provided in an embodiment of this disclosure. Figure 3 As shown, the training apparatus for the object detection model includes:

[0071] The set acquisition unit 301 is used to acquire a training image sample set; wherein, the training image sample set includes a number of training image samples, and each set of training image samples includes a sample image, a real target object image corresponding to the sample image, and a real category label corresponding to the real target object image;

[0072] The numerical determination unit 302 is used for each round of training for each set of training image samples, inputting the sample images in the training image samples into a preset target detection model to obtain a plurality of candidate target image boxes corresponding to the sample images; inputting the plurality of candidate target image boxes corresponding to the sample images into a preset classification regression model to obtain the predicted target object image corresponding to the sample images and the predicted category label corresponding to the predicted target object image; and determining the loss function value of the training image sample in this round based on the predicted target object image corresponding to the sample images and the predicted category label corresponding to the predicted target object image, as well as the real target object image corresponding to the sample images and the real category label corresponding to the real target object image.

[0073] The type determination unit 303 is used to determine the sample type of each group of training image samples based on the loss function value of each group of training image samples in each of the N rounds of training.

[0074] The model training unit 304 is used to train the target detection model and the classification regression model using the training image sample set, the sample type of each training image sample in the training image sample set, and the loss function weight value corresponding to the preset sample type, so as to obtain the trained target detection model and classification regression model.

[0075] Optionally, the target detection model is a convolutional neural network.

[0076] Optionally, the classification regression model is a self-attention network.

[0077] Optionally, the type determination unit 303 is used for:

[0078] For each group of training image samples in the training image sample set, the degree of loss reduction of the training image sample throughout the training process is determined based on the loss function value of the training image sample in each round of training.

[0079] The sample type of each group of training image samples is determined based on the degree of loss reduction throughout the training process of each group of training image samples in the training image sample set.

[0080] Optionally, the type determination unit 303 is used for:

[0081] Based on the loss function values ​​of the training image samples in each round of training, determine the average loss function value of the first M rounds of training and the loss function value of the next X rounds of training, where M and X are both less than N;

[0082] The degree of loss reduction of the training image samples throughout the training process is determined based on the average loss function value of the first M training rounds and the loss function value of the subsequent X training rounds.

[0083] Optionally, the type determination unit 303 is used for:

[0084] The training image sample sets are sorted from high to low based on the overall training loss reduction rate of each group of training image samples to obtain the sorting result;

[0085] The sample type of the training image sample located in the top n positions of the sorting result is determined as the first sample type;

[0086] The sample type of the training image sample located in the last n positions of the sorting result is determined as the second sample type;

[0087] The training image samples that are not in the first n or last n positions in the sorting results are determined as the third sample type.

[0088] Optionally, the weight value of the loss function corresponding to the training image sample of the third sample type is greater than the weight value of the loss function corresponding to the training image sample of the first sample type; the weight value of the loss function corresponding to the training image sample of the second sample type is less than the weight value of the loss function corresponding to the training image sample of the first sample type.

[0089] Optionally, the loss function of the target detection model and the classification regression model is a multi-class cross-entropy loss function or a logistic regression loss function.

[0090] The beneficial effects of this disclosure compared to the prior art are as follows: This disclosure provides a training apparatus for a model used in object detection. The apparatus includes: a set acquisition unit for acquiring a training image sample set; wherein the training image sample set includes several training image samples, each set of training image samples includes a sample image, a real target object image corresponding to the sample image, and a real category label corresponding to the real target object image; and a value determination unit for, for each round of training for each set of training image samples, inputting the sample images from the training image samples into a preset object detection model to obtain several candidate target image boxes corresponding to the sample images; and inputting the several candidate target image boxes corresponding to the sample images into a preset classification regression model to obtain a predicted target object image corresponding to the sample images and the predicted target object. The system includes: a predicted category label corresponding to the image; a loss function value for the current round of training image samples based on the predicted target object image and the predicted category label corresponding to the predicted target object image, as well as the real target object image and the real category label corresponding to the real target object image; a type determination unit for determining the sample type of each group of training image samples based on the loss function value of each group of training image samples in each round of training in the N rounds of training; and a model training unit for training the object detection model and the classification regression model using the training image sample set, the sample type of each training image sample in the training image sample set, and the preset loss function weight value corresponding to the sample type, to obtain the trained object detection model and classification regression model.As can be seen, in this embodiment, the loss function value of each training image sample in each training image sample set, along with the corresponding real target object image and the corresponding real category label, is first determined. Then, based on the loss function value of each training image sample in each training round, the training image samples are classified to determine the sample type of each training image sample set. This allows for a thorough global analysis and mining of the training image samples. By classifying the loss function values ​​of the training image samples during the first stage of training (i.e., N rounds of training), different sample types (e.g., simple samples, ordinary samples, and difficult samples) are identified. Since different sample types of training image samples have different degrees of influence on the training effect of the object detection model and the classification / regression model (e.g., affecting the generalization ability of the object detection model and the classification / regression model), different loss function weight values ​​can be set according to different sample types. This allows for the utilization of the training image sample set, the sample type of each training image sample in the training image sample set, and the pre-... By defining the loss function weights corresponding to different sample types, during the training of the object detection model and the classification regression model, the loss function weights of training image samples of sample types that do not affect the model training effect can be reduced, while the loss function weights of training image samples of sample types that have a greater impact on the model training effect can be increased. This fully utilizes the potential of training image samples of sample types that have a greater impact on the model training effect, and reduces the impact of training image samples of sample types that have no impact or a poor impact on the model training effect on the training of the object detection model and the classification regression model. It can be seen that this embodiment can effectively improve the distribution of training weights (i.e., loss function weights) of training image samples of different sample types, thereby making the training process of the object detection model and the classification regression model more thorough, improving the training efficiency and effect of the object detection model and the classification regression model, and thus improving the performance of the object detection model and the classification regression model, and further improving the detection effect of the object detection model and the classification regression model in actual business scenarios (such as improving the object detection accuracy of the object detection model and the classification regression model).

[0091] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this disclosure.

[0092] Figure 4 This is a schematic diagram of the computer device 4 provided in an embodiment of this disclosure. Figure 4As shown, the computer device 4 in this embodiment includes a processor 401, a memory 402, and a computer program 403 stored in the memory 402 and executable on the processor 401. When the processor 401 executes the computer program 403, it implements the steps in the various method embodiments described above. Alternatively, when the processor 401 executes the computer program 403, it implements the functions of each module / module in the various device embodiments described above.

[0093] Exemplarily, computer program 403 may be divided into one or more modules, which are stored in memory 402 and executed by processor 401 to perform the present disclosure. The one or more modules may be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of computer program 403 in computer device 4.

[0094] Computer device 4 can be a desktop computer, laptop, handheld computer, cloud server, or other similar computer device. Computer device 4 may include, but is not limited to, processor 401 and memory 402. Those skilled in the art will understand that... Figure 4 This is merely an example of computer device 4 and does not constitute a limitation on computer device 4. It may include more or fewer components than shown, or combine certain components, or different components. For example, computer device may also include input / output devices, network access devices, buses, etc.

[0095] Processor 401 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0096] The memory 402 can be an internal storage module of the computer device 4, such as a hard disk or RAM of the computer device 4. The memory 402 can also be an external storage device of the computer device 4, such as a plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card equipped on the computer device 4. Furthermore, the memory 402 can include both internal and external storage modules of the computer device 4. The memory 402 is used to store computer programs and other programs and data required by the computer device. The memory 402 can also be used to temporarily store data that has been output or will be output.

[0097] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described functional modules are used as examples. In practical applications, the functions described above can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. The functional modules in the embodiments can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated modules can be implemented in hardware or as software functional modules. Furthermore, the specific names of the functional modules are only for easy differentiation and are not intended to limit the scope of protection of this disclosure. The specific working process of the modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0098] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0099] Those skilled in the art will recognize that the modules and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this disclosure.

[0100] In the embodiments provided in this disclosure, it should be understood that the disclosed apparatus / computer devices and methods can be implemented in other ways. For example, the apparatus / computer device embodiments described above are merely illustrative. For instance, the division of modules or modules is only a logical functional division, and in actual implementation, there may be other division methods. Multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection between devices or modules may be electrical, mechanical, or other forms.

[0101] The modules described as separate components may or may not be physically separate. Similarly, the components shown as modules may or may not be physical modules; they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0102] Furthermore, the functional modules in the various embodiments of this disclosure can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0103] If an integrated module / module is implemented as a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program may include computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. A computer-readable medium may include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in a computer-readable medium may be appropriately added to or subtracted according to the requirements of legislation and patent practice in a jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media may not include electrical carrier signals and telecommunication signals.

[0104] The above embodiments are only used to illustrate the technical solutions of this disclosure, and are not intended to limit it. Although this disclosure has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this disclosure, and should all be included within the protection scope of this disclosure.

Claims

1. A training method for a model used for object detection, characterized in that, The method includes: Obtain a training image sample set; wherein, the training image sample set includes several training image samples, and each set of training image samples includes a sample image, a real target object image corresponding to the sample image, and a real category label corresponding to the real target object image; For each training round of each set of training image samples, the sample images in the training image samples are input into a preset target detection model to obtain several candidate target image boxes corresponding to the sample images; the several candidate target image boxes corresponding to the sample images are input into a preset classification and regression model to obtain the predicted target object image and the predicted category label corresponding to the predicted target object image; based on the predicted target object image and the predicted category label corresponding to the predicted target object image, as well as the real target object image and the real category label corresponding to the sample image, the loss function value of the training image sample in this round is determined; Based on the loss function value of each group of training image samples in each of the N training rounds, determine the sample type of each group of training image samples; The target detection model and the classification regression model are trained using the training image sample set, the sample type of each training image sample in the training image sample set, and the loss function weight value corresponding to the preset sample type, to obtain the trained target detection model and classification regression model. The step of determining the sample type of each group of training image samples based on the loss function value of each group of training image samples in each of the N training rounds includes: For each group of training image samples in the training image sample set, based on the loss function value of the training image sample in each round of training, determine the average loss function value of the first M rounds of training and the loss function value of the next X rounds of training, where M and X are both less than N; The degree of loss reduction of the training image samples throughout the training process is determined based on the average loss function value of the first M rounds of training and the loss function value of the last X rounds of training. The training image sample sets are sorted from high to low based on the overall training loss reduction rate of each group of training image samples to obtain the sorting result; The sample type of the training image sample located in the top n positions of the sorting result is determined as the first sample type; The sample type of the training image sample located in the last n positions of the sorting result is determined as the second sample type; The training image samples that are not in the first n or last n positions in the sorting results are determined as the third sample type.

2. The method according to claim 1, characterized in that, The target detection model is a convolutional neural network.

3. The method according to claim 1, characterized in that, The classification and regression model is a self-attention network.

4. The method according to claim 1, characterized in that, The weight value of the loss function corresponding to the training image sample of the third sample type is greater than the weight value of the loss function corresponding to the training image sample of the first sample type; the weight value of the loss function corresponding to the training image sample of the second sample type is less than the weight value of the loss function corresponding to the training image sample of the first sample type.

5. The method according to claim 1, characterized in that, The loss functions of the target detection model and the classification regression model are the multi-class cross-entropy loss function and the logistic regression loss function.

6. A training apparatus for a model used in object detection, characterized in that, The device includes: A set acquisition unit is used to acquire a training image sample set; wherein, the training image sample set includes several training image samples, and each set of training image samples includes a sample image, a real target object image corresponding to the sample image, and a real category label corresponding to the real target object image; The numerical determination unit is used for each round of training for each set of training image samples. It inputs the sample images from the training image samples into a preset target detection model to obtain several candidate target image boxes corresponding to the sample images; inputs the several candidate target image boxes corresponding to the sample images into a preset classification and regression model to obtain the predicted target object image and the predicted category label corresponding to the predicted target object image; and determines the loss function value of the training image sample for this round based on the predicted target object image and the predicted category label corresponding to the predicted target object image, as well as the real target object image and the real category label corresponding to the real target object image. The type determination unit is used to determine the sample type of each group of training image samples based on the loss function value of each group of training image samples in each of the N rounds of training. The model training unit is used to train the target detection model and the classification regression model using the training image sample set, the sample type of each training image sample in the training image sample set, and the loss function weight value corresponding to the preset sample type, so as to obtain the trained target detection model and classification regression model. The type determination unit is specifically used for: for each group of training image samples in the training image sample set, determining the average loss function value of the first M rounds of training and the loss function value of the last X rounds of training based on the loss function value of the training image samples in each round of training, where M and X are both less than N; determining the overall training loss reduction degree of the training image samples based on the average loss function value of the first M rounds of training and the loss function value of the last X rounds of training; sorting the overall training loss reduction degrees of each group of training image samples in the training image sample set from high to low to obtain a sorting result; determining the sample type of the training image samples in the first n positions of the sorting result as the first sample type; determining the sample type of the training image samples in the last n positions of the sorting result as the second sample type; and determining the sample type of the training image samples in the sorting result that are not in the first n positions or the last n positions as the third sample type.

7. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 5.

Images