Model training method, target detection method and device

Abstract

The embodiment of the application provides a model training method, a target detection method and device. In the model training stage, through the first reference bounding box, its corresponding real bounding box and real category, the to-be-trained model is prompted to continuously learn the bounding box distribution and target object category recognition, so that the prediction result output by the generated sub-model for the first reference bounding box is more real, and the target detection accuracy, generalization and data migration of the model are improved. And the loss value is determined based on the discrimination result set output by the discrimination sub-model, and then the model parameters are iteratively updated based on the loss value, wherein the discrimination result set includes the first discrimination result representing the similarity degree of the bounding box distribution and the second discrimination result representing the category similarity degree of the first prediction category and the real category, which can realize the accuracy of the target object position mark and the target object classification at the same time.

Description

Technical Field

[0001] This application relates to the field of object detection, and in particular to a model training method, an object detection method, and an apparatus. Background Technology

[0002] Currently, with the rapid development of artificial intelligence technology, there is an increasing demand for using pre-trained target detection models to detect targets in an image, thereby predicting the coordinates of the bounding boxes containing each target in the image, as well as the specific category of the target in the image region enclosed by the bounding boxes.

[0003] In the training process of object detection models in related technologies, the model loss value is mainly calculated based on the cross-entropy loss corresponding to bounding box regression and the cross-entropy loss corresponding to bounding box classification. Then, the model parameters are iteratively optimized. As a result, the accuracy of the model parameters of the trained object detection model is relatively high for a preset set of sample images, but the accuracy of the model parameters of the trained object detection model will decrease for the target image. This leads to poor generalization of the object detection model, and consequently, low accuracy of target object location labeling and target object classification when the model is applied. Summary of the Invention

[0004] The purpose of this application is to provide a model training method, a target detection method, and an apparatus that can improve the target detection accuracy, generalization, and data transferability of the trained model, thereby ensuring the accuracy of target object location labeling and target object classification during target detection.

[0005] To achieve the above technical solution, the embodiments of this application are implemented as follows:

[0006] In a first aspect, embodiments of this application provide a model training method, the method comprising:

[0007] A first specified number of first reference bounding boxes are obtained from a first candidate bounding box set, and the ground truth bounding box and the ground truth class corresponding to each first reference bounding box are obtained respectively; the first candidate bounding box set is obtained by extracting target regions from a preset sample image set using a preset region of interest extraction model;

[0008] The first reference bounding box, the ground truth bounding box, and the ground truth class are input into the model to be trained for iterative training until the current model iteration training result meets the model iteration training termination condition, thus obtaining the object detection model; wherein, the model to be trained includes a generator sub-model and a discriminator sub-model; the specific implementation method of each model training is as follows:

[0009] For each first reference bounding box: the generation sub-model makes predictions based on the first reference bounding box to obtain a first predicted bounding box and a first predicted category; the discrimination sub-model generates a discrimination result set based on the ground truth bounding box and the first predicted bounding box corresponding to the first reference bounding box, and the ground truth category and the first predicted category corresponding to the first reference bounding box; the discrimination result set includes a first discrimination result and a second discrimination result, wherein the first discrimination result represents the degree of similarity in the bounding box distribution between the first predicted bounding box and the ground truth bounding box under the condition of satisfying a preset constraint, wherein the preset constraint is that the category of the target object in the first predicted bounding box is predicted by the generation sub-model to be a target category that matches the ground truth category, and the second discrimination result represents the degree of category similarity between the first predicted category and the ground truth category;

[0010] Based on the first and second discrimination results corresponding to each of the first reference bounding boxes, the total loss value of the model to be trained is determined;

[0011] The model parameters of the generating sub-model and the discriminant sub-model are updated based on the total loss value.

[0012] Secondly, an embodiment of this application provides a target detection method, the method comprising:

[0013] Obtain a third specified number of second reference bounding boxes from the second candidate bounding box set; the second candidate bounding box set is obtained by extracting the target region of the image to be detected using a preset region of interest extraction model;

[0014] The second reference bounding box is input into the target detection model for target detection, and the second predicted bounding box and the second predicted category corresponding to each second reference bounding box are obtained.

[0015] Based on the second predicted bounding box and the second predicted category corresponding to each second reference bounding box, the target detection result of the image to be detected is generated.

[0016] Thirdly, this application provides a model training apparatus, the apparatus comprising:

[0017] The first bounding box acquisition module is configured to acquire a first specified number of first reference bounding boxes from a first candidate bounding box set, and to acquire the ground truth bounding box and the ground truth category corresponding to each first reference bounding box; the first candidate bounding box set is obtained by extracting target regions from a preset sample image set using a preset region of interest extraction model;

[0018] The model training module is configured to input the first reference bounding box, the ground truth bounding box, and the ground truth class into the model to be trained for iterative training until the current model iteration training result meets the model iteration training termination condition, thereby obtaining an object detection model; wherein, the model to be trained includes a generator sub-model and a discriminator sub-model; the specific implementation of each model training is as follows:

[0019] For each first reference bounding box: the generative sub-model makes predictions based on the first reference bounding box to obtain a first predicted bounding box and a first predicted category; the discriminative sub-model generates a discrimination result set based on the ground truth bounding box and the first predicted bounding box corresponding to the first reference bounding box, and the ground truth category and the first predicted category corresponding to the first reference bounding box; the discrimination result set includes a first discrimination result and a second discrimination result, wherein the first discrimination result represents the degree of similarity in the bounding box distribution between the first predicted bounding box and the ground truth bounding box under the condition of satisfying a preset constraint, wherein the preset constraint is that the category of the target object in the first predicted bounding box is predicted by the generative sub-model to be a target category that matches the ground truth category, and the second discrimination result represents the degree of category similarity between the first predicted category and the ground truth category; based on the first discrimination result and the second discrimination result corresponding to each first reference bounding box, the total loss value of the model to be trained is determined; the model parameters of the generative sub-model and the discriminative sub-model are updated based on the total loss value.

[0020] Fourthly, an embodiment of this application provides a target detection device, the device comprising:

[0021] The second bounding box acquisition module is configured to acquire a third specified number of second reference bounding boxes from the second candidate bounding box set; the second candidate bounding box set is obtained by extracting the target region of the image to be detected using a preset region of interest extraction model;

[0022] The object detection module is configured to input the second reference bounding box into the object detection model for object detection, and obtain a second predicted bounding box and a second predicted category corresponding to each second reference bounding box;

[0023] The detection result generation module is configured to generate target detection results for the image to be detected based on the second predicted bounding box and the second predicted category corresponding to each second reference bounding box.

[0024] Fifthly, an embodiment of this application provides a computer device, the device comprising:

[0025] A processor; and a memory arranged to store computer-executable instructions configured to be executed by the processor, the executable instructions including steps for performing the methods described in the first or second aspect.

[0026] Sixthly, embodiments of this application provide a storage medium for storing computer-executable instructions that cause a computer to perform steps in the methods described in the first or second aspect.

[0027] As can be seen in the embodiments of this application, during the model training phase, by discriminating the sub-model based on the real bounding box and the first predicted bounding box obtained from the first reference bounding box, a first discrimination result representing the similarity of the bounding box distribution is output. This prompts the model parameters related to bounding box regression to be continuously updated, thereby prompting the generating sub-model to continuously learn the bounding box distribution, making the predicted first predicted bounding box closer to the real bounding box, thus improving the accuracy, model generalization, and data transferability of the trained object detection model in predicting the bounding box of the target object's location. Furthermore, during the model training phase, by discriminating the sub-model based on the real category and the first predicted category corresponding to the first reference bounding box, a second discrimination result representing the class similarity between the first predicted category and the real category is output, thereby prompting the generating sub-model to continuously learn the bounding box distribution, making the predicted first predicted bounding box closer to the real bounding box. The model continuously learns the category of the target object in the image region within the bounding box, making the predicted category closer to the true category. Since the model training does not focus on the predicted category itself, but rather on the true / false discrimination results of the discriminant sub-model, the model parameters related to the target object category prediction are continuously updated, improving the target classification accuracy, model generalization, and data transferability of the trained target detection model. Furthermore, the discrimination result set output by the discriminant sub-model includes the first and second discrimination results. Based on this discrimination result set, the total loss value of the model to be trained is determined, and the model parameters of the generator sub-model and the discriminant sub-model are iteratively updated multiple times based on the total loss value, thus ensuring the accuracy of target object location labeling and target object classification in the target detection process. Attached Figure Description

[0028] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in one or more of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0029] Figure 1 A schematic flowchart illustrating the model training method provided in this application embodiment;

[0030] Figure 2 A flowchart illustrating each model training process in the model training method provided in the embodiments of this application;

[0031] Figure 3 A schematic diagram illustrating the first implementation principle of the model training method provided in this application embodiment;

[0032] Figure 4a A schematic diagram illustrating the second implementation principle of the model training method provided in the embodiments of this application;

[0033] Figure 4b A schematic diagram illustrating the third implementation principle of the model training method provided in the embodiments of this application;

[0034] Figure 5 A schematic flowchart of the target detection method provided in the embodiments of this application;

[0035] Figure 6 A schematic diagram illustrating the implementation principle of the target detection method provided in the embodiments of this application;

[0036] Figure 7 This is a schematic diagram of the module composition of the model training device provided in the embodiments of this application;

[0037] Figure 8 This is a schematic diagram of the module composition of the target detection device provided in the embodiments of this application;

[0038] Figure 9 A schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0039] To enable those skilled in the art to better understand the technical solutions in one or more of this application, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of one or more of this application, and not all embodiments. Based on the embodiments of one or more of this application, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of this application.

[0040] It should be noted that, unless otherwise specified, one or more embodiments and features described in this application can be combined with each other. The embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0041] Considering that directly calculating the cross-entropy regression loss between the first predicted bounding box and the ground truth bounding box, and then iteratively training the model parameters based on the cross-entropy regression loss, would result in a target detection model that heavily relies on the preset sample image set used during model training. This would lead to poor generalization and cross-data transfer capabilities, inevitably resulting in high accuracy in predicting bounding boxes for the preset sample image set but low accuracy for new target image data. Therefore, during model training, a new method is used to determine the first predicted bounding box obtained by the discriminant sub-model based on the ground truth bounding box and the first reference bounding box. The bounding box output represents the first discrimination result indicating the similarity of the bounding box distribution. This prompts continuous updates to the model parameters related to bounding box regression, enabling the generating sub-model to continuously learn the bounding box distribution. This makes the predicted first bounding box closer to the true bounding box, thereby improving the accuracy, generalization, and data transferability of the trained object detection model in predicting the bounding box of the target object's location. This ensures the accuracy of bounding box regression for new images using the object detection model, improving the data transfer adaptability of the trained object detection model when used for bounding box prediction. Furthermore, considering that directly calculating two bounding boxes based on the first predicted category and the true category... The model employs cross-entropy classification loss between categories, and then iteratively trains the model parameters based on this cross-entropy loss. However, this trained object detection model is highly dependent on the pre-set sample image set used during training. Consequently, the model exhibits poor generalization and cross-data transfer capabilities, inevitably resulting in high accuracy for classifying objects from the pre-set sample image set but low accuracy for new target image data. Therefore, during model training, a discriminant sub-model is used to determine the true and predicted categories based on the first reference bounding box, outputting a representation of the category similarity between the predicted and true categories. The second discrimination result of the degree prompts the generating sub-model to continuously learn the category of the target object in the image region within the bounding box, making the predicted first category closer to the true category. Since the predicted category itself is not considered during model training, but rather the true and false discrimination results of the discrimination sub-model are used, the model parameters related to the prediction of the target object category are continuously updated, which improves the target classification accuracy, model generalization and data transferability of the trained target detection model. This ensures that the target object classification accuracy of the target detection model is used for new images to be detected, and improves the data transfer adaptability of the trained target detection model when used for target object classification.

[0042] Figure 1 This is a flowchart illustrating a model training method provided in one or more embodiments of this application. Figure 2The method described above can be executed by an electronic device equipped with a model training device, which can be a terminal device or a designated server. The hardware device used for training the object detection model (i.e., the electronic device equipped with the model training device) and the hardware device for object detection (i.e., the electronic device equipped with the object detection device) can be the same or different. Specifically, regarding the training process of the object detection model, such as... Figure 1 As shown, the method includes at least the following steps:

[0043] S102, obtain a first specified number of first reference bounding boxes from the first candidate bounding box set, and obtain the ground truth bounding box and the ground truth category corresponding to each first reference bounding box; wherein, the first candidate bounding box set is obtained by extracting the target region from the preset sample image set using a preset region of interest extraction model, and the first specified number can be N, where N is a positive integer greater than 1;

[0044] Specifically, the process of determining the N first reference bounding boxes can be as follows: for each round of model training, perform a step of extracting the target region from a preset sample image set using a preset region of interest extraction model to obtain N first reference bounding boxes; or, perform the step of extracting the target region from a preset sample image set using a preset region of interest extraction model in advance, and then, for each round of model training, randomly sample from a large number of pre-extracted candidate bounding boxes to obtain N first reference bounding boxes.

[0045] Specifically, the preset sample image set may contain multiple sample target objects, and each sample target object may correspond to multiple first reference bounding boxes, that is, N first reference bounding boxes contain at least one first reference bounding box corresponding to each sample target object.

[0046] Specifically, before obtaining a first specified number of first reference bounding boxes from the first candidate bounding box set in step S102, the method further includes: inputting a preset sample image set into a preset region of interest extraction model to extract the region of interest, thereby obtaining a first candidate bounding box set, which includes a second specified number of candidate bounding boxes; wherein, the second specified number can be X, X≥N, where X is a positive integer greater than 1. That is, for the case where X=N, i.e., the second specified number is equal to the first specified number, for each round of model training, the preset region of interest extraction model is used to extract the region of interest from multiple sample image data in the preset sample image set to obtain a first specified number of first reference bounding boxes; for the case where X>N, i.e., the second specified number is greater than the first specified number, for each round of model training, a first specified number of first reference bounding boxes are randomly sampled from the first specified number of candidate bounding boxes.

[0047] One of the goals of model training is to continuously learn the bounding box distribution through iterative training of model parameters, thereby improving the model's generalization and data transferability (i.e., the model parameters do not depend on the sample data used in the model training process and can be better applied to the data to be identified in the model application process). In order to enable the model to learn the bounding box distribution better, it is necessary to ensure that the extracted first reference bounding box of the model to be trained follows a certain probability distribution (such as Gaussian distribution or Cauchy distribution). Thus, the larger the number N of anchor boxes extracted by the preset region of interest extraction model, the better it helps the model to learn the bounding box distribution. However, if X anchor boxes are extracted in real time each time using the preset region of interest extraction model (such as the region of interest extraction algorithm ROI) as the first reference bounding box and input into the model to be trained, it will inevitably lead to a large amount of data processing and high requirements for hardware devices.

[0048] In specific implementation, it is preferable to pre-extract X anchor boxes using a preset region of interest extraction model. Then, in each round of model training, N anchor boxes are randomly sampled from the X anchor boxes as first reference bounding boxes and input into the model to be trained. This ensures both the amount of data processing in each round of model training and the model's better learning of bounding box distribution. That is, it can simultaneously take into account the amount of data processing during model training and promote the learning of bounding box distribution. Based on this, the second specified number X is greater than the first specified number N. Correspondingly, step S102, obtaining the first specified number of first reference bounding boxes from the first candidate bounding box set, specifically includes: randomly selecting the first specified number of candidate bounding boxes as first reference bounding boxes from the second specified number of candidate bounding boxes. That is, the preset region of interest extraction model is used in advance to extract the region of interest from multiple sample image data in the preset sample image set to obtain X candidate bounding boxes. Then, for each round of model training, N first reference bounding boxes are randomly sampled from the X candidate bounding boxes.

[0049] In other words, the preferred implementation is to pre-extract X anchor boxes (i.e., a second specified number of candidate bounding boxes), and then, for each round of model training, randomly sample N anchor boxes (i.e., a first specified number of first reference bounding boxes) from the X anchor boxes, and then continue to execute the following step S104.

[0050] S104, the first reference bounding box, the ground truth bounding box, and the ground truth class are input into the model to be trained for iterative training until the current model iterative training result meets the model iterative training termination condition, and the object detection model is obtained; the above model iterative training termination condition may include: the current model training round number is equal to the total training round number, the model loss function converges, or the generator sub-model and the discriminator sub-model reach a balance.

[0051] Specifically, regarding the model iterative training process in step S104 above, the specific implementation process of model iterative training is explained below. Since the processing procedure is the same for each model training iteration, a detailed explanation is given using any single model training iteration as an example. Specifically, if the model to be trained includes a generator sub-model and a discriminator sub-model; such as... Figure 2 As shown, each model training iteration can be implemented using the following steps S1042 to S1046:

[0052] S1042, for each first reference bounding box: the generation sub-model makes predictions based on the first reference bounding box to obtain a first predicted bounding box and a first predicted category; the discrimination sub-model generates a discrimination result set based on the ground truth bounding box and the first predicted bounding box corresponding to the first reference bounding box, as well as the ground truth category and the first predicted category corresponding to the first reference bounding box; wherein, the discrimination result set includes a first discrimination result and a second discrimination result, the first discrimination result representing the degree of similarity in the bounding box distribution between the first predicted bounding box and the ground truth bounding box under the condition of satisfying a preset constraint, the preset constraint being that the category of the target object in the first predicted bounding box is predicted by the generation sub-model to be the target category that matches the corresponding ground truth category, and the second discrimination result representing the degree of category similarity between the first predicted category corresponding to the first reference bounding box and the corresponding ground truth category;

[0053] Specifically, since the aforementioned generative sub-model is used not only to predict bounding boxes based on the first reference bounding box to obtain the corresponding first predicted bounding box, but also to predict the category of the target object in the image region within the first reference bounding box or the first predicted bounding box, the model parameters of the generative sub-model include first model parameters related to bounding box regression and second model parameters related to target object category prediction. Therefore, during model training, the first and second model parameters need to be iteratively updated together. That is, the first model parameters are iteratively updated based on the first discrimination result corresponding to each first reference bounding box, and the second model parameters are iteratively updated based on the second discrimination result corresponding to each first reference bounding box.

[0054] Specifically, regarding the determination process of the first discriminant result representing the similarity of bounding box distributions, the KL divergence between the true bounding box and the corresponding first predicted bounding box can be directly calculated. However, in practice, considering that the discriminant sub-model can determine whether the first predicted bounding box predicted by the generator sub-model is sufficiently realistic, and when the generated bounding box (i.e., the first predicted bounding box) and the true bounding box (i.e., the true bounding box) are difficult to distinguish, due to the existence of the discriminant sub-model, adjusting the model parameters based on the discriminant sub-model's discrimination result can further make the first predicted bounding box predicted by the generator sub-model closer to the true bounding box. Therefore, to further improve the accuracy of the regression loss component corresponding to the similarity of bounding box distributions, and thus ensure that the first predicted bounding box predicted by the object detection model is more realistic, a discriminant sub-model can be used to determine the discrimination probability of the real bounding box and the corresponding first predicted bounding box, whether they originate from real data or generated data, for each first reference bounding box. Since the magnitude of the discrimination probability is related to the similarity of the probability distributions of the two bounding boxes (i.e., the real bounding box and the corresponding first predicted bounding box), the discrimination probability can characterize the distribution similarity between the real bounding box and the corresponding first predicted bounding box. The discriminant probability can determine the first regression loss component corresponding to the discriminant dimension, which considers the similarity of bounding box distributions, thereby prompting the model to perform bounding box regression learning. Specifically, for the true bounding box and the first predicted bounding box corresponding to a certain first reference bounding box, the discriminant submodel has the probability of judging that the true bounding box comes from real data and the probability of judging that the first predicted bounding box comes from generated data. The higher the probability of the discriminant submodel judging that the true bounding box comes from real data and the higher the probability of the first predicted bounding box coming from generated data, the more similar the probability distributions of the first predicted bounding box and the corresponding true bounding box are. The lower the degree, the larger the corresponding first regression loss component for the discrimination dimension of the similarity of bounding box distribution. Therefore, the distribution similarity between the first predicted bounding box and the corresponding real bounding box corresponding to a certain first reference bounding box is determined by the discrimination probability of the discriminant sub-model for whether the real bounding box and the first predicted bounding box come from real data or generated data, respectively. Therefore, the first discrimination result can be generated based on the discrimination probability of the discriminant sub-model. In this way, the first discrimination result can characterize the similarity of bounding box distribution, and the first regression loss component corresponding to the discrimination dimension of the similarity of bounding box distribution can be determined based on the discrimination probability in the first discrimination result.

[0055] Furthermore, since the generative sub-model can simultaneously predict bounding boxes and categories, considering the low accuracy of the first reference bounding box for category prediction, the first predicted bounding box corresponding to such a first reference bounding box may not truly reflect the bounding box prediction accuracy of the generative sub-model. Consequently, the discrimination result of the discriminant sub-model for the first predicted bounding box and the ground truth bounding box corresponding to such a first reference bounding box also cannot truly reflect the bounding box prediction accuracy of the generative sub-model. Therefore, in the process of determining the first discrimination result representing the similarity of bounding box distributions, the aforementioned preset constraint is introduced, and this preset constraint is used as a precondition (i.e., the first predicted category corresponding to the first reference bounding box satisfies a certain preset category matching condition). (With constraints), the discriminant submodel determines the conditional discriminant probability that the true bounding box comes from the real data and the conditional discriminant probability that the first predicted bounding box comes from the generated data. That is, under the condition that the category of the target object in the first predicted bounding box is predicted by the generating submodel to be the target category that matches the corresponding real category (i.e., the first predicted category matches the real category), the discriminant submodel determines the first discriminant probability that the true bounding box comes from the real data and the second discriminant probability that the first predicted bounding box comes from the generated data. In other words, the first discrimination result corresponding to the first reference bounding box is determined only when the first predicted category corresponding to the first reference bounding box satisfies the above-mentioned preset constraints.

[0056] Correspondingly, the process of determining the second discrimination result regarding the class similarity between the first predicted class and the true class can directly calculate the cross-entropy classification loss between the two classes based on the first predicted class and the true class. However, in practical implementation, considering that the discriminant sub-model can determine whether the first predicted class predicted by the generator sub-model is sufficiently realistic, and when it is difficult to distinguish between the generated target object class (i.e., the first predicted class) and the real target object class (i.e., the true class), the existence of the discriminant sub-model allows for adjusting the model parameters based on its discrimination result. This can further make the first predicted class predicted by the generator sub-model closer to the true class. Therefore, to further improve the accuracy of the classification loss component corresponding to class similarity, and thus ensure that the first predicted class predicted by the object detection model is more realistic, for each first reference bounding box, the discriminant sub-model can be used to determine whether the true class and the corresponding first predicted class of that first reference bounding box come from real data or generated data, respectively. Since the magnitude of the discrimination probability is related to the closeness of the two classes, the discrimination probability can characterize the difference between the true class and the corresponding first predicted class. The class similarity between the first predicted categories is determined by the discriminant submodel, which uses the discrimination probability to determine the classification loss component corresponding to the first reference bounding box, thereby prompting the model to learn target classification. Specifically, for the true category and the first predicted category corresponding to a certain first reference bounding box, the discriminant submodel determines the probability that the true category comes from real data and the probability that the first predicted category comes from generated data. The higher the discriminant submodel determines that the true category comes from real data and the higher the probability that the first predicted category comes from generated data, the lower the class similarity between the first predicted category and the corresponding true category, and the larger the classification loss component corresponding to the first reference bounding box. Therefore, the class similarity between the first predicted category and the corresponding true category corresponding to a certain first reference bounding box is determined by the discriminant submodel based on the discrimination probability of the true category and the first predicted category coming from real data or generated data, respectively. Thus, a second discrimination result can be generated based on the discrimination probability of the discriminant submodel. This second discrimination result can characterize the class similarity, and the classification loss component corresponding to the first reference bounding box can be determined based on the discrimination probability in the second discrimination result.

[0057] S1044, Based on the first and second discrimination results corresponding to each first reference bounding box, determine the total loss value of the model to be trained;

[0058] The total loss value mentioned above includes the regression loss value determined based on the first discrimination result corresponding to each first reference bounding box, and the classification loss value determined based on the second discrimination result corresponding to each first reference bounding box.

[0059] Specifically, after obtaining the discrimination result set for each first reference bounding box, the sub-loss value corresponding to each first reference bounding box can be obtained. The sub-loss value includes at least the first regression loss component corresponding to the first discrimination dimension from the perspective of the similarity of the bounding box distribution, and the classification loss component corresponding to the classification discrimination dimension from the perspective of the class similarity between the true class and the first predicted class. Then, based on the sub-loss value corresponding to each first reference bounding box, the total loss value used to adjust the model parameters can be determined.

[0060] S1046, Update the model parameters of the above-mentioned generating sub-model and discriminant sub-model based on the above total loss value.

[0061] Specifically, after determining the total loss value based on the sub-loss values ​​corresponding to each first reference bounding box, the gradient descent method is used to adjust the parameters of the generating sub-model and the discriminant sub-model based on the total loss value. Among them, since the classification loss value related to the target object classification is obtained based on the discrimination probability of the discriminant sub-model, it does not focus on the predicted category itself during the model training process, but relies on the true and false discrimination results of the discriminant sub-model, so that the target classification accuracy of the finally trained target detection model is higher. Furthermore, since the regression loss value related to the bounding box prediction at least reflects the first regression loss component corresponding to the regression loss discrimination dimension based on the similarity of the bounding box distribution, the finally trained target detection model can not only ensure that the probability distribution of the predicted first bounding box is closer to that of the true bounding box.

[0062] During model training, the discriminator sub-model tries to distinguish between the ground truth bounding box and the first predicted bounding box corresponding to the first reference bounding box, whether they come from real data or generated data, and also tries to distinguish between the ground truth class and the first predicted class corresponding to the first reference bounding box, whether they come from real data or generated data, in order to minimize the regression classification loss of the model to be trained. In order to maximize the discrimination error of the discriminator sub-model, the generator sub-model is forced to continuously learn the bounding box distribution and target object category recognition, prompting the generator sub-model and the discriminator sub-model to conduct multiple rounds of adversarial learning, so as to obtain a more accurate generator sub-model as the object detection model.

[0063] It should be noted that the process of iteratively training the model parameters based on the total loss value of the model to be trained to obtain the object detection model can be found in the existing process of using gradient descent to backpropagate and fine-tune the model parameters, which will not be repeated here.

[0064] Furthermore, the target detection model trained based on the model training method provided in this application embodiment can be applied to any specific application scenario that requires target detection of the image to be detected. For example, in specific application scenario 1, target detection is performed on the image to be detected obtained by image acquisition equipment at the entrance of a public place (such as the entrance of a shopping mall, subway entrance, scenic spot entrance, or performance venue entrance). In another example, in specific application scenario 2, target detection is performed on the image to be detected obtained by image acquisition equipment at each monitoring point in a breeding base.

[0065] Due to the different application scenarios of the target detection model, the preset sample image set used in the training process of the target detection model is also different. For specific application scenario 1, the preset sample image set can be historical sample images collected at the entrance of a designated public place within a preset historical time period. Correspondingly, the target object enclosed by the first reference bounding box is the target user entering the designated public place in the historical sample images. The true category and the first predicted category can be the category to which the target user belongs, such as at least one of age group, gender, height, and occupation. For specific application scenario 2, the preset sample image set can be historical sample images collected at various monitoring points in a designated breeding base within a preset historical time period. Correspondingly, the target object enclosed by the first reference bounding box is the target breeding object in the historical sample images. The true category and the first predicted category can be the category to which the target breeding object belongs, such as at least one of live status and body size.

[0066] like Figure 3 As shown, a schematic diagram illustrating the specific implementation principle of the object detection model training process is presented, including:

[0067] Obtain a first specified number of first reference bounding boxes, and obtain the ground truth bounding box and ground truth class corresponding to each first reference bounding box;

[0068] For each first reference bounding box: the above-mentioned generation sub-model performs bounding box prediction based on the first reference bounding box to obtain a first predicted bounding box, and performs target category prediction on the image region within the first reference bounding box or the first predicted bounding box to obtain a first predicted category; the above-mentioned discrimination sub-model generates a discrimination result set based on the ground truth bounding box and the first predicted bounding box corresponding to the first reference bounding box, and the ground truth category and the first predicted category corresponding to the first reference bounding box.

[0069] Based on the first and second discrimination results corresponding to each first reference bounding box, the total loss value of the model to be trained is determined.

[0070] Based on the total loss value mentioned above, the model parameters of the model to be trained are iteratively updated until the current model training result meets the preset model training termination condition, thus obtaining the object detection model.

[0071] Specifically, step S1042 above generates a discrimination result set based on the ground truth bounding box and the first predicted bounding box corresponding to the first reference bounding box, and the ground truth class and the first predicted class corresponding to the first reference bounding box, specifically including:

[0072] Under the condition that the first predicted category corresponding to the first reference bounding box satisfies the above-mentioned preset constraints, the true and false bounding boxes corresponding to the first reference bounding box are judged to obtain the first judgment result; and the true and false categories corresponding to the first reference bounding box are judged to obtain the second judgment result.

[0073] Furthermore, to improve the accuracy of the regression loss corresponding to each first reference bounding box, thereby improving the training accuracy of the model parameters, the bounding box regression loss is considered not only from the first discriminant dimension representing the similarity of the bounding box distribution, but also from the second discriminant dimension representing the degree of overlap of the bounding box coordinates. Based on this, the above-mentioned discrimination result set also includes a third discrimination result, which represents the degree of overlap between the bounding box coordinates of the first predicted bounding box and the real bounding box under the condition of satisfying the above-mentioned preset constraints. Correspondingly, step S1042 above generates a discrimination result set based on the real bounding box and the first predicted bounding box corresponding to the first reference bounding box, and the real category and the first predicted category corresponding to the first reference bounding box, specifically including:

[0074] Under the condition that the first predicted category corresponding to the first reference bounding box satisfies the above-mentioned preset constraints, the true and false bounding boxes corresponding to the first reference bounding box are judged to obtain the first judgment result.

[0075] The true and false categories corresponding to the first reference bounding box are determined to obtain the second determination result.

[0076] Under the condition that the first predicted category corresponding to the first reference bounding box satisfies the above-mentioned preset constraints, the cross-union ratio loss of the bounding box is calculated based on the real bounding box and the first predicted bounding box corresponding to the first reference bounding box, and the third discrimination result is obtained.

[0077] Specifically, regarding the determination process of the third discrimination result representing the degree of overlap between bounding box coordinates, the target cross-union ratio (CUI) loss can be obtained by considering only the cross-union ratio loss between a ground truth bounding box and its corresponding first predicted bounding box; alternatively, the target CUI loss can be determined by comprehensively considering the CUI loss between a ground truth bounding box and its corresponding first predicted bounding box, as well as the CUI loss between a ground truth bounding box and the first predicted bounding boxes corresponding to other ground truth bounding boxes. Since the magnitude of the target CUI loss can represent the degree of overlap between the ground truth bounding box and its corresponding first predicted bounding box, the second regression loss component corresponding to the discrimination dimension considering the degree of overlap between bounding box coordinates can be determined based on the target CUI loss, thereby prompting the model to perform bounding box regression learning. Specifically, for a ground truth bounding box corresponding to a first reference bounding box... For the true bounding box and the first predicted bounding box, the target intersection-union ratio (DIU) loss between the true bounding box and the first predicted bounding box is determined. The larger the DIU loss, the lower the degree of coordinate overlap between the first predicted bounding box and the corresponding true bounding box. The larger the corresponding second regression loss component for the discrimination dimension of the degree of bounding box coordinate overlap, the greater the degree of coordinate overlap between the first predicted bounding box and the corresponding true bounding box for a certain first reference bounding box. Therefore, the degree of coordinate overlap between the first predicted bounding box and the corresponding true bounding box is determined based on the DIU loss between the true bounding box and the first predicted bounding box. Thus, a third discrimination result can be generated based on the DIU loss. In this way, the third discrimination result can characterize the degree of bounding box coordinate overlap. Then, based on the DIU loss in the third discrimination result, the second regression loss component corresponding to the discrimination dimension of the degree of bounding box coordinate overlap can be determined.

[0078] Furthermore, since the third discrimination result is also used to determine the sub-regression loss value, the above-mentioned preset constraints are also introduced in the process of determining the third discrimination result, which characterizes the degree of overlap of bounding box coordinates. The above-mentioned preset constraints are used as a prerequisite (that is, the first predicted category corresponding to the first reference bounding box satisfies a certain preset category matching constraint condition) to determine the conditional intersection-union ratio loss between the true bounding box and the first predicted bounding box. That is, under the condition that the category of the target object in the first predicted bounding box is predicted by the generated sub-model as the target category that matches the corresponding true category (that is, the first predicted category matches the true category), the target intersection-union ratio loss corresponding to the first reference bounding box is determined. In other words, the third discrimination result corresponding to the first reference bounding box is determined only when the first predicted category corresponding to the first reference bounding box satisfies the above-mentioned preset constraints.

[0079] In cases where the discrimination result set also includes a third discrimination result, the total loss value mentioned above includes the regression loss value determined based on the first and third discrimination results corresponding to each first reference bounding box, and the classification loss value determined based on the second discrimination result corresponding to each first reference bounding box. Specifically, after obtaining the discrimination result set for each first reference bounding box, the sub-loss value corresponding to each first reference bounding box can be obtained. This sub-loss value includes at least the first regression loss component corresponding to the first discrimination dimension considering the similarity of the bounding box distribution, the classification loss component corresponding to the classification discrimination dimension considering the similarity between the true category and the first predicted category, and the second regression loss component corresponding to the second discrimination dimension considering the overlap of the bounding box coordinates. Then, based on the sub-loss values ​​corresponding to each first reference bounding box, the total loss value used to adjust the model parameters can be determined, that is, based on the first and second discrimination results corresponding to each first reference bounding box, the regression loss value is determined, and the first model parameters related to the bounding box regression are iteratively updated.

[0080] Specifically, the sub-loss values ​​corresponding to each first reference bounding box can include sub-classification loss values ​​and sub-regression loss values. Since the sub-regression loss values ​​related to bounding box prediction at least reflect the first regression loss component corresponding to the regression loss discrimination dimension based on the similarity of bounding box distributions, and the second regression loss component corresponding to the regression loss discrimination dimension based on the degree of overlap of bounding box coordinates, the regression loss values ​​used to adjust the model parameters also reflect the regression loss components corresponding to these two regression loss discrimination dimensions respectively. This ensures that the finally trained object detection model can not only ensure that the probability distribution of the predicted first bounding box is closer to that of the real bounding box, but also ensure that the degree of overlap between the coordinates of the predicted first bounding box and the real bounding box is higher.

[0081] In the embodiments provided in this application, it is also considered that if the model regression loss is determined solely from the coarse-grained comparison dimension of the similarity of bounding box distributions and the model parameters are adjusted accordingly, the precise location learning of the bounding boxes cannot be taken into account. Alternatively, if the model regression loss is determined solely from the fine-grained comparison dimension of the overlap of bounding box coordinates and the model parameters are adjusted accordingly, the edge ambiguity problem of the bounding boxes cannot be taken into account. Therefore, the model regression loss is determined by combining the coarse-grained comparison dimension of the similarity of bounding box distributions and the fine-grained comparison dimension of the overlap of bounding box coordinates. That is, the set of discrimination results output by the discriminant sub-model includes not only the first... The first and second discrimination results, as well as the third discrimination result representing the degree of overlap of bounding box coordinates, achieve the effect of compensating for the bounding box regression loss caused by similar bounding box distributions but specific positional deviations. Based on this set of discrimination results, the total loss value of the model to be trained is determined, and the model parameters of the generated sub-model and the discrimination sub-model are continuously updated in multiple rounds based on the total loss value. As the total loss value obtained based on the set of discrimination results is more accurate, the accuracy of the model parameters updated based on the total loss value is also higher, thereby further improving the accuracy of target object location marking in the target detection process.

[0082] Furthermore, considering that during model training, the gradient of the regression classification loss obtained based on the discrimination results output by the discriminant sub-model may suddenly decrease or even become zero, a loss compensation value is introduced to further improve the training accuracy of the model parameters. Therefore, the aforementioned discrimination result set also includes a fourth discrimination result. Correspondingly, the discrimination result set generated in S1042 based on the true bounding box and the first predicted bounding box corresponding to the first reference bounding box, and the true class and the first predicted class corresponding to the first reference bounding box, specifically includes:

[0083] Under the condition that the first predicted category corresponding to the first reference bounding box satisfies the above-mentioned preset constraints, the true and false bounding boxes corresponding to the first reference bounding box are judged to obtain the first judgment result.

[0084] The true and false categories corresponding to the first reference bounding box are judged to obtain the second discrimination result.

[0085] Under the condition that the first predicted category corresponding to the first reference bounding box satisfies the above-mentioned preset constraints, the cross-union ratio loss of the bounding box is calculated based on the real bounding box and the first predicted bounding box corresponding to the first reference bounding box to obtain the third discrimination result;

[0086] Based on the true bounding box and the first predicted bounding box corresponding to the first reference bounding box, the loss compensation value used to constrain the loss gradient of the loss function of the model to be trained is calculated, and the fourth discrimination result is obtained.

[0087] Specifically, since the determination process of the first and third discrimination results is related to whether the first predicted category meets the preset category matching constraints, for a first reference bounding box whose first predicted category does not meet the preset category matching constraints, the first and third discrimination results are empty, or the corresponding discrimination result set only includes the second and fourth discrimination results. That is, only the second and fourth discrimination results corresponding to the first reference bounding box are considered. In other words, for a first reference bounding box whose first predicted category does not meet the preset category matching constraints, only the classification loss corresponding to this type of first reference bounding box is included, and the regression loss corresponding to this type of first reference bounding box is not included. Conversely, for a first reference bounding box whose first predicted category meets the preset category matching constraints, not only the second and fourth discrimination results corresponding to the first reference bounding box are considered, but also the first and third discrimination results corresponding to the first reference bounding box are considered. That is, for a first reference bounding box whose first predicted category meets the preset category matching constraints, both the classification loss and the regression loss corresponding to this type of first reference bounding box are included.

[0088] Specifically, for each first reference bounding box, the set of discrimination results corresponding to the first reference bounding box includes not only the first discrimination result obtained from the perspective of the similarity of the bounding box distribution, the second discrimination result used to determine the classification loss, and the third discrimination result obtained from the perspective of the overlap of the bounding box coordinates, but also the loss compensation value used to constrain the loss gradient of the loss function. This can not only improve the accuracy of the total loss value, but also solve the problem of the loss gradient suddenly decreasing or even becoming zero.

[0089] In specific implementation, such as Figure 4a As shown, a schematic diagram illustrating the specific implementation principle of another object detection model training process is presented, including:

[0090] The target region is extracted from a preset sample image set using a preset region of interest extraction model, resulting in X anchor boxes. The preset sample image set includes multiple original sample images, each of which contains at least one target object. The feature information corresponding to each anchor box can include location information (x, y, w, h) and category information c, i.e., (x, y, w, h, c). Specifically, during model training, multiple parameter dimensions can be set to be independent of each other. Therefore, the iterative training process for the model parameters of each dimension is also independent of each other.

[0091] For each round of model training, N anchor boxes are randomly sampled from X anchor boxes as first reference bounding boxes, and the ground truth bounding box and ground truth class corresponding to each first reference bounding box are determined. Each target object in the preset sample image set can correspond to one ground truth bounding box and one ground truth class. For example, if the total number of target objects in the preset sample image set is d, then the number of ground truth bounding boxes before augmentation is d. To ensure that the ground truth bounding boxes correspond to the first predicted bounding boxes, the ground truth bounding boxes corresponding to multiple first reference bounding boxes containing the same target object can be... The process is the same: based on the target object defined by the first reference bounding box, the ground truth bounding box is expanded to obtain N ground truth bounding boxes (N>d). For example, if the target object in a certain original sample image is a cat A, and cat A corresponds to ground truth bounding box A, and if the number of first reference bounding boxes containing cat A is 4 (such as the first reference bounding boxes numbered 6, 7, 8, and 9), then the ground truth bounding box A is expanded to 4 ground truth bounding boxes A (i.e., ground truth bounding boxes numbered 6, 7, 8, and 9), and the ground truth category corresponding to the 4 expanded ground truth bounding boxes A is cat.

[0092] For each first reference bounding box, the generative sub-model predicts the bounding box based on the first reference bounding box to obtain a first predicted bounding box, and predicts the target category of the image region within the first reference bounding box or the first predicted bounding box to obtain a first predicted category. The discriminative sub-model generates a discrimination result set based on the ground truth bounding box and the first predicted bounding box corresponding to the first reference bounding box, as well as the ground truth category and the first predicted category corresponding to the first reference bounding box. Each first reference bounding box corresponds to a ground truth bounding box and its corresponding ground truth category, and a first predicted bounding box and its corresponding first predicted category. The first predicted bounding box is predicted by the generative sub-model through continuous bounding box regression learning, and the first predicted category is predicted by the generative sub-model through continuous target classification and recognition learning. Specifically, the target object enclosed by the first predicted bounding boxes numbered 6, 7, 8, and 9 in the N first predicted bounding boxes output by the generative sub-model is cat A.

[0093] For each first reference bounding box, at least one of the following loss components is determined: a first regression loss component is determined based on a first discrimination result in the discrimination result set of the first reference bounding box; a classification loss component is determined based on a second discrimination result in the discrimination result set of the first reference bounding box; a second regression loss component is determined based on a third discrimination result in the discrimination result set of the first reference bounding box; and a loss compensation component is determined based on a fourth discrimination result in the discrimination result set of the first reference bounding box. Specifically, if the first reference bounding box does not meet the above preset constraints, the first and third discrimination results are empty. Correspondingly, the first and second regression loss components are both equal to zero, that is, the sub-regression loss value of the first reference bounding box is not considered, and only its sub-classification loss value and loss compensation value are considered.

[0094] Based on at least one of the first regression loss component, classification loss component, second regression loss component, and loss compensation component corresponding to each first reference bounding box, the total loss value of the model to be trained is determined; using the stochastic gradient descent method, the model parameters of the above-mentioned generator sub-model and discriminator sub-model are adjusted based on the total loss value to obtain the generator sub-model and discriminator sub-model with updated parameters;

[0095] If the current model iteration training result meets the model iteration training termination condition, then the updated generated sub-model is determined as the trained object detection model.

[0096] If the current model iteration training result does not meet the model iteration training termination condition, then the updated generator sub-model and discriminator sub-model will be determined as the training models to be used in the next round of model training, until the model iteration training termination condition is met.

[0097] Specifically, during model training, for each round of model training, the model parameters of the discriminant sub-model can be adjusted based on the discriminant result set, and the model parameters of the generated sub-model can also be adjusted based on the discriminant result set. However, in practice, to improve the training accuracy of the generated sub-model's model parameters, for each round of model training, the model parameters of the discriminant sub-model are first adjusted t times based on the discriminant result set, and then the model parameters of the generated sub-model are adjusted once based on the discriminant result set. The resulting discriminant sub-model and generated sub-model with adjusted parameters are used as the models to be trained in the next round.

[0098] The total loss value of the model to be trained is determined by the sub-loss values ​​corresponding to multiple first reference bounding boxes. Each sub-loss value corresponding to a first reference bounding box is determined by multiple loss components. Therefore, in step S1044, based on the first and second discrimination results corresponding to each first reference bounding box, the total loss value of the model to be trained is determined, specifically including:

[0099] Determine the sub-loss value corresponding to each first reference bounding box; the sub-loss value corresponding to each first reference bounding box is determined based on target information, wherein the target information includes at least one of the following: whether the first predicted category corresponding to the first reference bounding box satisfies the above-mentioned preset constraints, the degree of similarity of the bounding box distribution represented by the first discrimination result corresponding to the first reference bounding box, the degree of similarity of the categories represented by the second discrimination result, the degree of overlap of the bounding box coordinates represented by the third discrimination result, and the loss compensation value represented by the fourth discrimination result;

[0100] The total loss value of the model to be trained is determined based on the sub-loss value corresponding to each first reference bounding box.

[0101] The sub-loss value corresponding to each first reference bounding box includes a sub-regression loss value and a sub-classification loss value. In specific implementation, the sub-loss value may also include a loss compensation value or may not include a loss compensation value. The sub-regression loss value may be determined based on the first regression loss component or may be determined based on both the first and second regression loss components. The sub-classification loss value is determined based on the classification loss component. Specifically, for a certain first reference bounding box, if the first predicted category corresponding to the first reference bounding box does not meet the above-mentioned preset constraints, the target information used to determine the sub-loss value corresponding to the first reference bounding box may include: the category similarity represented by the second discrimination result, that is, determining the sub-loss value corresponding to the first reference bounding box based on the classification loss component corresponding to the second discrimination result, that is, only considering the sub-classification loss value corresponding to the first reference bounding box, and not considering the sub-regression loss value corresponding to the first original bounding box; or the target information used to determine the sub-loss value corresponding to the first reference bounding box may also include: the category similarity represented by the second discrimination result and the loss compensation value represented by the fourth discrimination result, that is, determining the first reference bounding box based on the classification loss component corresponding to the second discrimination result and the loss compensation component corresponding to the fourth discrimination result. The sub-loss value corresponding to the bounding box, that is, only considering the sub-classification loss value and loss compensation value corresponding to the first reference bounding box, without considering the sub-regression loss value corresponding to the first reference bounding box; if the first predicted category corresponding to the first reference bounding box satisfies the above-mentioned preset constraints, the target information used to determine the sub-loss value corresponding to the first reference bounding box may include: the similarity of the bounding box distribution represented by the first discrimination result corresponding to the first reference bounding box, and the similarity of the category represented by the second discrimination result, that is, determining the sub-loss value corresponding to the first reference bounding box based on the first regression loss component corresponding to the first discrimination result and the classification loss component corresponding to the second discrimination result; or the target information used to determine the sub-loss value corresponding to the first reference bounding box may also include: the similarity of the bounding box distribution represented by the first discrimination result corresponding to the first reference bounding box, the similarity of the category represented by the second discrimination result, and the degree of overlap of the bounding box coordinates represented by the third discrimination result, that is, determining the sub-loss value corresponding to the first reference bounding box based on the first regression loss component corresponding to the first discrimination result, the classification loss component corresponding to the second discrimination result, and the second regression loss component corresponding to the third discrimination result, that is, simultaneously considering the sub-classification loss value and the sub-regression loss value corresponding to the first reference bounding box;Alternatively, the target information used to determine the sub-loss value corresponding to the first reference bounding box may also include: the similarity of the bounding box distribution represented by the first discrimination result corresponding to the first reference bounding box, the similarity of the categories represented by the second discrimination result, the degree of overlap of the bounding box coordinates represented by the third discrimination result, and the loss compensation value represented by the fourth discrimination result. That is, the sub-loss value corresponding to the first reference bounding box is determined based on the first regression loss component corresponding to the first discrimination result, the classification loss component corresponding to the second discrimination result, the second regression loss component corresponding to the third discrimination result, and the loss compensation component corresponding to the fourth discrimination result; that is, the sub-classification loss value, sub-regression loss value, and loss compensation value corresponding to the first reference bounding box are considered simultaneously.

[0102] Specifically, taking a first reference bounding box that satisfies the aforementioned preset constraints and considering the loss compensation component as an example, the sub-loss value corresponding to this first reference bounding box is equal to the weighted sum of the four loss components, which can be specifically expressed as follows:

[0103] V i (D,G)=λ1V i1 +λ2V i2 +λ3V i3 +λ4V i4

[0104] Where λ1 represents the first weight coefficient corresponding to the first regression loss component under the first discriminant dimension, V i1 V represents the first regression loss component under the first discriminant dimension (i.e., the regression loss component corresponding to the similarity to the bounding box distribution represented by the first discriminant result), λ2 represents the second weight coefficient corresponding to the classification loss component, and V i2 V represents the classification loss component (i.e., the classification loss component corresponding to the class similarity represented by the second discrimination result), λ3 represents the third weight coefficient corresponding to the second regression loss component under the second discrimination dimension, and V i3 V represents the second regression loss component under the second discriminant dimension (i.e., the regression loss component corresponding to the degree of overlap with the bounding box coordinates represented by the third discriminant result), λ4 represents the fourth weight coefficient corresponding to the loss compensation value, and V i4 This represents the loss compensation value (i.e., the loss compensation component); specifically, the first discriminant dimension can be a regression loss discriminant dimension based on the similarity of the bounding box distribution, and the second discriminant dimension can be a regression loss discriminant dimension based on the degree of overlap of the bounding box coordinates.

[0105] In practical implementation, for multiple first reference bounding boxes that satisfy the above-mentioned preset constraints, the first weight coefficient and the third weight coefficient can remain unchanged. However, considering that the first regression loss component and the second regression loss component correspond to different regression loss discrimination dimensions (i.e., the regression loss discrimination dimension based on the similarity of bounding box distribution and the regression loss discrimination dimension based on the overlap of bounding box coordinates), and that the focus of regression loss considerations for different regression loss discrimination dimensions is also different (e.g., the regression loss discrimination dimension based on the similarity of bounding box distribution focuses on considering the regression loss of the first reference bounding box corresponding to the real bounding box with blurred bounding box edges, while the regression loss discrimination dimension based on the overlap of bounding box coordinates focuses on considering the regression loss of the first reference bounding box with similar bounding box distribution but different specific positions), the magnitude relationship between the first regression loss component and the second regression loss component reflects, to some extent, which regression loss discrimination dimension can more accurately characterize the regression loss between the real bounding box and the first predicted bounding box. Based on this, For each first reference bounding box, the first and third weight coefficients are adjusted according to the relationship between the first and second regression loss components corresponding to that first reference bounding box. Specifically, if the absolute value of the difference between the first and second regression loss components is not greater than a preset loss threshold, the first and third weight coefficients remain unchanged. If the absolute value of the difference between the first and second regression loss components is greater than the preset loss threshold and the first regression loss component is greater than the second regression loss component, the first weight coefficient is increased according to a first preset adjustment method. If the absolute value of the difference between the first and second regression loss components is greater than the preset loss threshold and the first regression loss component is less than the second regression loss component, the third weight coefficient is increased according to a second preset adjustment method. This achieves the effect of prioritizing the regression loss component corresponding to the discriminative dimension that better reflects the regression loss of the bounding box for each first reference bounding box during model training, thereby further improving the accuracy of model parameter optimization.

[0106] It should be noted that the increase in the first weight coefficient corresponding to the first preset adjustment method and the increase in the third weight coefficient corresponding to the second preset adjustment method can be the same or different. The increase in the weight coefficient can be set according to actual needs, and this application does not limit it.

[0107] Specifically, regarding the process of obtaining the first discrimination result by considering the similarity of bounding box distributions under the condition that the first predicted category satisfies the preset constraints, the above-mentioned boundary box authenticity judgment between the real bounding box corresponding to the first reference bounding box and the first predicted bounding box to obtain the first discrimination result includes:

[0108] Step A1: Based on the ground truth bounding box corresponding to the first reference bounding box, determine the first discrimination probability that the ground truth bounding box is judged as real by the above discrimination sub-model; and based on the first predicted bounding box corresponding to the first reference bounding box, determine the second discrimination probability that the first predicted bounding box is judged as fake by the above discrimination sub-model.

[0109] Step A2: Based on the first discrimination probability and the second discrimination probability corresponding to the first reference bounding box, generate the first discrimination result corresponding to the first reference bounding box.

[0110] Specifically, for a first reference bounding box that satisfies the preset category matching constraints for the first predicted category, the discriminant sub-model determines the probability that the true bounding box corresponding to the first reference bounding box comes from real data. That is, for the true bounding box, the discriminant sub-model performs a true / false judgment on the true bounding box to obtain the first discriminant probability that the predicted true bounding box is real data. Similarly, the discriminant sub-model determines the probability that the first predicted bounding box corresponding to the first reference bounding box comes from generated data (i.e., the value 1 minus the probability that the discriminant sub-model determines that the first predicted bounding box comes from real data). That is, for the first predicted bounding box, the discriminant sub-model performs a true / false judgment on the first predicted bounding box to obtain the second discriminant probability that the first predicted bounding box is generated data.

[0111] Specifically, the discriminator sub-model compares the first probability distribution corresponding to the true bounding box with the second probability distribution corresponding to the first predicted bounding box from the perspective of the similarity of the bounding box distributions. This allows for the determination of whether the true bounding box and the first predicted bounding box are true or false, resulting in a corresponding discrimination probability. This discrimination probability characterizes the distributional similarity between the true bounding box and the corresponding first predicted bounding box. Therefore, after determining the first and second discrimination probabilities, a first discrimination result can be obtained, which characterizes the distributional similarity of the bounding boxes. Based on the first discrimination result, the first regression loss component corresponding to the discrimination dimension characterizing the distributional similarity of the bounding boxes can be determined. The larger the first and second discrimination probabilities, the lower the distributional similarity between the true bounding box and the corresponding first predicted bounding box corresponding to the first reference bounding box. Therefore, the first regression loss component corresponding to the first reference bounding box is larger. Then, the model parameters of the generator sub-model are updated based on the first regression loss component. This allows the generator sub-model's generation result to optimize the loss value of the model to be trained after being predicted by the discriminator sub-model, thereby optimizing the generator sub-model and improving its bounding box prediction performance.

[0112] Furthermore, in order to improve the accuracy of the first discrimination result corresponding to each first reference bounding box, so as to improve the accuracy of the first regression loss component corresponding to the discrimination dimension of the similarity of the bounding box distribution during the process of determining the sub-loss value based on the first discrimination result, step A2 above, which generates the first discrimination result corresponding to the first reference bounding box based on the first discrimination probability and the second discrimination probability corresponding to the first reference bounding box, specifically includes:

[0113] Step A21: Based on the first discrimination probability and the first prior probability of the true bounding box corresponding to the first reference bounding box, determine the first weighted probability; and based on the second discrimination probability and the second prior probability of the first reference bounding box, determine the second weighted probability.

[0114] Step A22: Based on the first weighted probability and the second weighted probability corresponding to the first reference bounding box, generate the first discrimination result corresponding to the first reference bounding box.

[0115] Specifically, in determining the first discrimination result representing the similarity of bounding box distributions, the first prior probability of the true bounding box and the second prior probability of the first reference bounding box are considered. The discrimination sub-model is then used to distinguish between the true and false bounding boxes, and the first and second discrimination probabilities are weighted to determine the first discrimination result (i.e., the first discrimination result may include the first and second weighted probabilities). Therefore, the first regression loss component related to the similarity of bounding box distributions, obtained based on the first discrimination result, can be expressed as:

[0116]

[0117] in, P represents the prior probability (i.e., the first prior probability) of the i-th true bounding box. i1 Let represent the first discriminant probability that the i-th ground truth bounding box is predicted as true by the discriminant submodel. P represents the prior probability (i.e., the second prior probability) of the i-th first reference bounding box appearing. i2 It represents the second discrimination probability that the i-th first predicted bounding box is predicted as a fake by the discrimination sub-model.

[0118] It should be noted that, in specific implementation, This can be the prior probability of the i-th first reference bounding box appearing. Since the first predicted bounding box is obtained by the generative sub-model based on the first reference bounding box, therefore... It can also be the prior probability of the i-th first predicted bounding box appearing.

[0119] Specifically, since the probabilities of both the true bounding box and the predicted bounding box appearing follow a certain probability distribution, such as a Gaussian distribution, the first prior probability and the second prior probability can be obtained in the following way:

[0120]

[0121] in, Let represent the true bounding box corresponding to the first reference bounding box with index i, and σ1 represent the variance of the probability distribution of the first specified number of true bounding boxes. This represents the mean of the probability distribution of the first specified number of true bounding boxes.

[0122]

[0123] in, Let represent the first reference bounding box with index i, and σ² represent the variance of the probability distribution of the first specified number of first reference bounding boxes. This represents the mean of the probability distribution of the first specified number of first reference bounding boxes.

[0124] Specifically, the process of determining the second discrimination result that can characterize the classification loss component corresponding to the first reference bounding box, i.e., the process of obtaining the second discrimination result by considering the discrimination dimension of the class similarity of the bounding boxes, involves class discrimination between the true class and the first predicted class corresponding to the first reference bounding box to obtain the second discrimination result, which includes:

[0125] Step B1: Determine the third discrimination probability that the true class corresponding to the first reference bounding box is judged as true by the above discrimination sub-model; and determine the fourth discrimination probability that the first predicted class corresponding to the first reference bounding box is judged as fake by the above discrimination sub-model.

[0126] Step B2: Based on the third and fourth discrimination probabilities corresponding to the first reference bounding box, generate the second discrimination result corresponding to the first reference bounding box.

[0127] Specifically, for each first reference bounding box, the discriminant sub-model determines the probability that the true category corresponding to the first reference bounding box comes from the real data. That is, for the true category, the discriminant sub-model performs a true / false judgment on the true category to obtain the third discriminant probability that the true category is the real data. Similarly, the discriminant sub-model determines the probability that the first predicted category corresponding to the first reference bounding box comes from the generated data (i.e., the value 1 minus the probability that the discriminant sub-model determines that the first predicted category comes from the real data). That is, for the first predicted category, the discriminant sub-model performs a true / false judgment on the first predicted category to obtain the fourth discriminant probability that the first predicted category is the generated data.

[0128] Specifically, the discriminant sub-model compares the third probability distribution corresponding to the true class with the fourth probability distribution corresponding to the first predicted class from the perspective of class similarity, thereby distinguishing between the true and false classes and obtaining the corresponding discrimination probabilities. These discrimination probabilities characterize the class similarity between the actual class and the corresponding first predicted class. Therefore, after determining the third and fourth discrimination probabilities, a second discrimination result can be obtained, which characterizes the class similarity. Based on the second discrimination result, the classification loss component can be determined. The larger the third and fourth discrimination probabilities, the lower the class similarity between the true class and the corresponding first predicted class corresponding to the first reference bounding box. Therefore, the larger the classification loss component corresponding to the first reference bounding box, the larger the classification loss component. Then, the model parameters of the generator sub-model are updated based on the classification loss component. This allows the generation result of the generator sub-model to optimize the loss value of the model to be trained after being predicted by the discriminant sub-model, thereby optimizing the generator sub-model and improving the target object class recognition accuracy of the generator sub-model.

[0129] Furthermore, in order to improve the accuracy of the second discrimination result corresponding to each first reference bounding box, so as to improve the accuracy of the classification loss component corresponding to the discrimination dimension of the class similarity between the first predicted class and the true class during the process of determining the sub-loss value based on the second discrimination result, based on this, step B2 above generates the second discrimination result corresponding to the first reference bounding box based on the third and fourth discrimination probabilities corresponding to the first reference bounding box, specifically including:

[0130] B21, based on the aforementioned third discrimination probability and the third prior probability that the category of the true bounding box corresponding to the first reference bounding box is the true category, a third weighted probability is determined; and based on the aforementioned fourth discrimination probability and the fourth prior probability that the category of the first reference bounding box is the true category, a fourth weighted probability is determined.

[0131] B22, based on the third and fourth weighted probabilities corresponding to the first reference bounding box, generates the second discrimination result corresponding to the first reference bounding box.

[0132] Specifically, in determining the second discrimination result representing the class similarity, the third prior probability that the true bounding box belongs to the true class and the fourth prior probability that the first reference bounding box belongs to the true class are considered. The discrimination sub-model is then used to distinguish between the true and false categories of the true and first predicted categories, respectively. The resulting third and fourth discrimination probabilities are weighted to determine the second discrimination result (i.e., the second discrimination result may include the third and fourth weighted probabilities). Therefore, the classification loss component related to class similarity obtained based on the second discrimination result can be expressed as:

[0133]

[0134] in, P represents the prior probability (i.e., the third prior probability) of the true class corresponding to the i-th first reference bounding box. i3 This represents the third discriminant probability that the true class corresponding to the i-th first reference bounding box is predicted as true by the discriminant submodel. P represents the prior probability (i.e., the fourth prior probability) of the first predicted class corresponding to the i-th first reference bounding box. i4 This represents the fourth discrimination probability that the first predicted category corresponding to the i-th first reference bounding box is predicted as a fake by the discrimination sub-model.

[0135] Specifically, since the probability of a certain category appearing follows a certain probability distribution, such as a Gaussian distribution, the third prior probability and the fourth prior probability can be obtained in the following way:

[0136]

[0137] in, Let represent the true class corresponding to the first reference bounding box with index i, and σ3 represent the variance of the probability distribution of the true class corresponding to the first specified number of first reference bounding boxes. This represents the mean of the distribution probability of the true class corresponding to the first specified number of first reference bounding boxes.

[0138]

[0139] in, Let represent the first predicted category corresponding to the first reference bounding box with index i, and σ4 represent the variance of the probability distribution of the first predicted category corresponding to the first specified number of first reference bounding boxes. This represents the mean of the probability distribution of the first predicted category corresponding to the first specified number of first reference bounding boxes.

[0140] Specifically, the total loss value mentioned above is equal to the sum of the sub-loss values ​​corresponding to the first specified number of first reference bounding boxes, which can be expressed as follows:

[0141]

[0142] Where, N reg This indicates the first specified quantity, where i represents the index of the first reference bounding box, and the value of i ranges from 1 to N. reg .

[0143] Specifically, regarding the process of obtaining a third discrimination result by considering the degree of overlap of bounding box coordinates under the condition that the first predicted category meets the preset constraints, the above-mentioned calculation of the boundary box intersection-union ratio loss based on the real bounding box corresponding to the first reference bounding box and the first predicted bounding box to obtain the third discrimination result includes:

[0144] Step C1: Calculate the intersection-union ratio (IURR) loss for the ground truth bounding box corresponding to the first reference bounding box and the first predicted bounding box corresponding to the first reference bounding box to obtain the first IURR loss.

[0145] Specifically, if the first predicted category corresponding to the first reference bounding box with index i satisfies the preset category matching constraint, then the cross-union ratio loss between the true bounding box with index i and the first predicted bounding box with index i is calculated to obtain the first cross-union ratio loss corresponding to the first reference bounding box with index i.

[0146] Step C2: Based on the first intersection-union ratio loss, determine the third discrimination result corresponding to the first reference bounding box.

[0147] Specifically, since the cross-union ratio loss between two bounding boxes can characterize the degree of overlap of the bounding box coordinates, a third discrimination result can be obtained based on the cross-union ratio loss between the true bounding box and the first predicted bounding box. Then, based on the third discrimination result, the second regression loss component corresponding to the discrimination dimension considering the degree of overlap of the bounding box coordinates can be determined, thereby prompting the model to perform bounding box regression learning.

[0148] Furthermore, regarding the determination process of the third discrimination result, only the first intersection-union ratio (IU / R) loss between the true bounding box and its corresponding first predicted bounding box can be considered. However, in order to improve the accuracy of the third discrimination result determination, thereby improving the accuracy of the second regression loss component corresponding to the discrimination dimension from the perspective of the degree of overlap of bounding box coordinates, and thus improving the accuracy of the total loss value used to adjust the model parameters, not only the first IU / R loss between the true bounding box and its corresponding first predicted bounding box is considered, but also the second IU / R loss between the true bounding box and other first predicted bounding boxes is considered. This allows the true bounding box to be compared with positive examples (i.e., the first predicted bounding box corresponding to a true bounding box obtained through bounding box regression learning) and negative examples (i.e., the first predicted bounding boxes corresponding to other true bounding boxes obtained through bounding box regression learning, excluding a certain true bounding box) on the discrimination dimension of the degree of overlap of bounding box coordinates, to learn the specific position representation of the true bounding box, thereby prompting the model to perform better bounding box regression learning. Based on this, step C2 above, based on the above first IU / R loss, determines the second discrimination result corresponding to the above first initial bounding box, specifically including:

[0149] C21, in the first predicted bounding boxes corresponding to the first specified number of first reference bounding boxes, determine the set of comparison bounding boxes;

[0150] The aforementioned set of comparison bounding boxes includes other first predicted bounding boxes besides the first predicted bounding box corresponding to the first reference bounding box, or other first predicted bounding boxes that do not contain the target object enclosed by the first reference bounding box.

[0151] Specifically, taking the first reference bounding box with index i that satisfies the preset category matching constraint as an example, the above-mentioned set of comparison bounding boxes can include other first predicted bounding boxes besides the first predicted bounding box with index i (i.e., the first predicted bounding box with index k, k≠p, p=i). In other words, all other first predicted bounding boxes besides the first predicted bounding box with index i are used as negative examples of the real bounding box with index i. To further improve the accuracy of selecting negative examples, the above-mentioned set of comparison bounding boxes can include other first predicted bounding boxes besides the first predicted bounding box with index i, and the other first predicted bounding boxes do not contain the target object enclosed by the first reference bounding box with index i (i.e., the first predicted bounding box with index k, k≠p, p=i or p=j, the first predicted bounding box with index j is the same as the target object enclosed by the first reference bounding box with index i). In other words, only other first predicted bounding boxes that contain different target objects from the first reference bounding box with index i are used as negative examples of the real bounding box with index i.

[0152] C22, calculate the boundary box intersection-union ratio loss for the real boundary box corresponding to the first reference boundary box and the other first predicted boundary boxes, and obtain the second intersection-union ratio loss;

[0153] Specifically, taking the first reference bounding box with index i that satisfies the preset category matching constraint as an example, for each other first predicted bounding box in the set of comparison bounding boxes, the cross-union ratio loss between the real bounding box with index i and the first predicted bounding box with index k is calculated, and the second cross-union ratio loss corresponding to the first predicted bounding box with index k is obtained.

[0154] C23. Based on the first cross-union ratio loss and the second cross-union ratio loss, determine the third discrimination result corresponding to the first reference bounding box.

[0155] Specifically, in determining the third discrimination result representing the degree of overlap of bounding box coordinates, a first crossover ratio (CRR) loss is calculated based on the true bounding box with index i and the first predicted bounding box with index i, and a second CRR loss (k≠p) is calculated based on the true bounding box with index i and the first predicted bounding box with index k, to determine the third discrimination result (i.e., the third discrimination result can include the first and second CRR losses). Then, based on the third discrimination result, a second regression loss component related to the degree of overlap of bounding box coordinates can be determined. Adjusting the model parameters based on the second regression loss component can make the degree of overlap between the true bounding box with index i and the first predicted bounding box with index i higher, while making the degree of overlap with other first predicted bounding boxes lower, thereby enhancing the globality of bounding box regression learning and further improving the accuracy of bounding box regression learning.

[0156] In practical implementation, the aforementioned second regression loss component is the logarithm of the target cross-union ratio (CUN) loss. This target CUN loss is the quotient of the exponent of the first CUN loss and the sum of the exponents of multiple second CUN losses. That is, taking p=i as an example, the second regression loss component can be expressed as:

[0157]

[0158] in, This represents the actual bounding box corresponding to the first reference bounding box with index i. This represents the first reference bounding box with index i. This indicates that, under the condition that the first predicted category corresponding to the first reference bounding box with index i satisfies the preset constraints, This represents the first predicted bounding box corresponding to the first reference bounding box with index i, under the condition of satisfying preset constraints. Indicates the first intersection and comparison loss. This represents the first reference bounding box with index k. This represents the first predicted bounding box corresponding to the first reference bounding box with index k, under the condition that the preset constraints are met. θ represents the second crossover ratio loss. g1 This represents the first model parameter related to bounding box regression in the generated sub-model, and correspondingly, θ g2 This represents the second model parameter related to the prediction of the target object category, and ω represents the preset adjustment factor.

[0159] Furthermore, considering that in the object detection process, the generating sub-model needs to determine not only the location of the target object but also its specific category, the model parameters of the generating sub-model in the model to be trained include first model parameters related to bounding box regression and second model parameters related to target object category prediction. That is, the first and second model parameters need to be iteratively updated simultaneously during model training. Based on this, to further improve the accuracy of the total loss value, the aforementioned preset category matching constraint is introduced in the process of determining the sub-loss value corresponding to the first predicted bounding box. Only when the true category corresponding to the first predicted bounding box matches the first predicted category are the corresponding sub-regression loss value and sub-classification loss value considered; otherwise, only the sub-classification loss value is considered. The corresponding sub-classification loss value, i.e., the sub-regression loss value corresponding to the first reference bounding box whose category prediction result does not meet the preset requirements, is used in specific implementations. The generation sub-model performs category prediction on the first reference bounding box or the first prediction bounding box, and the output result can be the first category prediction result. The first category prediction result includes the predicted probability of the target object enclosed by the first reference bounding box or the first prediction bounding box belonging to each candidate category. The candidate category corresponding to the maximum predicted probability is the first predicted category. That is, the category of the target object enclosed by the first reference bounding box or the first prediction bounding box is predicted by the generation sub-model as the first predicted category. In other words, the category of the target object in the image region within the first reference bounding box or the first prediction bounding box is predicted by the generation sub-model as the first predicted category.

[0160] Additionally, it should be noted that in specific implementations, considering that the positional information of the first reference bounding box and the first predicted bounding box will not deviate significantly, and the image features within the first reference bounding box and the image features within the first predicted bounding box will not deviate significantly either, the recognition of the target object category within the image region of the bounding box will not be affected. Based on this, for the case where bounding box prediction and category prediction are performed sequentially, category prediction can be performed on the first predicted bounding box to obtain the corresponding first category prediction result. That is, the first predicted bounding box is first predicted based on the first reference bounding box, and then category prediction is performed on the first predicted bounding box to obtain the first category prediction result. For the case where bounding box prediction and category prediction are performed simultaneously, category prediction can also be performed on the first reference bounding box while performing bounding box prediction based on the first reference bounding box to obtain the corresponding first category prediction result. That is, the first predicted bounding box is predicted based on the first reference bounding box, and then category prediction is performed on the first reference bounding box to obtain the first category prediction result.

[0161] Specifically, the aforementioned preset category matching constraints may include: constraints for a single matching method or constraints for a variable matching method. The preset category matching constraints may be related to the prediction result of the first category. For constraints for a single matching method, the category matching constraints used in each round of model training remain unchanged (i.e., independent of the current model training round number). For example, for each round of model training, if the true category is the same as the first predicted category, then the first predicted category corresponding to the first reference bounding box is determined to match the true category (i.e., the first predicted category satisfies the aforementioned preset constraints, meaning the category of the target object in the first predicted bounding box is predicted by the generation sub-model as the target category matching the true category). For constraints for a variable matching method, the category matching constraints used in each round of model training are related to the current model training round number. Specifically, constraints for a variable matching method can be further divided into staged category matching constraints or gradual category matching constraints.

[0162] The aforementioned staged category matching constraint can be that when the current model training epoch is less than a first preset epoch, the true class and the first predicted class belong to the same category group, and when the current model training epoch is greater than or equal to the first preset epoch, the true class and the first predicted class are the same. That is, based on the staged category matching constraint and the first category prediction result corresponding to the first reference bounding box, staged category matching constraints can be achieved. The aforementioned gradual category matching constraint can be that the sum of the first constraint term and the second constraint term is greater than a preset probability threshold. The first constraint term is the first predicted probability corresponding to the true class in the category prediction probability subset, and the second constraint term is the product of the sum of the second predicted probabilities (excluding the first predicted probability) in the category prediction probability subset and a preset adjustment factor. The preset adjustment factor gradually decreases as the current training epoch increases. That is, based on the gradual category matching constraint and the first reference bounding box corresponding to the first preset epoch, the staged category matching constraint can be achieved. A single-class prediction result enables gradual class matching constraints. Specifically, a subset of class prediction probabilities is determined based on the first class prediction result corresponding to the first reference bounding box. This subset includes the first prediction probability that the target object enclosed by the first prediction bounding box belongs to the true class, and the second prediction probability that it belongs to a non-true class in the target group. That is, the subset of class prediction probabilities includes the first prediction probability under the true class in the target group and the second prediction probability under the non-true class (i.e., candidate class other than the true class in the target group) obtained by the generation sub-model performing class prediction on the first reference bounding box or the first prediction bounding box. The target group is the class group in which the true class belongs. In specific implementation, multiple candidate classes associated with the target detection task are pre-determined. Based on the semantic information of each candidate class, the multiple candidate classes are grouped to obtain multiple class groups.

[0163] Specifically, since the first reference bounding box is obtained by extracting the region of interest using a preset region of interest extraction model, there may be cases where the region of the target object enclosed by the first reference bounding box is not accurate enough, which may lead to inaccurate category identification of the first predicted bounding box corresponding to such a first reference bounding box in the early stage of model training. Based on this, in the process of determining the sub-loss value corresponding to the first reference bounding box, the matching relationship between the first predicted category corresponding to the first reference bounding box and the true category of the first reference bounding box is referenced, that is, the first predicted bounding box is determined to satisfy the above preset constraints based on the above preset category matching constraints.

[0164] Furthermore, since the generated sub-model is used for both bounding box prediction and target object category prediction, iterative training is required during model training for the first model parameters related to bounding box regression and the second model parameters related to target object category prediction in the generated sub-model. Considering that the accuracy of the model parameters related to target object category prediction in the generated sub-model might be low in the early stages of model training, leading to inaccurate category identification for the first predicted bounding box corresponding to the first reference bounding box, the requirement for category accuracy is relaxed in the early stages of model training. If the true category corresponding to the first predicted bounding box and the first predicted category belong to the same category group, the above-mentioned preset constraint is satisfied, and the corresponding sub-regression loss value is considered. In the later stages of model training, the requirement for category accuracy is tightened. Only if the true category corresponding to the first predicted bounding box is the same as the first predicted category is satisfied, and the corresponding sub-regression loss value is considered. Based on this, the above-mentioned preset category matching constraints can include: constraints for the changing matching method (such as staged category matching constraints or gradual category matching constraints).

[0165] Furthermore, to ensure a smoother transition between the two categories of the preset category matching constraint that restricts the first predicted category to satisfy the above-mentioned preset constraint (i.e., the first predicted category belongs to the target group, and the first predicted category is the same as the true category), so that as the number of model training rounds increases, the preset category matching constraint gradually changes from restricting the first predicted category to fall into the target group to restricting the first predicted category to be the same as the true category, therefore, preferably, the above-mentioned preset category matching constraint includes: a category matching gradual constraint.

[0166] In practical implementation, for the case where the above-mentioned preset category matching constraint is a category matching gradient constraint, taking the first reference bounding box with index i as an example, the category matching gradient constraint can be expressed as:

[0167]

[0168] Where groups represent the target group, real i f represents the true class of the first reference bounding box with index i in the target group groups, where f ∈ groups\real i This represents the non-true category in the target group, and β represents the prediction moderating factor. This represents the first predicted probability (i.e., the first constraint term mentioned above). Indicates the second predicted probability. This represents the second constraint term mentioned above, where μ represents the preset probability threshold; specifically, The larger the value, the closer the first predicted class is to the true class. Since the preset adjustment factor decreases with each training epoch, the reference proportion of the second constraint term gradually decreases. This means that in the later stages of model training, the first constraint term (i.e., the first predicted probability under the true class) primarily determines whether the first predicted class matches the true class. Then, after a certain number of training epochs, the second constraint term becomes zero. When the probability exceeds the preset probability threshold, it indicates that the generated sub-model has determined the true class as the first predicted class.

[0169] Specifically, regarding the aforementioned preset adjustment factor, it decreases as the current model training epochs increase. If the current model training epochs are less than or equal to the target training epochs, the aforementioned second constraint term is positively correlated with the preset adjustment factor, and the aforementioned preset adjustment factor is negatively correlated with the current model training epochs. If the current model training epochs are greater than the target training epochs, the aforementioned second constraint term is zero, wherein the target training epochs are less than the total number of training epochs.

[0170] In practical implementation, to ensure the smoothness of the adjustment of the preset adjustment factor, a linear decreasing adjustment method can be used to gradually reduce the value of the preset adjustment factor β. Therefore, the process for determining the preset adjustment factor used in the current model training is as follows:

[0171] (1) For the first round of model training, the first preset value is determined as the preset adjustment factor used for the current model training;

[0172] Specifically, the first preset value can be set according to actual needs. To simplify the adjustment complexity, the first preset value can be set to 1, that is, the preset adjustment factor β = 1. In other words, in the case of the first round of model training, the above-mentioned gradual constraint condition for category matching can be:

[0173] Right now

[0174] In other words, for the first round of model training, based on the sum of the first and second predicted probabilities corresponding to the target group, it is determined whether the first predicted category corresponding to the first reference bounding box matches the true category.

[0175] (2) For non-first round model training, the preset adjustment factor used for the current model training is determined according to the factor decreasing adjustment method, based on the current model training round number, the target training round number and the above-mentioned first preset value.

[0176] Specifically, if the preset adjustment factor β = 1 for the first round of model training, then in the case of subsequent rounds of model training, the above-mentioned gradual category matching constraint can be:

[0177]

[0178] In other words, for non-first-round model training, the above-mentioned gradual category matching constraints... Furthermore, as the number of training rounds increases, the second constraint term... Their participation gradually decreased.

[0179] For example, the decreasing formula corresponding to the above factor decreasing adjustment method can be:

[0180]

[0181] in, express Take the maximum value between 0 and 0. The first item 1 represents the first preset value (i.e., the preset adjustment factor β used in the first round of training), δ represents the current training epoch of the model, and Z represents the target training epoch. That is, the target training epoch can be the total training epoch minus 1, or it can be a specified training epoch. The specified training epoch is less than the total training epoch. The difference between the total training epoch and the specified training epoch is the preset epoch Q. Q is greater than 2. That is, in the later stages of model training (not the last epoch), the preset adjustment factor β is set to 0. In other words, the judgment condition used in the later stages of model training, from δ = Z + 1 epochs to the last epoch, is...

[0182] It should be noted that, for the case where the target training round number Z is the total training round number minus 1, the above decreasing formula can be: In the final round of model training, the preset adjustment factor is set to 0, meaning that all decision criteria used in the final round of model training are... Furthermore, the decreasing formula shown above is only a relatively simple linear decreasing adjustment method. In practical applications, the decreasing rate of the preset adjustment factor β can be set according to actual needs. Therefore, the decreasing formula above does not constitute a limitation on the scope of protection of this application.

[0183] Furthermore, in practical implementation, the aforementioned preset category matching constraints can be related not only to the prediction result of the first category, but also to the third and fourth discriminant probabilities corresponding to the first reference bounding box. Specifically, considering that the sub-classification loss value (i.e., the classification loss component) corresponding to the first reference bounding box is determined based on the aforementioned third and fourth discriminant probabilities, and that the third and fourth discriminant probabilities can reflect the class similarity between the first predicted category and the true category, the aforementioned preset category matching constraints can also be related to the third and fourth discriminant probabilities corresponding to the first reference bounding box. Specifically, the constraint of the aforementioned single matching method can also be that the third discriminant probability is less than the first value and the fourth discriminant probability is less than the second value (i.e., it is unrelated to the current model training epochs). The smaller the third and fourth discriminant probabilities, the smaller the corresponding sub-classification loss value, indicating that the discriminant sub-classification loss is stronger. The more difficult it is for the model to distinguish between the true and false of the first predicted category and the true category, that is, the more similar the first predicted category and the true category are, the more likely the category of the target object in the first predicted bounding box is predicted by the generating sub-model as the target category that matches the true category. Correspondingly, the constraint condition of the above-mentioned changing matching method can also be that when the current model training rounds are less than the first preset rounds, the third discrimination probability is less than the third value and the fourth discrimination probability is less than the fourth value, and when the current model training rounds are greater than or equal to the first preset rounds, the third discrimination probability is less than the first value and the fourth discrimination probability is less than the second value (that is, it is related to the current model training rounds), the third value is greater than the first value, and the fourth value is greater than the second value. That is, as the number of model training rounds increases, the first and second values ​​that determine whether the preset category matching constraint condition is met become smaller, requiring the first predicted category to be closer to the true category, thereby realizing the staged category matching constraint.

[0184] Specifically, regarding the process of determining the aforementioned loss compensation value, based on the ground truth bounding box and the first predicted bounding box corresponding to the first reference bounding box, a loss compensation value is calculated to constrain the loss gradient of the loss function of the model to be trained, resulting in the fourth discrimination result, which includes:

[0185] Step D1: Based on the real bounding box and the first predicted bounding box corresponding to the first reference bounding box, generate the synthetic bounding box corresponding to the first reference bounding box.

[0186] Specifically, taking the first reference bounding box with index i as an example, according to the preset coordinate information sampling method, based on the first coordinate information set corresponding to the real bounding box with index i and the second coordinate information set corresponding to the first predicted bounding box with index i, the sampled coordinate information set is determined; based on the sampled coordinate information set, the synthetic bounding box with index i is determined.

[0187] Step D2: Determine the loss compensation value based on the similarity of the bounding box distribution between the synthetic bounding box corresponding to the first reference bounding box and the real bounding box.

[0188] Step D3: Based on the above loss compensation value, generate the fourth discrimination result corresponding to the first reference bounding box.

[0189] Specifically, after determining the composite bounding box corresponding to the first reference bounding box with index i... Then, the similarity in bounding box distribution between the composite bounding box with index i and the ground truth bounding box with index i is calculated, i.e. Next, calculate the compensation gradient with respect to the similarity of the bounding box distributions, i.e. Based on the L2 norm of the compensation gradient matrix, the loss compensation value corresponding to the first reference bounding box with index i is determined.

[0190] Specifically, regarding the process of determining the synthetic bounding box corresponding to a certain first reference bounding box, step D1 above, based on the ground truth bounding box and the first predicted bounding box corresponding to the first reference bounding box, generates the synthetic bounding box corresponding to the first reference bounding box, specifically including:

[0191] Step D11: Based on the first sampling ratio and the first set of first coordinate information of the real bounding box corresponding to the first reference bounding box, determine the first coordinate information subset;

[0192] Step D12: Based on the second sampling ratio and the second coordinate information set of the first predicted bounding box corresponding to the first reference bounding box, determine the second coordinate information subset; it should be noted that the first sampling ratio and the second sampling ratio can be preset according to the actual situation, and the sum of the first sampling ratio and the second sampling ratio is equal to 1.

[0193] Step D13: Based on the first subset of coordinate information and the second subset of coordinate information, generate the composite bounding box corresponding to the first reference bounding box.

[0194] Specifically, taking the first reference bounding box with index i as an example, according to the first sampling ratio, a subset of first coordinate information is randomly sampled from the first coordinate information set of the real bounding box with index i; and according to the second sampling ratio, a subset of second coordinate information is randomly sampled from the second coordinate information set of the first predicted bounding box with index i; the combination of the first and second coordinate information subsets is determined as the sampled coordinate information set, and the bounding box drawn based on the sampled coordinate information set is the synthetic bounding box with index i; wherein, since the synthetic bounding box is based on the coordinate information of the real bounding box with index i (i.e., the real data) and the first reference bounding box with index i, the second reference bounding box is randomly sampled from the second coordinate information set of the real bounding box with index i. The coordinate information of the predicted bounding box (i.e., generated data) is randomly sampled and mixed to obtain the bounding box. Therefore, part of the coordinate information of the synthesized bounding box comes from the real data, and the other part comes from the generated data. That is, the synthesized bounding box is determined by both the real data and the generated data and has a certain degree of randomness. This can compensate for the gradient of the total loss value when the gradient of the regression loss corresponding to the first discriminant dimension suddenly decreases or even becomes zero. This avoids the problem of the gradient of the total loss value suddenly decreasing due to the sudden decrease or even becoming zero of the gradient of the regression loss corresponding to the first discriminant dimension during model training, thereby further improving the training accuracy of the model parameters.

[0195] In practical implementation, the aforementioned model to be trained includes a generative sub-model and a discriminative sub-model, such as... Figure 4b As shown, a schematic diagram illustrating the specific implementation principle of another object detection model training process is presented, which includes:

[0196] (1) Use a preset region of interest extraction model to extract the target region from a preset set of sample images to obtain X anchor boxes;

[0197] (2) For each round of model training, N anchor boxes are randomly sampled from X anchor boxes as first reference bounding boxes, and the real bounding box and real class corresponding to each first reference bounding box are determined respectively.

[0198] (3) For each first reference bounding box, the generation sub-model performs bounding box prediction based on the first reference bounding box to obtain the first predicted bounding box, and performs target category prediction on the image region in the first reference bounding box or the first predicted bounding box to obtain the first category prediction result; and inputs the first predicted bounding box, the corresponding first category prediction result, the ground bounding box and the corresponding ground category into the discrimination sub-model; the discrimination sub-model generates a discrimination result set based on the ground bounding box and the first predicted bounding box corresponding to the first reference bounding box, the ground category and the first category prediction result corresponding to the first reference bounding box, and a certain preset category matching constraint condition;

[0199] Specifically, based on preset category matching constraints, it is determined whether the first predicted category meets the preset constraints. If the first predicted category does not meet the preset constraints, the discrimination result set includes a second discrimination result and a fourth discrimination result (at this time, the first and third discrimination results can be empty or preset information). Correspondingly, the classification loss component is determined based on the second discrimination result in the discrimination result set of the first reference bounding box, and the loss compensation component is determined based on the fourth discrimination result in the discrimination result set of the first reference bounding box. Then, the sub-loss value corresponding to the first reference bounding box is determined based on the classification loss component and the loss compensation component. If the first predicted category meets the preset constraints, the discrimination result set... The system includes a first discrimination result, a second discrimination result, a third discrimination result, and a fourth discrimination result. Correspondingly, a first regression loss component is determined based on the first discrimination result in the discrimination result set of the first reference bounding box, a classification loss component is determined based on the second discrimination result in the discrimination result set of the first reference bounding box, a second regression loss component is determined based on the third discrimination result in the discrimination result set of the first reference bounding box, and a loss compensation component is determined based on the fourth discrimination result in the discrimination result set of the first reference bounding box. Then, a sub-loss value corresponding to the first reference bounding box is determined based on the first regression loss component, the classification loss component, the second regression loss component, and the loss compensation component.

[0200] It should be noted that, since the discrimination sub-model considers whether the first predicted class and the true class meet the preset class matching constraints (i.e., whether the first predicted class meets the above preset constraints) during the process of generating the discrimination result set corresponding to each first reference bounding box, in the case where the first predicted class and the true class do not meet the preset class matching constraints, it is only necessary to perform true / false discrimination on the first predicted class and the true class to obtain the second discrimination result, and calculate the loss compensation value to obtain the fourth discrimination result, without having to perform true / false discrimination on the first predicted bounding box and the true bounding box to obtain the first discrimination result, nor is it necessary to calculate the intersection-union ratio loss on the first predicted bounding box and the true bounding box to obtain the third discrimination result. That is, the first discrimination result and the third discrimination result can be directly determined as empty or preset information, which can further improve the model training efficiency.

[0201] In other words, in determining the discrimination result set corresponding to the first reference bounding box, one can directly perform a true / false judgment on the first predicted bounding box and the true bounding box to obtain the first discrimination result, perform a true / false judgment on the first predicted category and the true category to obtain the second discrimination result, calculate the intersection-union ratio loss on the first predicted bounding box and the true bounding box to obtain the third discrimination result, and calculate the loss compensation value to obtain the fourth discrimination result, thus generating a discrimination result set; then, based on whether the first predicted category satisfies the above-mentioned preset constraints, it is determined whether to consider the first regression loss component and the second regression loss component corresponding to the first reference bounding box, that is, to determine whether the corresponding sub-regression loss value is zero; or, one can first determine whether to directly set the corresponding first discrimination result and the third discrimination result to empty or preset information based on whether the first predicted category satisfies the above-mentioned preset constraints, obtain the discrimination result set, and then determine the corresponding sub-loss value based on the discrimination result set;

[0202] Additionally, it should be noted that in specific implementation, the true category corresponding to each first reference bounding box can also be input into the generating sub-model. The generating sub-model determines whether the first predicted category meets the preset constraints based on the preset category matching constraints, the true category, and the first category prediction result. If the first predicted category does not meet the preset constraints, only the sub-classification loss value corresponding to the first reference bounding box is considered, and there is no need to calculate its corresponding sub-regression loss value. Therefore, it is not necessary to perform bounding box prediction on the first reference bounding box, which can further reduce the data processing volume of the generating sub-model.

[0203] (4) Based on the sub-loss values ​​corresponding to each first reference bounding box, determine the total loss value of the model to be trained; use the stochastic gradient descent method to adjust the model parameters of the above-mentioned generating sub-model and discriminant sub-model based on the total loss value to obtain the generating sub-model and discriminant sub-model with updated parameters;

[0204] (5) If the current model iteration training result meets the model iteration training termination condition, the above-updated generator sub-model is determined as the trained target detection model; if the current model iteration training result meets the model iteration training termination condition, the above-updated generator sub-model and discriminator sub-model are determined as the training models to be used in the next round of model training, until the model iteration training termination condition is met.

[0205] The model training method in this embodiment, during the model training phase, outputs a first discrimination result representing the similarity of the bounding box distribution by distinguishing the sub-model based on the real bounding box and the first predicted bounding box obtained from the first reference bounding box. This prompts the model parameters related to bounding box regression to be continuously updated, thereby enabling the generating sub-model to continuously learn the bounding box distribution and make the predicted first bounding box closer to the real bounding box. This improves the accuracy, generalization, and data transferability of the trained object detection model in predicting the bounding box of the target object's location. Furthermore, during the model training phase, a second discrimination result representing the class similarity between the first predicted class and the real class is also output by distinguishing the sub-model based on the real class and the first predicted class corresponding to the first reference bounding box. This prompts the generating sub-model to continuously learn the bounding box distribution, making the predicted first bounding box closer to the real bounding box. The model continuously learns the category of the target object in the image region within the bounding box, making the predicted first category closer to the true category. Since the model training process does not focus on the predicted category itself, but instead uses the true / false discrimination results of the discriminant sub-model, it prompts the continuous updating of model parameters related to the target object category prediction, thereby improving the target classification accuracy, model generalization, and data transferability of the trained target detection model. Furthermore, the discrimination result set output by the discriminant sub-model includes the first discrimination result and the second discrimination result. Based on this discrimination result set, the total loss value of the model to be trained is determined, and the model parameters of the generator sub-model and the discriminant sub-model are continuously updated in multiple rounds based on the total loss value, thus ensuring the accuracy of target object location labeling and target object classification in the target detection process.

[0206] Corresponding to the above Figures 1 to 4b Based on the same technical concept, the model training method described in this application also provides an object detection method. Figure 5 This is a flowchart illustrating the target detection method provided in an embodiment of this application. Figure 5 The method described above can be executed by an electronic device equipped with an object detection device. This electronic device can be a terminal device or a designated server. The hardware device used for object detection (i.e., the electronic device equipped with the object detection device) and the hardware device used for training the object detection model (i.e., the electronic device equipped with the model training device) can be the same or different. Figure 5 As shown, the method includes at least the following steps:

[0207] S502, obtain a third specified number of second reference bounding boxes from the second candidate bounding box set; wherein, the second candidate bounding box set is obtained by extracting the target region of the image to be detected using a preset region of interest extraction model, and the third specified number can be M;

[0208] Specifically, the process of obtaining the third specified number of second reference bounding boxes can refer to the process of obtaining the first specified number of first reference bounding boxes described above, and will not be repeated here.

[0209] S504, the second reference bounding box is input into the object detection model for object detection to obtain the second predicted bounding box and the second predicted category corresponding to each second reference bounding box; wherein, the object detection model is trained based on the above model training method, and the specific training process of the object detection model is described in the above embodiment, and will not be repeated here.

[0210] Specifically, the aforementioned object detection model includes a generative sub-model. For each second reference bounding box: during the object detection process, the generative sub-model makes predictions based on the second reference bounding box to obtain a second predicted bounding box and a second predicted category corresponding to the second reference bounding box. Since the model parameters of the generative sub-model include first model parameters related to bounding box regression and second model parameters related to object classification, the generative sub-model can not only predict the bounding box based on the second reference bounding box to obtain the second predicted bounding box corresponding to the second reference bounding box, but also predict the category based on the second reference bounding box to obtain the second predicted category corresponding to the second reference bounding box.

[0211] In practical implementation, the generation sub-model performs category prediction on the aforementioned second reference bounding box or the aforementioned second predicted bounding box, and the output result can be the second category prediction result. The second category prediction result includes the predicted probability that the target object enclosed by the second reference bounding box or the second predicted bounding box belongs to each candidate category. The candidate category corresponding to the highest predicted probability is the second predicted category. That is, the category of the target object enclosed by the second reference bounding box or the second predicted bounding box is predicted as the second predicted category by the generation sub-model; that is, the category of the target object in the image region within the second reference bounding box or the second predicted bounding box is predicted as the second predicted category by the generation sub-model. Furthermore, in practical implementation, considering that the positional information of the second reference bounding box and the second predicted bounding box will not deviate significantly, the image features within the second reference bounding box and the second predicted bounding box... The image features within the predicted bounding box will not deviate significantly. Therefore, it will not affect the recognition of the target object category within the image region of the bounding box. Based on this, for the case where bounding box prediction and category prediction are performed sequentially, category prediction can be performed on the second predicted bounding box to obtain the corresponding second category prediction result. That is, the second predicted bounding box is first predicted based on the second reference bounding box, and then the category prediction is performed on the second predicted bounding box to obtain the second category prediction result. For the case where bounding box prediction and category prediction are performed simultaneously, the category prediction can also be performed on the second reference bounding box while performing bounding box prediction based on the second reference bounding box to obtain the corresponding second category prediction result. That is, the second predicted bounding box is predicted based on the second reference bounding box, and then the category prediction is performed on the second reference bounding box to obtain the second category prediction result.

[0212] S506, Based on the second predicted bounding box and the second predicted category corresponding to each second reference bounding box, generate the target detection result of the image to be detected.

[0213] Specifically, based on the second predicted bounding box and the second predicted category corresponding to each second reference bounding box, the number of target objects contained in the image to be detected and the category to which each target object belongs can be determined. For example, the image to be detected contains a cat, a dog and a pedestrian.

[0214] In practical implementation, the aforementioned target detection model includes a generation sub-model, such as... Figure 6 The diagram illustrates the specific implementation principle of a target detection process, including:

[0215] A preset region of interest extraction model is used to extract the target region of the image to be detected, resulting in P anchor boxes;

[0216] M anchor boxes are randomly sampled from P anchor boxes as the second reference bounding box;

[0217] For each second reference bounding box, a sub-model is generated to predict the bounding box based on the second reference bounding box to obtain the second predicted bounding box, and to predict the category of the second predicted bounding box to obtain the second predicted category.

[0218] Based on the second predicted bounding box and the second predicted category corresponding to each second reference bounding box, the target detection result of the image to be detected is generated.

[0219] It should be noted that the target detection model trained based on the above model training method can be applied to any specific application scenario that requires target detection of the image to be detected. The image to be detected can be acquired by an image acquisition device set up at a certain location. Correspondingly, the target detection device can belong to the image acquisition device, specifically it can be the image processing device in the image acquisition device. The image processing device receives the image to be detected transmitted by the image acquisition device in the image acquisition device and performs target detection on the image to be detected. Alternatively, the target detection device can be a separate target detection device independent of the image acquisition device. The target detection device receives the image to be detected from the image acquisition device and performs target detection on the image to be detected.

[0220] Specifically, for specific application scenarios of object detection, for example, the image to be detected may be acquired by an image acquisition device set up at the entrance of a public place (such as a shopping mall entrance, subway entrance, scenic spot entrance, or performance venue entrance, etc.). Correspondingly, the target object to be detected in the image is the target user entering the public place. The above-mentioned object detection model is used to perform object detection on the image to be detected, so as to delineate a second predicted bounding box containing the target user entering the public place in the image to be detected, and determine the second predicted category corresponding to the second predicted bounding box (i.e., the category to which the target user contained in the second predicted bounding box belongs, such as age group, gender, body size, etc.). The target detection model obtains the target detection result of the image to be detected by selecting a target detection model (at least one of the following: high-level, professional, etc.); then, based on the target detection result, the user group identification result is determined (such as the flow of people entering the public place, or the attributes of the user group entering the public place, etc.); furthermore, based on the user group identification result, the corresponding business processing is executed (such as automatically triggering the entry restriction prompt operation, or pushing information to the target user, etc.); wherein, the higher the accuracy of the model parameters of the above target detection model, the higher the accuracy of the target detection result of the image to be detected output by the target detection model, and therefore, the higher the accuracy of triggering the execution of the corresponding business processing based on the target detection result.

[0221] For example, the image to be detected can be acquired by image acquisition devices set up at various monitoring points in a certain breeding base. Correspondingly, the target object to be detected in the image is the target breeding object within the breeding monitoring point. The above-mentioned target detection model is used to perform target detection on the image to be detected, so as to delineate a second predicted bounding box containing the target breeding object in the image to be detected, and determine the second predicted category corresponding to the second predicted bounding box (i.e., the category to which the target breeding object contained in the second predicted bounding box belongs, such as at least one of live status and body size), to obtain the target detection result of the image to be detected; then, based on the target detection result, the breeding pair is determined. The target detection model identifies the target population (e.g., the survival rate or growth rate of the target animals within the monitoring point). Based on this, corresponding control measures are implemented (e.g., if a decrease in survival rate is detected, an alarm is automatically issued; if a slowdown in growth rate is detected, the amount or frequency of feeding is automatically increased). The higher the accuracy of the model parameters of the target detection model, the higher the accuracy of the target detection results in the output image. Therefore, the accuracy of triggering corresponding control measures based on the target detection results is also higher.

[0222] The target detection method in this embodiment first extracts multiple candidate bounding boxes using a preset region of interest extraction model during the target detection process. Then, a certain number of candidate bounding boxes are randomly sampled from these candidate bounding boxes as second reference bounding boxes. For each second reference bounding box, a sub-model performs bounding box prediction and category prediction based on that second reference bounding box, obtaining a second predicted bounding box and a second predicted category. Then, based on the second predicted bounding box and the second predicted category corresponding to each second reference bounding box, the target detection result of the image to be detected is generated. During the training process of the sub-model's model parameters, a first discrimination result representing the similarity of bounding box distribution is output by distinguishing between the sub-model based on the real bounding box and the first predicted bounding box obtained from the first reference bounding box. This prompts the model parameters related to bounding box regression to be continuously updated, enabling the sub-model to continuously learn the bounding box distribution, making the predicted first predicted bounding box closer to the real bounding box, thereby improving the accuracy of the trained target detection model in predicting the bounding box of the target object's location. The model exhibits excellent generalization and data transferability. Furthermore, during model training, the discriminator sub-model outputs a second discrimination result representing the class similarity between the first predicted class and the true class, based on the true class and the first predicted class corresponding to the first reference bounding box. This prompts the generator sub-model to continuously learn the class of the target object within the image region of the bounding box, making the predicted first class closer to the true class. Since the model training process does not focus on the predicted class itself, but rather uses the true / false discrimination result of the discriminator sub-model to continuously update the model parameters related to the target object class prediction, the target classification accuracy, model generalization, and data transferability of the trained target detection model are improved. Moreover, the discrimination result set output by the discriminator sub-model includes both the first and second discrimination results. Based on this set of discrimination results, the total loss value of the model to be trained is determined, and the model parameters of the generator and discriminator sub-models are iteratively updated multiple times based on the total loss value, simultaneously ensuring the accuracy of target object location labeling and target object classification during the target detection process.

[0223] It should be noted that this embodiment in this application is based on the same inventive concept as the previous embodiment in this application. Therefore, the specific implementation of this embodiment can be referred to the implementation of the aforementioned model training method, and the repeated parts will not be described again.

[0224] Corresponding to the above Figures 1 to 4b Based on the same technical concept, this application also provides a model training apparatus for the described model training method. Figure 7 This is a schematic diagram of the module composition of the model training device provided in the embodiments of this application. The device is used to perform... Figures 1 to 4b The model training method described, such as Figure 7 As shown, the device includes:

[0225] The first bounding box acquisition module 702 is configured to acquire a first specified number of first reference bounding boxes from a first candidate bounding box set, and to acquire the ground truth bounding box and the ground truth category corresponding to each first reference bounding box; the first candidate bounding box set is obtained by extracting target regions from a preset sample image set using a preset region of interest extraction model.

[0226] The model training module 704 is configured to input the first reference bounding box, the ground truth bounding box, and the ground truth class into the model to be trained for iterative training until the current model iterative training result meets the model iterative training termination condition, thereby obtaining an object detection model; wherein, the model to be trained includes a generator sub-model and a discriminator sub-model; the specific implementation of each model training is as follows:

[0227] For each first reference bounding box: the generative sub-model makes predictions based on the first reference bounding box to obtain a first predicted bounding box and a first predicted category; the discriminative sub-model generates a discrimination result set based on the ground truth bounding box and the first predicted bounding box corresponding to the first reference bounding box, and the ground truth category and the first predicted category corresponding to the first reference bounding box; the discrimination result set includes a first discrimination result and a second discrimination result, wherein the first discrimination result represents the degree of similarity in the bounding box distribution between the first predicted bounding box and the ground truth bounding box under the condition of satisfying a preset constraint, wherein the preset constraint is that the category of the target object in the first predicted bounding box is predicted by the generative sub-model to be a target category that matches the ground truth category, and the second discrimination result represents the degree of category similarity between the first predicted category and the ground truth category; based on the first discrimination result and the second discrimination result corresponding to each first reference bounding box, the total loss value of the model to be trained is determined; the model parameters of the generative sub-model and the discriminative sub-model are updated based on the total loss value.

[0228] The model training device in this embodiment, during the model training phase, outputs a first discrimination result representing the similarity of the bounding box distribution by distinguishing the sub-model based on the real bounding box and the first predicted bounding box obtained from the first reference bounding box. This prompts the model parameters related to bounding box regression to be continuously updated, thereby enabling the generated sub-model to continuously learn the bounding box distribution and make the predicted first predicted bounding box closer to the real bounding box. This improves the accuracy, model generalization, and data transferability of the trained object detection model in predicting the bounding box of the target object's location. Furthermore, during the model training phase, it also outputs a second discrimination result representing the class similarity between the first predicted class and the real class by distinguishing the sub-model based on the real class and the first predicted class corresponding to the first reference bounding box. This prompts the generated sub-model to continuously learn the bounding box distribution, making the predicted first predicted bounding box closer to the real bounding box. The model continuously learns the category of the target object in the image region within the bounding box, making the predicted category closer to the true category. Since the model training does not focus on the predicted category itself, but rather on the true / false discrimination results of the discriminant sub-model, the model parameters related to the target object category prediction are continuously updated, improving the target classification accuracy, model generalization, and data transferability of the trained target detection model. Furthermore, the discrimination result set output by the discriminant sub-model includes the first and second discrimination results. Based on this discrimination result set, the total loss value of the model to be trained is determined, and the model parameters of the generator sub-model and the discriminant sub-model are iteratively updated multiple times based on the total loss value, thus ensuring the accuracy of target object location labeling and target object classification in the target detection process.

[0229] It should be noted that the embodiments of the model training device in this application and the specific embodiments of the model training method in this application are based on the same inventive concept. Therefore, the specific implementation of this embodiment can be referred to the implementation of the corresponding model training method mentioned above, and the repeated parts will not be described again.

[0230] Corresponding to the above Figures 5 to 6 Based on the same technical concept, this application also provides a target detection device in its embodiments regarding the target detection method described. Figure 8 This is a schematic diagram of the module composition of the target detection device provided in the embodiments of this application. The device is used to perform... Figures 5 to 6 The described object detection method, such as Figure 8 As shown, the device includes:

[0231] The second bounding box acquisition module 802 is configured to acquire a third specified number of second reference bounding boxes from the second candidate bounding box set; the second candidate bounding box set is obtained by extracting the target region of the image to be detected using a preset region of interest extraction model.

[0232] The target detection module 804 is configured to input the second reference bounding box into the target detection model to perform target detection, and obtain a second predicted bounding box and a second predicted category corresponding to each second reference bounding box;

[0233] The detection result generation module 806 is configured to generate the target detection result of the image to be detected based on the second predicted bounding box and the second predicted category corresponding to each second reference bounding box.

[0234] In the target detection device of this application embodiment, during the target detection process, multiple candidate bounding boxes are first extracted using a preset region of interest extraction model. Then, a certain number of candidate bounding boxes are randomly sampled from these candidate bounding boxes as second reference bounding boxes. For each second reference bounding box, a sub-model performs bounding box prediction and category prediction based on that second reference bounding box, obtaining a second predicted bounding box and a second predicted category. Then, based on the second predicted bounding box and the second predicted category corresponding to each second reference bounding box, a target detection result for the image to be detected is generated. During the training process of the sub-model's model parameters, a first discrimination result representing the similarity of bounding box distribution is output by distinguishing between the sub-model based on the real bounding box and the first predicted bounding box obtained from the first reference bounding box. This prompts the model parameters related to bounding box regression to be continuously updated, enabling the sub-model to continuously learn the bounding box distribution, making the predicted first predicted bounding box closer to the real bounding box, thereby improving the accuracy of the trained target detection model in predicting the bounding box of the target object's location. The model exhibits excellent generalization and data transferability. Furthermore, during model training, the discriminator sub-model outputs a second discrimination result representing the class similarity between the first predicted class and the true class, based on the true class and the first predicted class corresponding to the first reference bounding box. This prompts the generator sub-model to continuously learn the class of the target object within the image region of the bounding box, making the predicted first class closer to the true class. Since the model training process does not focus on the predicted class itself, but rather uses the true / false discrimination result of the discriminator sub-model to continuously update the model parameters related to the target object class prediction, the target classification accuracy, model generalization, and data transferability of the trained target detection model are improved. Moreover, the discrimination result set output by the discriminator sub-model includes both the first and second discrimination results. Based on this set of discrimination results, the total loss value of the model to be trained is determined, and the model parameters of the generator and discriminator sub-models are iteratively updated multiple times based on the total loss value, simultaneously ensuring the accuracy of target object location labeling and target object classification during the target detection process.

[0235] It should be noted that the embodiments of the target detection device in this application and the specific embodiments of the target detection method in this application are based on the same inventive concept. Therefore, the specific implementation of this embodiment can be referred to the implementation of the corresponding target detection method mentioned above, and the repeated parts will not be described again.

[0236] Furthermore, corresponding to the above Figures 1 to 6 Based on the same technical concept, embodiments of this application also provide a computer device for executing the aforementioned model training method or object detection method, such as... Figure 9 As shown.

[0237] Computer devices can vary significantly due to differences in configuration or performance. They may include one or more processors 901 and memory 902, with memory 902 storing one or more application programs or data. Memory 902 can be temporary or persistent storage. The application programs stored in memory 902 may include one or more modules (not shown), each module including a series of computer-executable instructions for the computer device. Furthermore, processor 901 may be configured to communicate with memory 902 and execute the series of computer-executable instructions stored in memory 902 on the computer device. The computer device may also include one or more power supplies 903, one or more wired or wireless network interfaces 904, one or more input / output interfaces 905, one or more keyboards 906, etc.

[0238] In one specific embodiment, the computer device includes a memory and one or more programs, wherein the one or more programs are stored in the memory, and the one or more programs may include one or more modules, and each module may include a series of computer-executable instructions for use in the computer device, and is configured to be executed by one or more processors. The one or more programs include computer-executable instructions for performing the following:

[0239] A first specified number of first reference bounding boxes are obtained from a first candidate bounding box set, and the ground truth bounding box and the ground truth class corresponding to each first reference bounding box are obtained respectively; the first candidate bounding box set is obtained by extracting target regions from a preset sample image set using a preset region of interest extraction model;

[0240] The first reference bounding box, the ground truth bounding box, and the ground truth class are input into the model to be trained for iterative training until the current model iteration training result meets the model iteration training termination condition, thus obtaining the object detection model; wherein, the model to be trained includes a generator sub-model and a discriminator sub-model; the specific implementation method of each model training is as follows:

[0241] For each first reference bounding box: the generation sub-model makes predictions based on the first reference bounding box to obtain a first predicted bounding box and a first predicted category; the discrimination sub-model generates a discrimination result set based on the ground truth bounding box and the first predicted bounding box corresponding to the first reference bounding box, and the ground truth category and the first predicted category corresponding to the first reference bounding box; the discrimination result set includes a first discrimination result and a second discrimination result, wherein the first discrimination result represents the degree of similarity in the bounding box distribution between the first predicted bounding box and the ground truth bounding box under the condition of satisfying a preset constraint, wherein the preset constraint is that the category of the target object in the first predicted bounding box is predicted by the generation sub-model to be a target category that matches the ground truth category, and the second discrimination result represents the degree of category similarity between the first predicted category and the ground truth category;

[0242] Based on the first and second discrimination results corresponding to each of the first reference bounding boxes, the total loss value of the model to be trained is determined;

[0243] The model parameters of the generating sub-model and the discriminant sub-model are updated based on the total loss value.

[0244] In another specific embodiment, the computer device includes a memory and one or more programs, wherein the one or more programs are stored in the memory, and the one or more programs may include one or more modules, and each module may include a series of computer-executable instructions for use in the computer device, and is configured to be executed by one or more processors. The one or more programs include computer-executable instructions for performing the following:

[0245] Obtain a third specified number of second reference bounding boxes from the second candidate bounding box set; the second candidate bounding box set is obtained by extracting the target region of the image to be detected using a preset region of interest extraction model;

[0246] The second reference bounding box is input into the target detection model for target detection, and the second predicted bounding box and the second predicted category corresponding to each second reference bounding box are obtained.

[0247] Based on the second predicted bounding box and the second predicted category corresponding to each second reference bounding box, the target detection result of the image to be detected is generated.

[0248] In the model training phase, the computer device in this application, by distinguishing between the sub-model based on the real bounding box and the first predicted bounding box obtained from the first reference bounding box, outputs a first discrimination result representing the similarity of the bounding box distribution. This prompts the model parameters related to bounding box regression to be continuously updated, thereby enabling the generating sub-model to continuously learn the bounding box distribution and make the predicted first bounding box closer to the real bounding box. This improves the accuracy, model generalization, and data transferability of the trained object detection model in predicting the bounding box of the target object's location. Furthermore, during the model training phase, the device also outputs a second discrimination result representing the class similarity between the first predicted class and the real class by distinguishing between the sub-model based on the real class and the first predicted class corresponding to the first reference bounding box. This prompts the generating sub-model to continuously learn the bounding box distribution, making the predicted first bounding box closer to the real bounding box. The model continuously learns the category of the target object in the image region within the bounding box, making the predicted category closer to the true category. Since the model training does not focus on the predicted category itself, but rather on the true / false discrimination results of the discriminant sub-model, the model parameters related to the target object category prediction are continuously updated, improving the target classification accuracy, model generalization, and data transferability of the trained target detection model. Furthermore, the discrimination result set output by the discriminant sub-model includes the first and second discrimination results. Based on this discrimination result set, the total loss value of the model to be trained is determined, and the model parameters of the generator sub-model and the discriminant sub-model are iteratively updated multiple times based on the total loss value, thus ensuring the accuracy of target object location labeling and target object classification in the target detection process.

[0249] It should be noted that the embodiments concerning computer devices in this application and the specific embodiments concerning model training methods in this application are based on the same inventive concept. Therefore, the specific implementation of this embodiment can be referred to the implementation of the corresponding model training method described above, and the repeated parts will not be described again.

[0250] Furthermore, corresponding to the above Figures 1 to 6 Based on the same technical concept, this application also provides a storage medium for storing computer-executable instructions. In one specific embodiment, the storage medium can be a USB flash drive, optical disc, hard disk, etc. When the computer-executable instructions stored in the storage medium are executed by a processor, they can achieve the following process:

[0251] A first specified number of first reference bounding boxes are obtained from a first candidate bounding box set, and the ground truth bounding box and the ground truth class corresponding to each first reference bounding box are obtained respectively; the first candidate bounding box set is obtained by extracting target regions from a preset sample image set using a preset region of interest extraction model;

[0252] The first reference bounding box, the ground truth bounding box, and the ground truth class are input into the model to be trained for iterative training until the current model iteration training result meets the model iteration training termination condition, thus obtaining the object detection model; wherein, the model to be trained includes a generator sub-model and a discriminator sub-model; the specific implementation method of each model training is as follows:

[0253] For each first reference bounding box: the generation sub-model makes predictions based on the first reference bounding box to obtain a first predicted bounding box and a first predicted category; the discrimination sub-model generates a discrimination result set based on the ground truth bounding box and the first predicted bounding box corresponding to the first reference bounding box, and the ground truth category and the first predicted category corresponding to the first reference bounding box; the discrimination result set includes a first discrimination result and a second discrimination result, wherein the first discrimination result represents the degree of similarity in the bounding box distribution between the first predicted bounding box and the ground truth bounding box under the condition of satisfying a preset constraint, wherein the preset constraint is that the category of the target object in the first predicted bounding box is predicted by the generation sub-model to be a target category that matches the ground truth category, and the second discrimination result represents the degree of category similarity between the first predicted category and the ground truth category;

[0254] Based on the first and second discrimination results corresponding to each of the first reference bounding boxes, the total loss value of the model to be trained is determined;

[0255] The model parameters of the generating sub-model and the discriminant sub-model are updated based on the total loss value.

[0256] In another specific embodiment, the storage medium can be a USB flash drive, optical disc, hard disk, etc., and the computer-executable instructions stored on the storage medium can achieve the following process when executed by the processor:

[0257] Obtain a third specified number of second reference bounding boxes from the second candidate bounding box set; the second candidate bounding box set is obtained by extracting the target region of the image to be detected using a preset region of interest extraction model;

[0258] The second reference bounding box is input into the target detection model for target detection, and the second predicted bounding box and the second predicted category corresponding to each second reference bounding box are obtained.

[0259] Based on the second predicted bounding box and the second predicted category corresponding to each second reference bounding box, the target detection result of the image to be detected is generated.

[0260] In the embodiments of this application, the computer-executable instructions stored in the storage medium, when executed by the processor, during the model training phase, output a first discrimination result representing the similarity of the bounding box distribution by distinguishing the sub-model based on the real bounding box and the first predicted bounding box obtained by the first reference bounding box. This prompts the model parameters related to bounding box regression to be continuously updated, thereby enabling the generating sub-model to continuously learn the bounding box distribution and make the predicted first predicted bounding box closer to the real bounding box. This improves the accuracy, model generalization, and data transferability of the trained object detection model in predicting the bounding box of the target object's location. Furthermore, during the model training phase, a second discrimination result representing the class similarity between the first predicted class and the real class is also output by distinguishing the sub-model based on the real class and the first predicted class corresponding to the first reference bounding box. The results encourage the generator sub-model to continuously learn the category of the target object in the image region within the bounding box, making the predicted first category closer to the true category. Since the model training does not focus on the predicted category itself, but rather on the true / false discrimination results of the discriminator sub-model, the model parameters related to the target object category prediction are continuously updated, improving the target classification accuracy, model generalization, and data transferability of the trained target detection model. Furthermore, the discrimination result set output by the discriminator sub-model includes the first and second discrimination results. Based on this discrimination result set, the total loss value of the model to be trained is determined, and the model parameters of the generator sub-model and the discriminator sub-model are iteratively updated multiple times based on the total loss value, thus ensuring the accuracy of target object location labeling and target object classification in the target detection process.

[0261] It should be noted that the embodiments concerning storage media in this application and the specific embodiments concerning model training methods in this application are based on the same inventive concept. Therefore, the specific implementation of this embodiment can be referred to the implementation of the corresponding model training method described above, and the repeated parts will not be described again.

[0262] The foregoing has described specific embodiments of this application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired results. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0263] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, embodiments of this application can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0264] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0265] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0266] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0267] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0268] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0269] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0270] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0271] The embodiments of this application can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. One or more embodiments of this application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In a distributed computing environment, program modules can reside in local and remote computer storage media, including storage devices.

[0272] The various embodiments in this application are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0273] The above description is merely an embodiment of this document and is not intended to limit the scope of this document. Various modifications and variations can be made to this document by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this document should be included within the scope of the claims of this document.

Claims

1. A model training method, characterized in that, The method includes: A first specified number of first reference bounding boxes are obtained from a first candidate bounding box set, and the ground truth bounding boxes and ground truth categories corresponding to each first reference bounding box are obtained respectively; the first candidate bounding box set is obtained by extracting target regions from a preset sample image set using a preset region of interest extraction model; The first reference bounding box, the ground truth bounding box, and the ground truth class are input into the model to be trained for iterative training until the current model iteration training result meets the model iteration training termination condition, thus obtaining the object detection model; wherein, the model to be trained includes a generator sub-model and a discriminator sub-model; the specific implementation method of each model training is as follows: For each first reference bounding box: the generation sub-model makes predictions based on the first reference bounding box to obtain a first predicted bounding box and a first predicted category; the discrimination sub-model generates a discrimination result set based on the ground truth bounding box and the first predicted bounding box corresponding to the first reference bounding box, and the ground truth category and the first predicted category corresponding to the first reference bounding box; the discrimination result set includes a first discrimination result and a second discrimination result, wherein the first discrimination result represents the degree of similarity in the bounding box distribution between the first predicted bounding box and the ground truth bounding box under the condition of satisfying a preset constraint, wherein the preset constraint is that the category of the target object in the first predicted bounding box is predicted by the generation sub-model to be a target category that matches the ground truth category, and the second discrimination result represents the degree of category similarity between the first predicted category and the ground truth category; Based on the first and second discrimination results corresponding to each of the first reference bounding boxes, the total loss value of the model to be trained is determined; The model parameters of the generating sub-model and the discriminant sub-model are updated based on the total loss value.

2. The method according to claim 1, characterized in that, The step of generating a discrimination result set based on the ground truth bounding box and the first predicted bounding box corresponding to the first reference bounding box, and the ground truth class and the first predicted class corresponding to the first reference bounding box, includes: Under the condition that the first predicted category satisfies the preset constraint, the true and false boundary boxes corresponding to the first reference boundary box are judged to obtain a first judgment result; the true and false category corresponding to the first reference boundary box are judged to obtain a second judgment result.

3. The method according to claim 2, characterized in that, The discrimination result set further includes a third discrimination result; the step of generating the discrimination result set based on the ground truth bounding box and the first predicted bounding box corresponding to the first reference bounding box, and the ground truth class and the first predicted class corresponding to the first reference bounding box further includes: Under the condition that the first predicted category satisfies the preset constraint, the boundary box intersection-union ratio loss is calculated based on the real boundary box corresponding to the first reference boundary box and the first predicted boundary box to obtain the third discrimination result.

4. The method according to claim 2, characterized in that, The discrimination result set further includes a fourth discrimination result; the generation of the discrimination result set based on the ground truth bounding box and the first predicted bounding box corresponding to the first reference bounding box, and the ground truth class and the first predicted class corresponding to the first reference bounding box further includes: Based on the ground truth bounding box and the first predicted bounding box corresponding to the first reference bounding box, a loss compensation value is calculated to constrain the loss gradient of the loss function of the model to be trained, and a fourth discrimination result is obtained.

5. The method according to claim 3 or 4, characterized in that, The step of determining the total loss value of the model to be trained based on the first and second discrimination results corresponding to each of the first reference bounding boxes includes: Determine the sub-loss value corresponding to each of the first reference bounding boxes; the sub-loss value corresponding to the first reference bounding box is determined based on target information, the target information including at least one of the following: whether the first predicted category corresponding to the first reference bounding box satisfies the preset constraint, the degree of similarity of the bounding box distribution represented by the first discrimination result corresponding to the first reference bounding box, the degree of similarity of the categories represented by the second discrimination result, the degree of overlap of the bounding box coordinates represented by the third discrimination result, and the loss compensation value represented by the fourth discrimination result; The total loss value of the model to be trained is determined based on the sub-loss value corresponding to each of the first reference bounding boxes.

6. The method according to claim 2, characterized in that, The step of determining the authenticity of the true bounding box and the first predicted bounding box corresponding to the first reference bounding box to obtain a first determination result includes: Based on the ground truth bounding box corresponding to the first reference bounding box, a first discrimination probability is determined that the ground truth bounding box is judged as real by the discrimination sub-model; based on the first predicted bounding box corresponding to the first reference bounding box, a second discrimination probability is determined that the first predicted bounding box is judged as fake by the discrimination sub-model. A first discrimination result is generated based on the first discrimination probability and the second discrimination probability.

7. The method according to claim 6, characterized in that, The step of generating a first discrimination result based on the first discrimination probability and the second discrimination probability includes: Based on the first discrimination probability and the first prior probability of the true bounding box corresponding to the first reference bounding box, a first weighted probability is determined. The second weighted probability is determined based on the second discrimination probability and the second prior probability of the first reference bounding box; A first discrimination result is generated based on the first weighted probability and the second weighted probability.

8. The method according to claim 2, characterized in that, The step of performing a class authenticity determination on the true category and the first predicted category corresponding to the first reference bounding box to obtain a second determination result includes: Determine a third discrimination probability that the true category corresponding to the first reference bounding box is judged as true by the discrimination sub-model; determine a fourth discrimination probability that the first predicted category corresponding to the first reference bounding box is judged as fake by the discrimination sub-model; A second discrimination result is generated based on the third discrimination probability and the fourth discrimination probability.

9. The method according to claim 8, characterized in that, The step of generating a second discrimination result based on the third discrimination probability and the fourth discrimination probability includes: The third weighted probability is determined based on the third discrimination probability and the third prior probability that the category of the true bounding box corresponding to the first reference bounding box is the true category; Based on the fourth discrimination probability and the fourth prior probability that the category of the first reference bounding box is the true category, a fourth weighted probability is determined; A second discrimination result is generated based on the third weighted probability and the fourth weighted probability.

10. The method according to claim 4, characterized in that, The step of calculating a loss compensation value to constrain the loss gradient of the loss function of the model to be trained, based on the ground truth bounding box and the first predicted bounding box corresponding to the first reference bounding box, to obtain a fourth discrimination result includes: Based on the ground bounding box and the first predicted bounding box corresponding to the first reference bounding box, a synthetic bounding box corresponding to the first reference bounding box is generated. The loss compensation value is determined based on the similarity of the bounding box distribution between the synthetic bounding box corresponding to the first reference bounding box and the real bounding box. Based on the loss compensation value, a fourth discrimination result corresponding to the first reference bounding box is generated.

11. The method according to claim 10, characterized in that, The step of generating a synthetic bounding box corresponding to the first reference bounding box based on the ground truth bounding box and the first predicted bounding box includes: Based on the first sampling ratio and the first set of first coordinate information of the real bounding box corresponding to the first reference bounding box, a subset of first coordinate information is determined; Based on the second sampling ratio and the second coordinate information set of the first predicted bounding box corresponding to the first reference bounding box, a subset of second coordinate information is determined; the sum of the first sampling ratio and the second sampling ratio is equal to 1; Based on the first subset of coordinate information and the second subset of coordinate information, a composite bounding box corresponding to the first reference bounding box is generated.

12. The method according to claim 1, characterized in that, The method further includes: A preset set of sample images is input into a preset region of interest extraction model to extract the region of interest, resulting in a first set of candidate bounding boxes; the first set of candidate bounding boxes includes a second specified number of candidate bounding boxes, the second specified number being greater than the first specified number. The step of obtaining a first specified number of first reference bounding boxes from the first candidate bounding box set includes: randomly selecting a first specified number of the candidate bounding boxes as first reference bounding boxes from the second specified number of candidate bounding boxes.

13. A target detection method, characterized in that, The method includes: Obtain a third specified number of second reference bounding boxes from the second candidate bounding box set; the second candidate bounding box set is obtained by extracting the target region of the image to be detected using a preset region of interest extraction model; The second reference bounding box is input into the target detection model for target detection to obtain the second predicted bounding box and the second predicted category corresponding to each second reference bounding box. The target detection model is trained based on the model training method of any one of claims 1-12. Based on the second predicted bounding box and the second predicted category corresponding to each second reference bounding box, the target detection result of the image to be detected is generated.

14. The method according to claim 13, characterized in that, The target detection model includes a generation sub-model; For each second reference bounding box: during the target detection process, the generating sub-model makes predictions based on the second reference bounding box to obtain a second predicted bounding box and a second predicted category corresponding to the second reference bounding box.

15. A model training device, characterized in that, The device includes: The first bounding box acquisition module is configured to acquire a first specified number of first reference bounding boxes from a first candidate bounding box set, and to acquire the ground truth bounding box and the ground truth category corresponding to each first reference bounding box; the first candidate bounding box set is obtained by extracting target regions from a preset sample image set using a preset region of interest extraction model; The model training module is configured to input the first reference bounding box, the ground truth bounding box, and the ground truth class into the model to be trained for iterative training until the current model iteration training result meets the model iteration training termination condition, thereby obtaining an object detection model; wherein, the model to be trained includes a generator sub-model and a discriminator sub-model; the specific implementation of each model training is as follows: For each first reference bounding box: the generative sub-model makes predictions based on the first reference bounding box to obtain a first predicted bounding box and a first predicted category; the discriminative sub-model generates a discrimination result set based on the ground truth bounding box and the first predicted bounding box corresponding to the first reference bounding box, and the ground truth category and the first predicted category corresponding to the first reference bounding box; the discrimination result set includes a first discrimination result and a second discrimination result, wherein the first discrimination result represents the degree of similarity in the bounding box distribution between the first predicted bounding box and the ground truth bounding box under the condition of satisfying a preset constraint, wherein the preset constraint is that the category of the target object in the first predicted bounding box is predicted by the generative sub-model to be a target category that matches the ground truth category, and the second discrimination result represents the degree of category similarity between the first predicted category and the ground truth category; based on the first discrimination result and the second discrimination result corresponding to each first reference bounding box, the total loss value of the model to be trained is determined; the model parameters of the generative sub-model and the discriminative sub-model are updated based on the total loss value.

16. A target detection device, characterized in that, The device includes: The second bounding box acquisition module is configured to acquire a third specified number of second reference bounding boxes from the second candidate bounding box set; the second candidate bounding box set is obtained by extracting the target region of the image to be detected using a preset region of interest extraction model; The object detection module is configured to input the second reference bounding box into the object detection model for object detection, and obtain a second predicted bounding box and a second predicted category corresponding to each second reference bounding box. The object detection model is trained based on the model training method of any one of claims 1-12. The detection result generation module is configured to generate target detection results for the image to be detected based on the second predicted bounding box and the second predicted category corresponding to each second reference bounding box.

17. A computer device, characterized in that, The device includes: Processor; and A memory configured to store computer-executable instructions configured to be executed by the processor, the executable instructions including steps for performing the method as described in any one of claims 1-12 or 13-14.

18. A storage medium, characterized in that, The storage medium is used to store computer-executable instructions that cause the computer to perform the method as described in any one of claims 1-12 or 13-14.

Images