Model training methods, object detection methods and devices

Abstract

This application provides a model training method, an object detection method, and an apparatus. The method includes: during the model training phase, by using a first reference bounding box and its corresponding actual bounding box, the object detection model to be trained is prompted to continuously learn the bounding box distribution, so that the predicted first bounding box is closer to the corresponding actual bounding box, thereby improving the bounding box prediction accuracy, model generalization, and data transferability of the trained object detection model; and the set of comparison results used to determine the bounding box regression loss value includes not only the first comparison result representing the similarity of the bounding box distribution, but also the second comparison result representing the degree of overlap of the bounding box coordinates, so that the accuracy of the bounding box regression loss value obtained based on the first comparison result and the second comparison result is higher, thereby further improving the accuracy of the model parameters updated based on the bounding box regression loss value.

Description

Technical Field

[0001] This application relates to the field of object detection, and in particular to a model training method, an object detection method, and an apparatus. Background Technology

[0002] Currently, with the rapid development of artificial intelligence technology, there is an increasing demand for using pre-trained target detection models to detect targets in an image, thereby predicting the coordinates and classification information of the bounding boxes containing each target in the image.

[0003] However, in the training process of object detection models in related technologies, the main focus is on learning the similarity between the predicted bounding box and the image features in the real bounding box. As a result, the accuracy of the model parameters of the trained object detection model is relatively high for sample image datasets, but the accuracy of the model parameters of the trained object detection model will decrease for the target image. This leads to poor generalization of the object detection model, and consequently, low object detection accuracy in the application stage. Summary of the Invention

[0004] The purpose of this application is to provide a model training method, object detection method, and apparatus that enable the predicted first bounding box to be closer to the corresponding actual bounding box, thereby improving the bounding box prediction accuracy, model generalization, and data transferability of the trained object detection model; and to make the bounding box regression loss value obtained based on the first and second alignment results more accurate, thereby further improving the accuracy of the model parameters updated based on the bounding box regression loss value.

[0005] To achieve the above technical solution, the embodiments of this application are implemented as follows:

[0006] In a first aspect, embodiments of this application provide a model training method, the method comprising:

[0007] A first subset of bounding boxes is obtained from a first set of candidate bounding boxes, and the actual bounding boxes corresponding to each first reference bounding box in the first subset of bounding boxes are obtained respectively; the first subset of bounding boxes includes a first specified number of first reference bounding boxes, and the first set of candidate bounding boxes is obtained by extracting the target region from the sample image dataset using a preset region of interest extraction model.

[0008] The first reference bounding box and the actual bounding box are input into the target detection model to be trained for iterative training until the result of this iteration meets the preset termination condition, thus obtaining the trained target detection model; wherein, the target detection model includes a bounding box prediction sub-model; the specific implementation of each model training is as follows:

[0009] For each of the first reference bounding boxes: the bounding box prediction sub-model performs bounding box prediction based on the first reference bounding box to obtain a first predicted bounding box; based on the actual bounding box corresponding to the first reference bounding box and the first predicted bounding box corresponding to the first reference bounding box, a bounding box comparison result set is generated; the bounding box comparison result set includes a first comparison result characterizing the similarity of the bounding box distribution and a second comparison result characterizing the overlap of the bounding box coordinates;

[0010] Based on the first alignment result and the second alignment result corresponding to the first reference bounding box in the first bounding box subset, the bounding box regression loss value is determined.

[0011] The parameters of the bounding box prediction sub-model are updated based on the bounding box regression loss value.

[0012] Secondly, this application provides a target detection method, the method comprising:

[0013] Obtain a subset of second bounding boxes corresponding to the image to be detected from the set of second candidate bounding boxes; the subset of second bounding boxes includes a third specified number of second reference bounding boxes, and the set of second candidate bounding boxes is obtained by extracting the target region of the image to be detected using a preset region of interest extraction model;

[0014] The second reference bounding box is input into the target detection model for target detection, and the second predicted bounding box and the second category prediction result corresponding to each second reference bounding box are obtained.

[0015] Based on the second predicted bounding box and the second category prediction result corresponding to each second reference bounding box, the target detection result of the image to be detected is generated.

[0016] Thirdly, this application provides a model training apparatus, the apparatus comprising:

[0017] The bounding box acquisition module is configured to acquire a first bounding box subset from a first candidate bounding box set, and to acquire the actual bounding boxes corresponding to each first reference bounding box in the first bounding box subset; the first bounding box subset includes a first specified number of first reference bounding boxes, and the first candidate bounding box set is obtained by extracting the target region from the sample image dataset using a preset region of interest extraction model.

[0018] The model training module is configured to input the first reference bounding box and the actual bounding box into the target detection model to be trained for iterative training until the result of this iteration meets the preset model iteration training termination condition, thus obtaining the trained target detection model; wherein, the target detection model includes a bounding box prediction sub-model; the specific implementation of each model training is as follows:

[0019] For each first reference bounding box: the bounding box prediction sub-model predicts bounding boxes based on the first reference bounding boxes to obtain first predicted bounding boxes; based on the actual bounding boxes corresponding to the first reference bounding boxes and the first predicted bounding boxes corresponding to the first reference bounding boxes, a bounding box comparison result set is generated; the bounding box comparison result set includes a first comparison result characterizing the similarity of bounding box distributions and a second comparison result characterizing the overlap of bounding box coordinates; based on the first comparison result and the second comparison result corresponding to the first reference bounding boxes in the first bounding box subset, a bounding box regression loss value is determined; the parameters of the bounding box prediction sub-model are updated based on the bounding box regression loss value.

[0020] Fourthly, an embodiment of this application provides a target detection device, the device comprising:

[0021] The bounding box acquisition module is configured to acquire a second bounding box subset corresponding to the image to be detected from a second candidate bounding box set; the second bounding box subset includes a third specified number of second reference bounding boxes, and the second candidate bounding box set is obtained by extracting the target region of the image to be detected using a preset region of interest extraction model.

[0022] The target detection module is configured to input the second reference bounding box into the target detection model for target detection, and obtain the second predicted bounding box and the second category prediction result corresponding to each second reference bounding box;

[0023] The detection result generation module is configured to generate target detection results for the image to be detected based on the second predicted bounding box corresponding to each second reference bounding box and the second category prediction result.

[0024] Fifthly, an embodiment of this application provides a computer device, the device comprising:

[0025] A processor; and a memory arranged to store computer-executable instructions configured to be executed by the processor, the executable instructions including steps for performing the methods described in the first or second aspect.

[0026] Sixthly, embodiments of this application provide a storage medium for storing computer-executable instructions that cause a computer to perform steps in the methods described in the first or second aspect.

[0027] As can be seen in the embodiments of this application, during the model training phase, the bounding box prediction sub-model predicts the first predicted bounding box based on the first reference bounding box. Then, based on the first predicted bounding box and its corresponding actual bounding box, the target detection model to be trained continuously learns the bounding box distribution, making the predicted first predicted bounding box closer to the corresponding actual bounding box, thereby improving the bounding box prediction accuracy, model generalization, and data transferability of the trained target detection model. Furthermore, the set of comparison results used to determine the bounding box regression loss value includes not only the first comparison result representing the similarity of the bounding box distribution, but also the second comparison result representing the degree of overlap of the bounding box coordinates. The bounding box regression loss value is then obtained based on the first and second comparison results corresponding to each first reference bounding box. This makes the bounding box regression loss value include the regression loss obtained based on the coarse-grained comparison dimension of the similarity of the bounding box distribution and the regression loss obtained based on the fine-grained comparison dimension of the degree of overlap of the bounding box coordinates. This can improve the accuracy of the bounding box regression loss value, thereby further improving the accuracy of the model parameters updated based on the bounding box regression loss value. Attached Figure Description

[0028] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in one or more of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0029] Figure 1 A schematic flowchart illustrating the model training method provided in this application embodiment;

[0030] Figure 2 A flowchart illustrating each model training process in the model training method provided in the embodiments of this application;

[0031] Figure 3 A schematic diagram illustrating the first implementation principle of the model training method provided in this application embodiment;

[0032] Figure 4a A schematic diagram illustrating the second implementation principle of the model training method provided in the embodiments of this application;

[0033] Figure 4bA schematic diagram illustrating the third implementation principle of the model training method provided in the embodiments of this application;

[0034] Figure 5 A schematic flowchart of the target detection method provided in the embodiments of this application;

[0035] Figure 6 A schematic diagram illustrating the implementation principle of the target detection method provided in the embodiments of this application;

[0036] Figure 7 This is a schematic diagram of the module composition of the model training device provided in the embodiments of this application;

[0037] Figure 8 This is a schematic diagram of the module composition of the target detection device provided in the embodiments of this application;

[0038] Figure 9 A schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0039] To enable those skilled in the art to better understand the technical solutions in one or more of this application, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of one or more of this application, and not all embodiments. Based on the embodiments of one or more of this application, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of this application.

[0040] It should be noted that, unless otherwise specified, one or more embodiments and features described in this application can be combined with each other. The embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0041] Considering that using deep networks to extract features and then teaching the model to learn image features within bounding boxes, continuously learning the similarity between predicted and actual bounding box features, and adjusting model parameters, would make the trained object detection model highly dependent on the sample dataset used during training. This would result in poor generalization and data transfer capabilities, inevitably leading to high accuracy on the sample dataset but low accuracy on new image data. Therefore, during model training, a bounding box prediction sub-model predicts a first predicted bounding box based on a first reference bounding box. Then, based on this first predicted bounding box and its corresponding actual bounding box, the trained object detection model continuously learns the bounding box distribution, making the predicted first bounding box closer to the corresponding actual bounding box. This not only improves the accuracy of the trained object detection model in predicting the bounding box of the target object's location in the image but also enhances its generalization and data transfer capabilities. This ensures accurate object detection on new images and improves the model's data transfer adaptability. Furthermore, considering that determining the model regression loss solely based on coarse-grained comparisons of bounding box distribution similarity and adjusting model parameters would fail to address the precise location learning of the bounding boxes, and determining the model regression loss solely based on fine-grained comparisons of bounding box coordinate overlap would fail to address the edge ambiguity issue, the set of comparison results used to determine the bounding box regression loss value includes not only the first comparison result representing the similarity of the bounding box distribution but also the second comparison result representing the overlap of the bounding box coordinates. Then, based on the first comparison result corresponding to each first reference bounding box and... The second comparison result yields the bounding box regression loss value, achieving the effect of simultaneously considering the regression loss brought by bounding boxes with similar distributions but different specific locations, as well as the regression loss brought by the first predicted bounding box corresponding to the actual bounding box with ambiguous edges. This makes the bounding box regression loss value include the regression loss obtained from the coarse-grained comparison dimension based on the similarity of the bounding box distribution, and the regression loss obtained from the fine-grained comparison dimension based on the degree of overlap of the bounding box coordinates. This can improve the accuracy of the bounding box regression loss value, thereby further improving the accuracy of the model parameters updated based on the bounding box regression loss value.

[0042] Figure 1 This is a flowchart illustrating a model training method provided in one or more embodiments of this application. Figure 1The method described above can be executed by an electronic device equipped with a model training device, which can be a terminal device or a designated server. The hardware device used for model training (i.e., the electronic device equipped with the model training device) and the hardware device for object detection (i.e., the electronic device equipped with the object detection device) can be the same or different. Specifically, the training process for the object detection model, such as... Figure 1 As shown, it includes at least the following steps:

[0043] S102, obtain a first bounding box subset from the first candidate bounding box set, and obtain the actual bounding box corresponding to each first reference bounding box in the first bounding box subset; wherein, the first bounding box subset includes a first specified number of first reference bounding boxes, and the first candidate bounding box set is obtained by extracting the target region from the sample image dataset using a preset region of interest extraction model;

[0044] Specifically, the process of determining the first specified number of first reference bounding boxes can be as follows: for each round of model training, perform a step of extracting the target region of the sample image dataset using a preset region of interest extraction model to obtain the first specified number of first reference bounding boxes; or, perform the step of extracting the target region of the sample image dataset using a preset region of interest extraction model in advance, and then, for each round of model training, randomly sample from a large number of pre-extracted candidate bounding boxes to obtain the first specified number of first reference bounding boxes.

[0045] Specifically, the sample image dataset may contain multiple sample target objects, and each sample target object may correspond to multiple first reference bounding boxes. That is, a first specified number of first reference bounding boxes contain at least one first reference bounding box corresponding to each sample target object.

[0046] Specifically, in step S102 above, before obtaining the first subset of bounding boxes from the first set of candidate bounding boxes, the method further includes: inputting the sample image dataset into a preset region of interest extraction model to extract the region of interest, thereby obtaining the first set of candidate bounding boxes; wherein, the first set of candidate bounding boxes includes a second specified number of candidate bounding boxes; the second specified number is greater than the first specified number, that is, for the case where the second specified number is equal to the first specified number, for each round of model training, the preset region of interest extraction model is used to extract the region of interest from multiple sample image data in the sample image dataset, thereby obtaining the first specified number of first reference bounding boxes; for the case where the second specified number is greater than the first specified number, for each round of model training, the first specified number of first reference bounding boxes are randomly sampled from the first specified number of candidate bounding boxes.

[0047] One of the goals of model training is to continuously learn the bounding box distribution through iterative training of model parameters, thereby improving the model's generalization and data transferability (i.e., the model parameters do not depend on the sample data used in the model training process and can be better applied to the data to be identified in the model application process). In order to enable the target detection model to learn the bounding box distribution better, it is necessary to ensure that the extracted first reference bounding box of the target detection model to be trained follows a certain probability distribution (such as Gaussian distribution or Cauchy distribution). Thus, the larger the number N of anchor boxes extracted by the preset region of interest extraction model, the better it helps the target detection model to learn the bounding box distribution. However, if the preset region of interest extraction model (such as the region of interest extraction algorithm ROI) is used to extract N anchor boxes as the first reference bounding box in real time each time and input them into the target detection model to be trained, it will inevitably lead to a large amount of data processing and high requirements for hardware devices.

[0048] In specific implementation, it is preferable to pre-extract N anchor boxes using a preset region of interest extraction model. Then, in each round of model training, m anchor boxes are randomly sampled from the N anchor boxes as first reference bounding boxes and input into the target detection model to be trained. This ensures both the amount of data processing in each round of model training and the model's better learning of bounding box distribution. That is, it can simultaneously take into account the amount of data processing during model training and promote the learning of bounding box distribution. Based on this, the second specified number is greater than the first specified number. Correspondingly, step S102, obtaining a subset of first bounding boxes from the first candidate bounding box set, specifically includes: randomly selecting a first specified number of candidate bounding boxes from the second specified number of candidate bounding boxes as first reference bounding boxes to obtain a subset of first bounding boxes. That is, the preset region of interest extraction model is used in advance to extract regions of interest from multiple sample image data in the sample image dataset to obtain a second specified number of candidate bounding boxes. Then, for each round of model training, a first specified number of first reference bounding boxes are randomly sampled from the second specified number of candidate bounding boxes.

[0049] In other words, the preferred implementation is to pre-extract N anchor boxes (i.e., a second specified number of candidate bounding boxes), and then, for each round of model training, randomly sample m anchor boxes (i.e., a first specified number of first reference bounding boxes) from the N anchor boxes, and then continue to execute the following step S104.

[0050] S104, Input the first reference bounding box and the actual bounding box into the target detection model to be trained for iterative training until the result of this iteration training meets the preset model iteration training termination condition, and obtain the trained target detection model; the preset model iteration training termination condition may include: the current training round number equals the total training round number, or the model loss function converges.

[0051] Specifically, regarding the model iterative training process in step S104 above, the specific implementation process of model iterative training is explained below. Since the processing procedure is the same for each model training iteration, a detailed explanation is given using any single model training example. Specifically, if the target detection model to be trained includes a bounding box prediction sub-model; such as... Figure 2 As shown, the specific implementation method for each model training step includes the following steps S1042 to S1046:

[0052] S1042, for each first reference bounding box: the bounding box prediction sub-model performs bounding box prediction based on the first reference bounding box to obtain the first predicted bounding box; based on the actual bounding box corresponding to the first reference bounding box and the first predicted bounding box corresponding to the first reference bounding box, a bounding box comparison result set is generated; wherein, the bounding box comparison result set includes a first comparison result characterizing the similarity of the bounding box distribution, and a second comparison result characterizing the overlap of the bounding box coordinates;

[0053] Specifically, regarding the process of determining the first alignment result, which characterizes the similarity of bounding box distributions, the first alignment result can be obtained by calculating the relative entropy KL divergence between the actual bounding box and the corresponding first predicted bounding box. Since the magnitude of the relative entropy KL divergence reflects the degree of difference in probability distributions between the two bounding boxes (i.e., the actual bounding box and the corresponding first predicted bounding box), a greater difference in probability distributions corresponds to a lower degree of similarity in bounding box distributions. Therefore, the relative entropy KL divergence can characterize the distribution similarity between the actual bounding box and the corresponding first predicted bounding box. Thus, based on the relative entropy KL divergence, the first regression loss component corresponding to the alignment dimension considering the similarity of bounding box distributions can be determined, thereby prompting the model to perform bounding box regression learning. Specifically, for a certain... For a given first reference bounding box, the larger the relative entropy KL divergence between the actual bounding box and the first predicted bounding box, the lower the similarity in probability distribution between the first predicted bounding box and the corresponding actual bounding box. This also means a larger first regression loss component for the comparison dimension of bounding box distribution similarity. Therefore, the similarity in distribution between the first predicted bounding box and the corresponding actual bounding box corresponding to a given first reference bounding box is determined based on the relative entropy KL divergence between the actual bounding box and the first predicted bounding box. Thus, a first comparison result can be generated based on the relative entropy KL divergence. This first comparison result can characterize the similarity in bounding box distribution, and the first regression loss component corresponding to the comparison dimension of bounding box distribution similarity can be determined based on the relative entropy KL divergence in the first comparison result.

[0054] Correspondingly, regarding the determination process of the second alignment result representing the degree of overlap between bounding box coordinates, the target cross-union ratio (CUI) loss can be obtained by considering only the cross-union ratio loss between a certain actual bounding box and its corresponding first predicted bounding box; alternatively, the target CUI loss can be determined by comprehensively considering the CUI loss between a certain actual bounding box and its corresponding first predicted bounding box, as well as the CUI loss between a certain actual bounding box and the first predicted bounding boxes corresponding to other actual bounding boxes. Since the magnitude of the target CUI loss can represent the degree of overlap between the actual bounding box and its corresponding first predicted bounding box, the second regression loss component corresponding to the alignment dimension considered from the perspective of the degree of overlap between bounding box coordinates can be determined based on the target CUI loss, thereby prompting the model to perform bounding box regression learning; specifically, for a certain first reference bounding box... For the actual bounding box and the first predicted bounding box, the target intersection-union ratio (DIU) loss between the actual bounding box and the first predicted bounding box is determined. The larger the DIU loss, the lower the degree of coordinate overlap between the first predicted bounding box and the corresponding actual bounding box. The larger the corresponding second regression loss component for the comparison dimension of the degree of bounding box coordinate overlap, the greater the degree of coordinate overlap. Therefore, the degree of coordinate overlap between the first predicted bounding box and the corresponding actual bounding box corresponding to a certain first reference bounding box is determined based on the DIU loss between the actual bounding box and the first predicted bounding box. Thus, a second comparison result can be generated based on the DIU loss. In this way, the second comparison result can characterize the degree of bounding box coordinate overlap. Then, based on the DIU loss in the second comparison result, the second regression loss component corresponding to the comparison dimension of the degree of bounding box coordinate overlap can be determined.

[0055] S1044, Based on the first alignment result and the second alignment result corresponding to the first reference bounding box in the first bounding box subset, determine the bounding box regression loss value of the target detection model to be trained.

[0056] Specifically, after obtaining the bounding box alignment result set for each first reference bounding box, the sub-regression loss value corresponding to each first reference bounding box can be obtained. This sub-regression loss value includes at least the first regression loss component corresponding to the first alignment dimension from the perspective of the similarity of the bounding box distribution, and the second regression loss component corresponding to the second alignment dimension from the perspective of the overlap of the bounding box coordinates. Then, based on the sub-regression loss value corresponding to each first reference bounding box, the bounding box regression loss value used to adjust the model parameters can be determined.

[0057] It should be noted that, in the specific implementation, when determining the sub-regression loss value corresponding to the first reference bounding box, both the similarity of the bounding box distribution and the degree of overlap of the bounding box coordinates can be considered simultaneously, or only the similarity of the bounding box distribution can be considered. That is, the set of bounding box alignment results corresponding to the first reference bounding box includes the first alignment result, and the sub-regression loss value corresponding to the first reference bounding box is determined based on the first regression loss component corresponding to the first alignment result.

[0058] S1046, Update the parameters of the bounding box prediction sub-model based on the bounding box regression loss value.

[0059] Specifically, after determining the bounding box regression loss value based on the sub-regression loss value corresponding to each first reference bounding box, the gradient descent method is used to adjust the parameters of the bounding box prediction sub-model based on the above bounding box regression loss value. Here, since the sub-regression loss value reflects at least the first regression loss component corresponding to the regression loss comparison dimension based on the similarity of the bounding box distribution and the second regression loss component corresponding to the regression loss comparison dimension based on the overlap of the bounding box coordinates, the bounding box regression loss value used to adjust the model parameters also reflects the regression loss components corresponding to these two regression loss comparison dimensions respectively. This ensures that the finally trained object detection model can not only ensure that the probability distribution of the predicted first bounding box is closer to that of the actual bounding box, but also ensure that the coordinate overlap between the first predicted bounding box and the actual bounding box is higher.

[0060] It should be noted that the model parameters are iteratively trained based on the bounding box regression loss value of the target detection model to be trained, and the target detection model can be obtained by referring to the existing process of using gradient descent method backpropagation to fine-tune the model parameters, which will not be repeated here.

[0061] In addition, it should be noted that the target detection model trained based on the model training method provided in the embodiments of this application can be applied to any specific application scenario that requires target detection of the image to be detected. For example, in specific application scenario 1, target detection is performed on the image to be detected obtained by the image acquisition device at the entrance of a public place (such as the entrance of a shopping mall, subway entrance, scenic spot entrance, or performance venue entrance). In another example, in specific application scenario 2, target detection is performed on the image to be detected obtained by the image acquisition device at each monitoring point in a breeding base.

[0062] Due to the different application scenarios of the target detection model, the sample image dataset used in the model training process also varies. For specific application scenario 1, the sample image dataset can be historical sample images collected at the entrance of a designated public place within a preset historical time period. Correspondingly, the target object enclosed by the first reference bounding box is the target user entering the designated public place in the historical sample images. The actual category and the first predicted category can be the category to which the target user belongs, such as at least one of age group, gender, height, and occupation. For specific application scenario 2, the sample image dataset can be historical sample images collected at various monitoring points in a designated breeding base within a preset historical time period. Correspondingly, the target object enclosed by the first reference bounding box is the target breeding object in the historical sample images. The actual category and the first predicted category can be the category to which the target breeding object belongs, such as at least one of live status and body size.

[0063] like Figure 3 As shown, a schematic diagram illustrating the specific implementation principle of the object detection model training process is presented, including:

[0064] Obtain a first specified number of first reference bounding boxes, and obtain the actual bounding boxes corresponding to each first reference bounding box;

[0065] For each first reference bounding box: the bounding box prediction sub-model predicts bounding boxes based on the first reference bounding box to obtain the first predicted bounding box; then, the comparison result generation module generates a bounding box comparison result set based on the actual bounding box corresponding to the first reference bounding box and the first predicted bounding box corresponding to the first reference bounding box.

[0066] Based on the first and second alignment results corresponding to each first reference bounding box, the bounding box regression loss value of the target detection model to be trained is determined.

[0067] Based on the bounding box regression loss value, the model parameters of the target detection model to be trained are iteratively updated until the training result of this iteration meets the preset model iteration training termination condition, thus obtaining the target detection model.

[0068] Specifically, regarding the process of determining the bounding box alignment result set, step S1042 above, which generates the bounding box alignment result set based on the actual bounding box corresponding to the first reference bounding box and the first predicted bounding box corresponding to the first reference bounding box, specifically includes:

[0069] Based on the actual bounding box and the first predicted bounding box corresponding to the first reference bounding box, the relative entropy KL divergence is calculated to obtain the first alignment result; based on the actual bounding box and the first predicted bounding box corresponding to the first reference bounding box, the bounding box intersection-union ratio loss is calculated to obtain the second alignment result.

[0070] Specifically, for each first reference bounding box, the bounding box alignment result set corresponding to the first reference bounding box includes not only the first alignment result obtained from the perspective of the similarity of the bounding box distribution, but also the second alignment result obtained from the perspective of the overlap of the bounding box coordinates. This can improve the comprehensiveness of the bounding box alignment result set, thereby improving the accuracy of the bounding box regression loss obtained based on the bounding box alignment result set.

[0071] In specific implementation, such as Figure 4a As shown, a schematic diagram illustrating the specific implementation principle of another object detection model training process is presented, including:

[0072] The target region is extracted from the sample image dataset in advance using a pre-defined region of interest extraction model, resulting in N anchor boxes. The sample image dataset includes multiple original sample images, each of which contains at least one target object. The feature information corresponding to each anchor box can include location information (x, y, w, h) and category information c, i.e., (x, y, w, h, c). Specifically, during model training, multiple parameter dimensions can be set to be independent of each other. Therefore, the iterative training process for the model parameters of each dimension is also independent of each other.

[0073] For each round of model training, m anchor boxes are randomly sampled from N anchor boxes as first reference bounding boxes, and the actual bounding boxes corresponding to each first reference bounding box are determined. Each target object in the sample image dataset can correspond to an actual bounding box. For example, if the total number of target objects in the sample image dataset is d, then the number of actual bounding boxes before expansion is d. In order to make the actual bounding boxes correspond to the first predicted bounding boxes, the actual bounding boxes corresponding to multiple first reference bounding boxes containing the same target object can be the same. That is, based on the target object enclosed by the first reference bounding box, the actual bounding boxes are expanded to obtain m actual bounding boxes (m>d). For example, if the target object in a certain original sample image is a cat A, and cat A corresponds to actual bounding box A, if the number of first reference bounding boxes containing cat A is 4 (such as the first reference bounding boxes with serial numbers 6, 7, 8, and 9), then the actual bounding box A is expanded into 4 actual bounding boxes A (i.e., actual bounding boxes with serial numbers 6, 7, 8, and 9).

[0074] For each first reference bounding box, the bounding box prediction sub-model predicts the bounding box based on the first reference bounding box to obtain the first predicted bounding box. Then, the comparison result generation module generates a bounding box comparison result set based on the actual bounding box corresponding to the first reference bounding box and the corresponding first predicted bounding box. Each first reference bounding box corresponds to one actual bounding box and one first predicted bounding box. The first predicted bounding box is predicted by the bounding box prediction sub-model through continuous bounding box regression learning. Specifically, the target object enclosed by the first predicted bounding boxes with indices 6, 7, 8, and 9 in the m first predicted bounding boxes output by the bounding box prediction sub-model is cat A.

[0075] For each first reference bounding box, a first regression loss component is determined based on the first alignment result in the bounding box alignment result set of the first reference bounding box, and a second regression loss component is determined based on the second alignment result in the bounding box alignment result set of the first reference bounding box.

[0076] Based on the first regression loss component and the second regression loss component corresponding to each first reference bounding box, the bounding box regression loss value of the target detection model to be trained is determined; using the stochastic gradient descent method, the model parameters of the above bounding box prediction sub-model are adjusted based on the bounding box regression loss value to obtain the parameter-updated bounding box prediction sub-model.

[0077] If the training result of this model iteration meets the preset model iteration training termination condition, then the updated bounding box prediction sub-model will be determined as the trained object detection model.

[0078] If the results of this model iteration training do not meet the preset model iteration training termination condition, the updated bounding box prediction sub-model will be determined as the target detection model to be trained in the next round of model training, until the preset model iteration training termination condition is met.

[0079] The bounding box regression loss value of the target detection model to be trained is determined by the sub-regression loss values ​​corresponding to multiple first reference bounding boxes. Each sub-regression loss value corresponding to a first reference bounding box is determined by multiple regression loss components. Therefore, step S1044, which determines the bounding box regression loss value based on the first and second alignment results corresponding to the first reference bounding boxes in the first bounding box subset, specifically includes:

[0080] Determine the sub-regression loss value corresponding to each first reference bounding box in the first bounding box subset; the sub-regression loss value corresponding to each first reference bounding box is determined based on target information, wherein the target information includes one or a combination of the following: the similarity of the bounding box distribution represented by the first comparison result corresponding to the first reference bounding box, and the degree of overlap of the bounding box coordinates represented by the second comparison result;

[0081] Based on the sub-regression loss values ​​corresponding to each first reference bounding box in the first bounding box subset, the bounding box regression loss value of the target detection model to be trained is determined.

[0082] Specifically, in implementation, when determining the sub-regression loss value corresponding to the first reference bounding box, one can consider only the first regression loss component corresponding to the first comparison result, or simultaneously consider both the first regression loss component corresponding to the first comparison result and the second regression loss component corresponding to the second comparison result. Taking the simultaneous consideration of two bounding box comparison dimensions as an example, for each first reference bounding box, the corresponding sub-regression loss value is equal to the weighted sum of the two regression loss components, which can be expressed as follows:

[0083] V i (D,G)=λ1V i1 +λ2V i2

[0084] Where λ1 represents the first weight coefficient corresponding to the first regression loss component in the first alignment dimension, V i1 V represents the first regression loss component in the first alignment dimension (i.e., the regression loss component corresponding to the similarity to the bounding box distribution represented by the first alignment result), λ2 represents the second weight coefficient corresponding to the second regression loss component in the second alignment dimension, and V i2 This represents the second regression loss component under the second comparison dimension (i.e., the regression loss component corresponding to the degree of overlap of the bounding box coordinates represented by the second comparison result); specifically, the first comparison dimension can be a regression loss comparison dimension based on the similarity of the bounding box distribution, and the second comparison dimension can be a regression loss comparison dimension based on the degree of overlap of the bounding box coordinates.

[0085] In practical implementation, for multiple first reference bounding boxes, the first and second weight coefficients can remain unchanged. However, considering that the first and second regression loss components correspond to different regression loss comparison dimensions (i.e., the regression loss comparison dimension based on the similarity of bounding box distributions and the regression loss comparison dimension based on the overlap of bounding box coordinates), and that the focus of regression loss considerations differs between different regression loss comparison dimensions (e.g., the regression loss comparison dimension based on the similarity of bounding box distributions focuses on the regression loss of the first reference bounding box corresponding to the actual bounding box with blurred edges, while the regression loss comparison dimension based on the overlap of bounding box coordinates focuses on the regression loss of the first reference bounding box with similar bounding box distributions but different specific positions), the magnitude relationship between the first and second regression loss components reflects, to some extent, which regression loss comparison dimension can more accurately characterize the regression loss between the actual bounding box and the first predicted bounding box. Based on this, for each first reference bounding box... A reference bounding box is used. The magnitudes of the first and second weighting coefficients are adjusted based on the relationship between the first and second regression loss components corresponding to this first reference bounding box. Specifically, if the absolute value of the difference between the first and second regression loss components is not greater than a preset loss threshold, the first and second weighting coefficients remain unchanged. If the absolute value of the difference between the first and second regression loss components is greater than the preset loss threshold and the first regression loss component is greater than the second regression loss component, the first weighting coefficient is increased according to a first preset adjustment method. If the absolute value of the difference between the first and second regression loss components is greater than the preset loss threshold and the first regression loss component is less than the second regression loss component, the second weighting coefficient is increased according to a second preset adjustment method. This achieves the effect of prioritizing the regression loss component corresponding to the comparison dimension that best reflects the regression loss of the bounding box for each first reference bounding box during model training, thereby further improving the accuracy of model parameter optimization.

[0086] It should be noted that the increase in the first weight coefficient corresponding to the first preset adjustment method and the increase in the second weight coefficient corresponding to the second preset adjustment method can be the same or different. The increase in the weight coefficient can be set according to actual needs, and this application does not limit it.

[0087] Specifically, the process of obtaining the first alignment result by considering the similarity of bounding box distributions involves calculating the relative entropy KL divergence based on the actual bounding box corresponding to the first reference bounding box and the first predicted bounding box, thus obtaining the first alignment result. This process includes:

[0088] Step A1: Determine the first probability distribution of the actual bounding box corresponding to the first reference bounding box, and determine the second probability distribution of the first predicted bounding box corresponding to the first reference bounding box.

[0089] Step A2: Calculate the KL divergence value between the first probability distribution and the second probability distribution; wherein, the KL divergence value is used to characterize the distribution similarity between the first predicted bounding box and the actual bounding box;

[0090] Step A3: Based on the above KL divergence values, determine the first alignment result corresponding to the first reference bounding box.

[0091] Specifically, for each first reference bounding box, from the perspective of the similarity of bounding box distributions, the KL divergence value between the first probability distribution corresponding to the actual bounding box and the second probability distribution corresponding to the first predicted bounding box is calculated. This KL divergence value can characterize the similarity of the bounding box distributions between the actual bounding box and the corresponding first predicted bounding box. The smaller the KL divergence value, the smaller the difference in the bounding box distributions, and correspondingly, the greater the similarity of the bounding box distributions. Therefore, after determining the KL divergence value between the actual bounding box and the first predicted bounding box, the first alignment result can be obtained, which can characterize the similarity of the bounding box distributions. Then, based on the first alignment result, the first regression loss component corresponding to the alignment dimension characterizing the similarity of the bounding box distributions can be determined. The larger the KL divergence value, the lower the similarity of the distributions between the actual bounding box corresponding to the first reference bounding box and the corresponding first predicted bounding box, and therefore, the larger the first regression loss component corresponding to the first reference bounding box. Then, the model parameters of the bounding box prediction sub-model are updated based on the first regression loss component to improve the bounding box prediction performance of the bounding box prediction sub-model.

[0092] Specifically, since the probabilities of both the actual and predicted bounding boxes appearing follow a certain probability distribution (such as a Gaussian distribution), if the first probability distribution is... The second probability distribution is The KL divergence value can then be expressed as: The first and second probability distributions can be determined in the following ways:

[0093]

[0094] in, This represents the first reference bounding box with index i, σ1 represents the first variance, and b ground θ represents the mean of the actual bounding box. d Represents parameters related to the true bounding box distribution;

[0095]

[0096] in, This represents the first reference bounding box with index i, σ² represents the second variance, and b estimation θ represents the mean of the first predicted bounding box. g This represents the model parameters of the bounding box prediction sub-model.

[0097] Specifically, the bounding box regression loss value is equal to the sum of the sub-regression loss values ​​corresponding to the first specified number of first reference bounding boxes, which can be expressed as:

[0098]

[0099] Where, N reg This indicates the first specified quantity, where i represents the index of the first reference bounding box, and the value of i ranges from 1 to N. reg .

[0100] Specifically, the process of obtaining the second alignment result by considering the degree of overlap of bounding box coordinates involves calculating the boundary box intersection-union ratio (IUU) loss based on the actual bounding box corresponding to the first reference bounding box and the first predicted bounding box, to obtain the second alignment result. This process includes:

[0101] Step B1: Calculate the intersection-union ratio (IUR) loss for the actual bounding box corresponding to the first reference bounding box and the first predicted bounding box corresponding to the first reference bounding box to obtain the first IUR loss.

[0102] Specifically, taking the first reference bounding box with index i as an example, the cross-union ratio loss between the actual bounding box with index i and the first predicted bounding box with index i is calculated to obtain the first cross-union ratio loss corresponding to the first reference bounding box with index i.

[0103] Step B2: Based on the first intersection-union ratio loss mentioned above, determine the second alignment result corresponding to the first reference bounding box; wherein, the bounding box intersection-union ratio loss can characterize the degree of overlap of the bounding box coordinates.

[0104] Specifically, since the cross-union ratio loss between two bounding boxes can characterize the degree of overlap of the bounding box coordinates, the second alignment result can be obtained based on the cross-union ratio loss between the actual bounding box and the first predicted bounding box. Then, based on the second alignment result, the second regression loss component corresponding to the alignment dimension considering the degree of overlap of the bounding box coordinates can be determined, thereby prompting the model to perform bounding box regression learning.

[0105] Furthermore, regarding the determination process of the second alignment result, only the first intersection-union ratio (IUU) loss between the actual bounding box and its corresponding first predicted bounding box can be considered. However, in order to improve the accuracy of the second alignment result determination, thereby improving the accuracy of the second regression loss component corresponding to the alignment dimension from the perspective of bounding box coordinate overlap, and thus improving the accuracy of the bounding box regression loss value used to adjust model parameters, not only the first IUU loss between the actual bounding box and its corresponding first predicted bounding box is considered, but also the second IUU loss between the actual bounding box and other first predicted bounding boxes. This allows the actual bounding box to be compared with positive example samples (i.e., the first predicted bounding box corresponding to a certain actual bounding box obtained through bounding box regression learning) and negative example samples (i.e., the first predicted bounding boxes corresponding to other actual bounding boxes obtained through bounding box regression learning, excluding a certain actual bounding box) in the alignment dimension of bounding box coordinate overlap, to learn the specific position representation of the actual bounding box, thereby prompting the model to perform better bounding box regression learning. Based on this, step B2 above, based on the above first IUU loss, determines the second alignment result corresponding to the first reference bounding box, specifically including:

[0106] B21. Among the first predicted bounding boxes corresponding to the first specified number of first reference bounding boxes, a set of comparison bounding boxes is determined; wherein, the set of comparison bounding boxes includes other first predicted bounding boxes besides the first predicted bounding boxes corresponding to the first reference bounding boxes, or other first predicted bounding boxes that do not contain the target object enclosed by the first reference bounding boxes.

[0107] Specifically, taking the first reference bounding box with index i as an example, the above-mentioned set of comparison bounding boxes can include other first predicted bounding boxes besides the first predicted bounding box with index i (i.e., the first predicted bounding box with index k, k≠p, p=i). In other words, all other first predicted bounding boxes besides the first predicted bounding box with index i are used as negative examples of the actual bounding box with index i. To further improve the accuracy of the selection of negative examples, the above-mentioned set of comparison bounding boxes can include other first predicted bounding boxes besides the first predicted bounding box with index i, and the other first predicted bounding boxes do not contain the target object enclosed by the first reference bounding box with index i (i.e., the first predicted bounding box with index k, k≠p, p=i or p=j, the first predicted bounding box with index j has the same target object enclosed as the first reference bounding box with index i). In other words, only other first predicted bounding boxes that contain different target objects from the first reference bounding box with index i are used as negative examples of the actual bounding box with index i.

[0108] B22, calculate the boundary box intersection-union ratio loss for the actual boundary box corresponding to the first reference boundary box and other first predicted boundary boxes respectively, and obtain the second intersection-union ratio loss;

[0109] Specifically, taking the first reference bounding box with index i as an example, for each other first predicted bounding box in the set of comparison bounding boxes, the cross-union ratio loss between the actual bounding box with index i and the first predicted bounding box with index k is calculated, and the second cross-union ratio loss corresponding to the first predicted bounding box with index k is obtained.

[0110] B23. Based on the first cross-union ratio loss and the second cross-union ratio loss mentioned above, determine the second alignment result corresponding to the first reference bounding box.

[0111] Specifically, in determining the second alignment result representing the degree of overlap of bounding box coordinates, a first crossover ratio (CRR) loss is calculated based on the actual bounding box with index i and the first predicted bounding box with index i, and a second CRR loss (k≠p) is calculated based on the actual bounding box with index i and the first predicted bounding box with index k, to determine the second alignment result (i.e., the second alignment result can include both the first and second CRR losses). Then, based on the second alignment result, a second regression loss component related to the degree of overlap of bounding box coordinates can be determined. Adjusting the model parameters based on the second regression loss component can increase the degree of overlap between the actual bounding box with index i and the first predicted bounding box with index i, while decreasing the degree of overlap with other first predicted bounding boxes, thereby enhancing the globality of bounding box regression learning and further improving the accuracy of bounding box regression learning.

[0112] In practical implementation, the aforementioned second regression loss component is the logarithm of the target cross-union ratio (CUN) loss. This target CUN loss is the quotient of the exponent of the first CUN loss and the sum of the exponents of multiple second CUN losses. That is, taking p=i as an example, the second regression loss component can be expressed as:

[0113]

[0114] in, This represents the actual bounding box corresponding to the first reference bounding box with index i. This represents the first reference bounding box with index i. This represents the first predicted bounding box corresponding to the first reference bounding box with index i. Indicates the first intersection and comparison loss. This represents the first reference bounding box with index k. This represents the first predicted bounding box corresponding to the first reference bounding box with index k. θ represents the second crossover ratio loss. g This represents the model parameters of the bounding box prediction sub-model, and ω represents the preset adjustment factor.

[0115] Furthermore, considering that in the object detection process, the object detection model needs to determine not only the location of the target object but also its specific category, there may be a problem of low accuracy in category recognition for certain first reference bounding boxes during the training process. Given that the first predicted bounding box corresponding to such a first reference bounding box may not truly reflect the bounding box prediction accuracy of the bounding box prediction sub-model, the autoregressive loss between the first predicted bounding box and the actual bounding box corresponding to such first original bounding boxes also cannot truly reflect the bounding box prediction accuracy of the bounding box prediction sub-model. Therefore, to further improve the accuracy of the bounding box regression loss value, in the process of determining the sub-regression loss value corresponding to the first predicted bounding box, the first predicted category corresponding to the first predicted bounding box is considered. Only when the actual category corresponding to the first predicted bounding box matches the first predicted category is its corresponding sub-regression loss value considered; otherwise, only its corresponding sub-classification loss value is considered, that is, the sub-regression loss value corresponding to the first reference bounding box whose category prediction result does not meet the preset requirements is excluded. Based on this, the above object detection model also includes a bounding box classification sub-model; the specific implementation method of each model training also includes: the bounding box classification sub-model performs... The classification process yields a first category prediction result. In practice, the bounding box classification sub-model predicts the category of the first reference bounding box or the first predicted bounding box, and the output can be the first category prediction result. The first category prediction result includes the predicted probability of the target object enclosed by the first reference bounding box or the first predicted bounding box belonging to each candidate category. The candidate category corresponding to the highest predicted probability is the first predicted category. In other words, the category of the target object enclosed by the first reference bounding box or the first predicted bounding box is predicted as the first predicted category by the bounding box classification sub-model. This means the category of the target object within the image region of the first reference bounding box or the first predicted bounding box is determined by the bounding box classification sub-model. The model predicts the first predicted category. In addition, in specific implementation, considering that the positional information of the first reference bounding box and the first predicted bounding box will not deviate significantly, and the image features within the first reference bounding box and the image features within the first predicted bounding box will not deviate significantly, the identification of the target object category in the image region within the bounding box will not be affected. Based on this, for the case where bounding box prediction and category prediction are performed sequentially, the first predicted bounding box can be input into the bounding box classification sub-model for category prediction to obtain the corresponding first category prediction result. That is, the first predicted bounding box is first predicted based on the first reference bounding box, and then the category is predicted on the first predicted bounding box to obtain the first category prediction result.For cases where bounding box prediction and class prediction are performed simultaneously, the first reference bounding box can be input into the bounding box classification sub-model for class prediction to obtain the corresponding first class prediction result. That is, the first predicted bounding box is obtained based on the first reference bounding box prediction, and the class is predicted based on the first reference bounding box to obtain the first class prediction result.

[0116] It should be noted that the iterative training process of the model parameters of the above bounding box classification sub-model can refer to the existing classification model training process, and will not be repeated here.

[0117] Specifically, the aforementioned target information also includes the category matching result between the first predicted category represented by the first category prediction result corresponding to the first reference bounding box and the actual category of the first reference bounding box. Specifically, in the process of determining the sub-regression loss value corresponding to each first reference bounding box, if the corresponding category matching result is that the first predicted category and the actual category do not match, then the sub-regression loss value corresponding to the first reference bounding box is zero; if the corresponding category matching result is that the first predicted category and the actual category match, then the sub-regression loss value corresponding to the first reference bounding box is a sub-regression loss value determined based on at least one of the first regression loss component corresponding to the similarity of the bounding box distribution and the second regression loss component corresponding to the overlap of the bounding box coordinates.

[0118] Specifically, the preset category matching constraint for determining whether the first predicted category corresponding to the first reference bounding box matches the actual category can be related to the first category prediction result. Specifically, it can include: constraint for a single matching method or constraint for a variable matching method. For constraint for a single matching method, the category matching constraint used in each round of model training remains unchanged (i.e., it is independent of the current model training round). For example, for each round of model training, if the actual category is the same as the first predicted category, then it is determined that the first predicted category corresponding to the first reference bounding box matches the actual category. For constraint for a variable matching method, the category matching constraint used in each round of model training is related to the current model training round. Specifically, constraint for a variable matching method can be further divided into category matching stage constraint or category matching gradual constraint.

[0119] The aforementioned staged category matching constraint can be that when the current model training epochs are less than a first preset number of epochs, the actual category and the first predicted category belong to the same category group; and when the current model training epochs are greater than or equal to the first preset number of epochs, the actual category and the first predicted category are the same. That is, based on the staged category matching constraint and the category prediction result corresponding to the first reference bounding box, staged category matching constraints can be achieved. The aforementioned gradual category matching constraint can be that the sum of the first constraint term and the second constraint term is greater than a preset probability threshold. The first constraint term is the first predicted probability corresponding to the actual category in the category prediction probability subset, and the second constraint term is the product of the sum of the second predicted probabilities (excluding the first predicted probability) in the category prediction probability subset and a preset adjustment factor. The preset adjustment factor gradually decreases as the current training epochs increase. That is, based on the gradual category matching constraint and the category corresponding to the first reference bounding box... The prediction results enable gradual category matching constraints. Specifically, a subset of category prediction probabilities is determined based on the category prediction results corresponding to the first reference bounding box. This subset includes the first prediction probability that the target object enclosed by the first prediction bounding box belongs to the actual category, and the second prediction probability that it belongs to a non-actual category in the target group. That is, the subset of category prediction probabilities includes the first prediction probability under the actual category in the target group and the second prediction probability under the non-actual category (i.e., candidate category other than the actual category) in the target group, obtained by the bounding box classification sub-model performing category prediction on the first reference bounding box or the first prediction bounding box. The target group is the category group to which the actual category belongs. In specific implementation, multiple candidate categories associated with the target detection task are pre-determined. Based on the semantic information of each candidate category, multiple candidate categories are grouped to obtain multiple category groups.

[0120] Specifically, since the first reference bounding box is obtained by extracting the region of interest using a preset region of interest extraction model, there may be cases where the region of the target object enclosed by the first reference bounding box is not accurate enough, which may lead to inaccurate category identification of the first predicted bounding box corresponding to such a first reference bounding box in the early stage of model training. Based on this, in the process of determining the sub-regression loss value corresponding to the first reference bounding box, the category matching result between the first predicted category corresponding to the first reference bounding box and the actual category of the first reference bounding box is referenced. That is, the category matching result used to characterize whether the first predicted category corresponding to the first reference bounding box matches the actual category is determined based on the above preset category matching constraints.

[0121] Furthermore, the bounding box classification sub-model can be pre-trained, or its model parameters can be trained simultaneously during the training of the bounding box prediction sub-model. Specifically, the classification loss value is determined based on the first predicted class and the actual class, and the model parameters of the bounding box classification sub-model are iteratively trained based on the classification loss value. Regarding the case of simultaneously training the bounding box classification sub-model's parameters, it is also considered that the accuracy of the model parameters in the bounding box classification sub-model within the target detection model to be trained might be low in the early stages of model training, thus affecting the accuracy of the first predicted bounding box corresponding to the first reference bounding box. In cases where the category identification of the bounding box is inaccurate, the requirements for category accuracy are relaxed in the early stage of model training. As long as the actual category corresponding to the first predicted bounding box and the first predicted category belong to the same category group, the corresponding sub-regression loss value is considered. In the later stage of model training, the requirements for category accuracy are tightened. The corresponding sub-regression loss value is only considered if the actual category corresponding to the first predicted bounding box and the first predicted category are the same. Based on this, the above-mentioned preset category matching constraints may include: the constraints of the above-mentioned changing matching method (such as the category matching stage constraint or the category matching gradual constraint).

[0122] Furthermore, to ensure a smoother transition between the two categories of category matching constraints that limit the first predicted category and the actual category to satisfy the category matching result (i.e., the first predicted category belongs to the target group, and the first predicted category is the same as the actual category), so that as the number of model training rounds increases, the preset category matching constraints gradually change from limiting the first predicted category to fall into the target group to limiting the first predicted category to be the same as the actual category, therefore, preferably, the above-mentioned preset category matching constraints include: a gradual category matching constraint.

[0123] In practical implementation, for the case where the above-mentioned preset category matching constraint is a category matching gradient constraint, taking the first reference bounding box with index i as an example, the category matching gradient constraint can be expressed as:

[0124]

[0125] Where groups represent the target group, real i f represents the actual category of the first reference bounding box with index i in the target group groups, where f ∈ groups\real i This represents the non-actual category in the target group, and β represents the prediction moderating factor. This represents the first predicted probability (i.e., the first constraint term mentioned above). Indicates the second predicted probability. This represents the second constraint term mentioned above, where μ represents the preset probability threshold; specifically, The larger the value, the closer the first predicted class is to the actual class. Since the preset adjustment factor decreases with each training epoch, the reference proportion of the second constraint term gradually decreases. This means that in the later stages of model training, the first constraint term (i.e., the first predicted probability under the actual class) primarily determines whether the first predicted class matches the actual class. Then, after a certain number of training epochs, the second constraint term becomes zero. When the probability exceeds the preset probability threshold, it indicates that the bounding box classification sub-model has determined the actual category as the first predicted category.

[0126] Specifically, regarding the aforementioned preset adjustment factor, it decreases as the current model training epochs increase. If the current model training epochs are less than or equal to the target training epochs, the aforementioned second constraint term is positively correlated with the preset adjustment factor, and the aforementioned preset adjustment factor is negatively correlated with the current model training epochs. If the current model training epochs are greater than the target training epochs, the aforementioned second constraint term is zero, wherein the target training epochs are less than the total number of training epochs.

[0127] In practical implementation, to ensure the smoothness of the adjustment of the preset adjustment factor, a linear decreasing adjustment method can be used to gradually reduce the value of the preset adjustment factor β. Therefore, the process for determining the preset adjustment factor used in the current model training is as follows:

[0128] (1) For the first round of model training, the first preset value is determined as the preset adjustment factor used for the current model training;

[0129] Specifically, the first preset value can be set according to actual needs. To simplify the adjustment complexity, the first preset value can be set to 1, that is, the preset adjustment factor β = 1. In other words, in the case of the first round of model training, the above-mentioned gradual constraint condition for category matching can be:

[0130] Right now

[0131] In other words, for the first round of model training, based on the sum of the first and second predicted probabilities corresponding to the target group, it is determined whether the first predicted category corresponding to the first reference bounding box matches the actual category.

[0132] (2) For non-first round model training, the preset adjustment factor used for the current model training is determined according to the factor decreasing adjustment method, based on the current model training round number, the target training round number and the above-mentioned first preset value.

[0133] Specifically, if the preset adjustment factor β = 1 for the first round of model training, then in the case of subsequent rounds of model training, the above-mentioned gradual category matching constraint can be:

[0134] β < 1;

[0135] In other words, for non-first-round model training, the above-mentioned gradual category matching constraints... Furthermore, as the number of training rounds increases, the second constraint term... Their participation gradually decreased.

[0136] For example, the decreasing formula corresponding to the above factor decreasing adjustment method can be:

[0137]

[0138] in, express Take the maximum value between 0 and 0. The first item 1 represents the first preset value (i.e., the preset adjustment factor β used in the first round of training), δ represents the current training epoch of the model, and Z represents the target training epoch. That is, the target training epoch can be the total training epoch minus 1, or it can be a specified training epoch. The specified training epoch is less than the total training epoch. The difference between the total training epoch and the specified training epoch is the preset epoch Q. Q is greater than 2. That is, in the later stages of model training (not the last epoch), the preset adjustment factor β is set to 0. In other words, the judgment condition used in the later stages of model training, from δ = Z + 1 epochs to the last epoch, is...

[0139] It should be noted that, for the case where the target training round number Z is the total training round number minus 1, the above decreasing formula can be: In the final round of model training, the preset adjustment factor is set to 0, meaning that all decision criteria used in the final round of model training are... Furthermore, the decreasing formula shown above is only a relatively simple linear decreasing adjustment method. In practical applications, the decreasing rate of the preset adjustment factor β can be set according to actual needs. Therefore, the decreasing formula above does not constitute a limitation on the scope of protection of this application.

[0140] In practical implementation, the target detection model to be trained includes a bounding box prediction sub-model and a bounding box classification sub-model, such as... Figure 4b As shown, a schematic diagram illustrating the specific implementation principle of another object detection model training process is presented, which includes:

[0141] (1) Use a pre-defined region of interest extraction model to extract the target region from the sample image dataset to obtain N anchor boxes;

[0142] (2) For each round of model training, randomly sample m anchor boxes from N anchor boxes as first reference bounding boxes, and determine the actual bounding box corresponding to each first reference bounding box;

[0143] (3) For each first reference bounding box, the bounding box prediction sub-model performs bounding box prediction based on the first reference bounding box to obtain the first predicted bounding box; then, the comparison result generation module generates a bounding box comparison result set based on the actual bounding box corresponding to the first reference bounding box and the corresponding first predicted bounding box; the bounding box classification sub-model performs category prediction on the first predicted bounding box to obtain the category prediction result; the category matching result is determined according to the preset category matching constraint, the actual category of the actual bounding box corresponding to the first reference bounding box and the category prediction result of the first predicted bounding box corresponding to the first reference bounding box; if the category matching result indicates that the first predicted category and the actual category do not meet the preset category matching constraint, then the sub-regression loss value corresponding to the first reference bounding box is zero; if the category matching result indicates that the first predicted category and the actual category meet the preset category matching constraint, then the first regression loss component is determined based on the first comparison result in the bounding box comparison result set of the first reference bounding box, the second regression loss component is determined based on the second comparison result in the bounding box comparison result set of the first reference bounding box, and then the sub-regression loss value corresponding to the first reference bounding box is determined based on the first regression loss component and the second regression loss component.

[0144] It should be noted that the process of determining the above-mentioned category matching results can be considered during the process of determining the bounding box regression loss value based on the bounding box comparison result set, or it can be considered during the process of generating a bounding box comparison result set for a certain first reference bounding box. In this way, if the first predicted category and the actual category do not meet the preset category matching constraints, the corresponding bounding box comparison result set can be directly determined to be empty or preset information, without having to generate a bounding box comparison result set based on the actual bounding box corresponding to the first reference bounding box and the corresponding first predicted bounding box, which can further improve the model training efficiency; specifically, refer to Figure 4bAs shown, the comparison result generation module generates a bounding box comparison result set based on the actual bounding box and the first predicted bounding box corresponding to the first reference bounding box, as well as the actual category and category prediction result corresponding to the first reference bounding box. Specifically, it determines the category matching result based on the actual category of the actual bounding box corresponding to the first reference bounding box and the category prediction result of the first predicted bounding box corresponding to the first reference bounding box. If the category matching result indicates that the first predicted category and the actual category do not meet the preset category matching constraint, the corresponding bounding box comparison result set is empty or contains preset information. Therefore, the sub-regression loss value determined based on the bounding box comparison result set is zero. If the category matching result indicates that the first predicted category and the actual category meet the preset category matching constraint, a bounding box comparison result set is generated based on the actual bounding box corresponding to the first reference bounding box and the corresponding first predicted bounding box. Therefore, the sub-regression loss value determined based on the bounding box comparison result set is determined based on the first regression loss component corresponding to the first comparison result and the second regression loss component corresponding to the second comparison result in the bounding box comparison result set.

[0145] In other words, in determining whether the sub-regression loss value corresponding to the first reference bounding box is zero, one approach is to directly generate a bounding box comparison result set based on the actual bounding box and the corresponding first predicted bounding box of the first reference bounding box; then determine the category matching result between the first predicted category and the actual category based on the category prediction result (i.e., the category matching result, indicating whether the first predicted category and the actual category meet the preset category matching constraint); if the category matching result is a category mismatch, then determine the corresponding sub-regression loss value as zero; if the category matching result is a category match, then determine the corresponding sub-regression loss value based on multiple comparison results in the bounding box comparison result set; alternatively, one approach is to first determine the category matching result between the first predicted category and the actual category based on the category prediction result; if the category matching result is a category mismatch, then determine that the corresponding bounding box comparison result set is empty or contains preset information, and determine that the corresponding sub-regression loss value is zero; if the category matching result is a category match, then generate a bounding box comparison result set based on the actual bounding box and the corresponding first predicted bounding box of the first reference bounding box, and determine the corresponding sub-regression loss value based on multiple comparison results in the bounding box comparison result set.

[0146] (4) Based on the sub-regression loss values ​​corresponding to each first reference bounding box, determine the bounding box regression loss value of the target detection model to be trained; use the stochastic gradient descent method to adjust the model parameters of the above bounding box prediction sub-model based on the bounding box regression loss value to obtain the parameter-updated bounding box prediction sub-model.

[0147] (5) If the training result of this model iteration meets the preset model iteration training termination condition, the updated bounding box prediction sub-model is determined as the trained target detection model; if the training result of this model iteration does not meet the preset model iteration training termination condition, the updated bounding box prediction sub-model is determined as the target detection model to be trained in the next round of model training, until the preset model iteration training termination condition is met.

[0148] In the model training method of this application embodiment, during the model training phase, the bounding box prediction sub-model predicts a first predicted bounding box based on a first reference bounding box. Then, based on the first predicted bounding box and its corresponding actual bounding box, the target detection model to be trained continuously learns the bounding box distribution, making the predicted first predicted bounding box closer to the corresponding actual bounding box, thereby improving the bounding box prediction accuracy, model generalization, and data transferability of the trained target detection model. Furthermore, the set of comparison results used to determine the bounding box regression loss value includes not only the first comparison result representing the similarity of the bounding box distribution, but also the second comparison result representing the overlap of the bounding box coordinates. The bounding box regression loss value is then obtained based on the first and second comparison results corresponding to each first reference bounding box. This makes the bounding box regression loss value include the regression loss obtained based on the coarse-grained comparison dimension of the similarity of the bounding box distribution and the regression loss obtained based on the fine-grained comparison dimension of the overlap of the bounding box coordinates. This can improve the accuracy of the bounding box regression loss value, thereby further improving the accuracy of the model parameters updated based on the bounding box regression loss value.

[0149] Corresponding to the above Figures 1 to 4b Based on the same technical concept, this application also provides an object detection method for the described model training method. Figure 5 This is a flowchart illustrating the target detection method provided in an embodiment of this application. Figure 5 The method described above can be executed by an electronic device equipped with an object detection device. This electronic device can be a terminal device or a designated server. The hardware device used for object detection (i.e., the electronic device equipped with the object detection device) and the hardware device used for training the object detection model (i.e., the electronic device equipped with the object detection model training device) can be the same or different. Figure 5 As shown, the method includes at least the following steps:

[0150] S502, Obtain a subset of second bounding boxes corresponding to the image to be detected from the set of second candidate bounding boxes; wherein, the subset of second bounding boxes includes a third specified number of second reference bounding boxes, and the set of second candidate bounding boxes is obtained by extracting the target region of the image to be detected using a preset region of interest extraction model;

[0151] Specifically, the process of obtaining the third specified number of second reference bounding boxes can refer to the process of obtaining the first specified number of first reference bounding boxes described above, and will not be repeated here.

[0152] S504, the above-mentioned second reference bounding box is input into the target detection model for target detection to obtain the second predicted bounding box and the second category prediction result corresponding to each second reference bounding box; wherein, the target detection model is trained based on the above model training method, and the specific training process of the target detection model is described in the above embodiment, and will not be repeated here.

[0153] Specifically, the above-mentioned object detection model includes a bounding box classification sub-model and a bounding box prediction sub-model. For each second reference bounding box: during the object detection process, the bounding box prediction sub-model predicts the bounding box based on the second reference bounding box to obtain the second predicted bounding box corresponding to the second reference bounding box; the bounding box classification sub-model classifies the second reference bounding box or the second predicted bounding box to obtain the second predicted category corresponding to the second reference bounding box.

[0154] In practical implementation, the bounding box classification sub-model predicts the category of the second reference bounding box or the second predicted bounding box, and the output result can be the second category prediction result. The second category prediction result includes the predicted probability that the target object enclosed by the second reference bounding box or the second predicted bounding box belongs to each candidate category. The candidate category corresponding to the highest predicted probability is the second predicted category. That is, the category of the target object enclosed by the second reference bounding box or the second predicted bounding box is predicted as the second predicted category by the bounding box classification sub-model; in other words, the category of the target object in the image region within the second reference bounding box or the second predicted bounding box is predicted as the second predicted category by the bounding box classification sub-model. Furthermore, in practical implementation, considering that the positional information of the second reference bounding box and the second predicted bounding box will not deviate significantly, the image features within the second reference bounding box... The image features within the second predicted bounding box will not deviate significantly. Therefore, it will not affect the recognition of the target object category within the image region of the bounding box. Based on this, for the case where bounding box prediction and category prediction are performed sequentially, the second predicted bounding box can be input into the bounding box classification sub-model for category prediction to obtain the corresponding second category prediction result. That is, the second predicted bounding box is first predicted based on the second reference bounding box, and then the category is predicted on the second predicted bounding box to obtain the second category prediction result. For the case where bounding box prediction and category prediction are performed simultaneously, the second reference bounding box can also be input into the bounding box classification sub-model for category prediction to obtain the corresponding second category prediction result. That is, the second predicted bounding box is predicted based on the second reference bounding box, and the category is predicted on the second reference bounding box to obtain the second category prediction result.

[0155] S506, Based on the second predicted bounding box and the second category prediction results corresponding to each second reference bounding box, generate the target detection results of the image to be detected.

[0156] Specifically, based on the second predicted bounding box and the second predicted category corresponding to each second reference bounding box, the number of target objects contained in the image to be detected and the category to which each target object belongs can be determined. For example, the image to be detected contains a cat, a dog and a pedestrian.

[0157] In practical implementation, the aforementioned object detection model includes a bounding box prediction sub-model and a bounding box classification sub-model, such as... Figure 6 The diagram illustrates the specific implementation principle of a target detection process, including:

[0158] A preset region of interest extraction model is used to extract the target region of the image to be detected, resulting in P anchor boxes;

[0159] Randomly sample n anchor boxes from P anchor boxes as the second reference bounding box;

[0160] For each second reference bounding box, the bounding box prediction sub-model predicts the bounding box based on the second reference bounding box to obtain the second predicted bounding box; the bounding box classification sub-model predicts the category of the second predicted bounding box to obtain the second predicted category.

[0161] Based on the second predicted bounding box and the second predicted category corresponding to each second reference bounding box, the target detection result of the image to be detected is generated.

[0162] It should be noted that the target detection model trained based on the above target detection model training method can be applied to any specific application scenario that requires target detection of the image to be detected. The image to be detected can be acquired by an image acquisition device set up at a certain location. Correspondingly, the target detection device can belong to the image acquisition device, specifically it can be the image processing device in the image acquisition device. The image processing device receives the image to be detected transmitted by the image acquisition device in the image acquisition device and performs target detection on the image to be detected. Alternatively, the target detection device can be a separate target detection device independent of the image acquisition device. The target detection device receives the image to be detected from the image acquisition device and performs target detection on the image to be detected.

[0163] Specifically, for specific application scenarios of object detection, for example, the image to be detected may be acquired by an image acquisition device set up at the entrance of a public place (such as a shopping mall entrance, subway entrance, scenic spot entrance, or performance venue entrance, etc.). Correspondingly, the target object to be detected in the image is the target user entering the public place. The above-mentioned object detection model is used to perform object detection on the image to be detected, so as to delineate a second predicted bounding box containing the target user entering the public place in the image to be detected, and determine the second predicted category corresponding to the second predicted bounding box (i.e., the category to which the target user contained in the second predicted bounding box belongs, such as age group, gender, body size, etc.). The target detection model obtains the target detection result of the image to be detected by selecting a target detection model (at least one of the following: high-level, professional, etc.); then, based on the target detection result, the user group identification result is determined (such as the flow of people entering the public place, or the attributes of the user group entering the public place, etc.); furthermore, based on the user group identification result, the corresponding business processing is executed (such as automatically triggering the entry restriction prompt operation, or pushing information to the target user, etc.); wherein, the higher the accuracy of the model parameters of the above target detection model, the higher the accuracy of the target detection result of the image to be detected output by the target detection model, and therefore, the higher the accuracy of triggering the execution of the corresponding business processing based on the target detection result.

[0164] For example, the image to be detected can be acquired by image acquisition devices set up at various monitoring points in a certain breeding base. Correspondingly, the target object to be detected in the image is the target breeding object within the breeding monitoring point. The above-mentioned target detection model is used to perform target detection on the image to be detected, so as to delineate a second predicted bounding box containing the target breeding object in the image to be detected, and determine the second predicted category corresponding to the second predicted bounding box (i.e., the category to which the target breeding object contained in the second predicted bounding box belongs, such as at least one of live status and body size), to obtain the target detection result of the image to be detected; then, based on the target detection result, the breeding pair is determined. The target detection model identifies the target population (e.g., the survival rate or growth rate of the target animals within the monitoring point). Based on this, corresponding control measures are implemented (e.g., if a decrease in survival rate is detected, an alarm is automatically issued; if a slowdown in growth rate is detected, the amount or frequency of feeding is automatically increased). The higher the accuracy of the model parameters of the target detection model, the higher the accuracy of the target detection results in the output image. Therefore, the accuracy of triggering corresponding control measures based on the target detection results is also higher.

[0165] The target detection method in this embodiment first extracts multiple candidate bounding boxes using a preset region of interest extraction model during the target detection process. Then, a third specified number of candidate bounding boxes are randomly sampled from these candidate bounding boxes as second reference bounding boxes. For each second reference bounding box, a bounding box prediction sub-model predicts the bounding box based on the second reference bounding box to obtain a second predicted bounding box. A classification sub-model predicts the category of the second predicted bounding box to obtain a second predicted category. Then, based on the second predicted bounding box and the second predicted category corresponding to each second reference bounding box, a target detection result for the image to be detected is generated. During the model training phase, the bounding box prediction sub-model predicts the first predicted bounding box based on the first reference bounding box. Then, based on the first predicted bounding box and its corresponding actual bounding box, the target detection model to be trained continuously learns the bounding box distribution, enabling... The predicted first bounding box is closer to the corresponding actual bounding box, thereby improving the bounding box prediction accuracy, model generalization, and data transferability of the trained object detection model. Furthermore, the set of comparison results used to determine the bounding box regression loss value includes not only the first comparison result representing the similarity of the bounding box distribution, but also the second comparison result representing the degree of overlap of the bounding box coordinates. The bounding box regression loss value is then obtained based on the first and second comparison results corresponding to each first reference bounding box. This results in the bounding box regression loss value containing the regression loss obtained from the coarse-grained comparison dimension based on the similarity of the bounding box distribution, as well as the regression loss obtained from the fine-grained comparison dimension based on the degree of overlap of the bounding box coordinates. This improves the accuracy of the bounding box regression loss value, thereby further improving the accuracy of the model parameters updated based on the bounding box regression loss value.

[0166] It should be noted that this embodiment in this application is based on the same inventive concept as the previous embodiment in this application. Therefore, the specific implementation of this embodiment can be referred to the implementation of the aforementioned model training method, and the repeated parts will not be described again.

[0167] Corresponding to the above Figures 1 to 4b Based on the same technical concept, this application also provides a model training apparatus for the described model training method. Figure 7 This is a schematic diagram of the module composition of the model training device provided in the embodiments of this application. The device is used to perform... Figures 1 to 4b The model training method described, such as Figure 7 As shown, the device includes:

[0168] The bounding box acquisition module 702 is configured to acquire a first bounding box subset from a first candidate bounding box set, and to acquire the actual bounding boxes corresponding to each first reference bounding box in the first bounding box subset; the first bounding box subset includes a first specified number of first reference bounding boxes, and the first candidate bounding box set is obtained by extracting the target region from the sample image dataset using a preset region of interest extraction model.

[0169] The model training module 704 is configured to input the first reference bounding box and the actual bounding box into the target detection model to be trained for iterative model training until the result of this iteration meets the preset model iteration training termination condition, thereby obtaining the trained target detection model; wherein, the target detection model includes a bounding box prediction sub-model; the specific implementation of each model training is as follows:

[0170] For each first reference bounding box: the bounding box prediction sub-model predicts bounding boxes based on the first reference bounding boxes to obtain first predicted bounding boxes; based on the actual bounding boxes corresponding to the first reference bounding boxes and the first predicted bounding boxes corresponding to the first reference bounding boxes, a bounding box comparison result set is generated; the bounding box comparison result set includes a first comparison result characterizing the similarity of bounding box distributions and a second comparison result characterizing the overlap of bounding box coordinates; based on the first comparison result and the second comparison result corresponding to the first reference bounding boxes in the first bounding box subset, a bounding box regression loss value is determined; the parameters of the bounding box prediction sub-model are updated based on the bounding box regression loss value.

[0171] In the model training apparatus of this application embodiment, during the model training phase, the bounding box prediction sub-model predicts a first predicted bounding box based on a first reference bounding box. Then, based on the first predicted bounding box and its corresponding actual bounding box, the target detection model to be trained continuously learns the bounding box distribution, making the predicted first predicted bounding box closer to the corresponding actual bounding box, thereby improving the bounding box prediction accuracy, model generalization, and data transferability of the trained target detection model. Furthermore, the set of comparison results used to determine the bounding box regression loss value includes not only the first comparison result representing the similarity of the bounding box distribution but also the second comparison result representing the overlap of the bounding box coordinates. The bounding box regression loss value is then obtained based on the first and second comparison results corresponding to each first reference bounding box. This makes the bounding box regression loss value include the regression loss obtained based on the coarse-grained comparison dimension of the similarity of the bounding box distribution and the regression loss obtained based on the fine-grained comparison dimension of the overlap of the bounding box coordinates. This can improve the accuracy of the bounding box regression loss value, thereby further improving the accuracy of the model parameters updated based on the bounding box regression loss value.

[0172] It should be noted that the embodiments of the model training device in this application and the embodiments of the model training method in this application are based on the same inventive concept. Therefore, the specific implementation of this embodiment can be referred to the implementation of the corresponding model training method mentioned above, and the repeated parts will not be described again.

[0173] Corresponding to the above Figures 5 to 6 Based on the same technical concept, this application also provides a target detection device in its embodiments regarding the target detection method described. Figure 8 This is a schematic diagram of the module composition of the target detection device provided in the embodiments of this application. The device is used to perform... Figures 5 to 6 The described object detection method, such as Figure 8 As shown, the device includes:

[0174] The bounding box acquisition module 802 is configured to acquire a second bounding box subset corresponding to the image to be detected from a second candidate bounding box set; the second bounding box subset includes a third specified number of second reference bounding boxes, and the second candidate bounding box set is obtained by extracting the target region of the image to be detected using a preset region of interest extraction model.

[0175] The target detection module 804 is configured to input the second reference bounding box into the target detection model for target detection, and obtain the second predicted bounding box and the second category prediction result corresponding to each second reference bounding box;

[0176] The detection result generation module 806 is configured to generate target detection results for the image to be detected based on the second predicted bounding box corresponding to each of the second reference bounding boxes and the second category prediction results.

[0177] In the target detection device of this application embodiment, during the target detection process, firstly, multiple candidate bounding boxes are extracted using a preset region of interest extraction model. Then, a third specified number of candidate bounding boxes are randomly sampled from the candidate bounding boxes as second reference bounding boxes. For each second reference bounding box, a bounding box prediction sub-model predicts the bounding box based on the second reference bounding box to obtain a second predicted bounding box. A classification sub-model predicts the category of the second predicted bounding box to obtain a second predicted category. Then, based on the second predicted bounding box and the second predicted category corresponding to each second reference bounding box, the target detection result of the image to be detected is generated. During the model training phase, the bounding box prediction sub-model predicts the first predicted bounding box based on the first reference bounding box. Then, based on the first predicted bounding box and its corresponding actual bounding box, the target detection model to be trained continuously learns the bounding box distribution. This approach ensures that the predicted first bounding box is closer to the corresponding actual bounding box, thereby improving the bounding box prediction accuracy, model generalization, and data transferability of the trained object detection model. Furthermore, the set of comparison results used to determine the bounding box regression loss value includes not only the first comparison result representing the similarity of the bounding box distribution but also the second comparison result representing the overlap of the bounding box coordinates. The bounding box regression loss value is then obtained based on the first and second comparison results corresponding to each first reference bounding box. This results in the bounding box regression loss value containing the regression loss obtained from the coarse-grained comparison dimension based on the similarity of the bounding box distribution and the regression loss obtained from the fine-grained comparison dimension based on the overlap of the bounding box coordinates. This improves the accuracy of the bounding box regression loss value, thereby further improving the accuracy of the model parameters updated based on the bounding box regression loss value.

[0178] It should be noted that the embodiments of the target detection device in this application and the embodiments of the target detection method in this application are based on the same inventive concept. Therefore, the specific implementation of this embodiment can be referred to the implementation of the corresponding target detection method mentioned above, and the repeated parts will not be described again.

[0179] Furthermore, corresponding to the above Figures 1 to 4b Based on the same technical concept, embodiments of this application also provide a computer device for executing the aforementioned model training method or object detection method, such as... Figure 9 As shown.

[0180] Computer devices can vary significantly due to differences in configuration or performance. They may include one or more processors 901 and memory 902, with memory 902 storing one or more application programs or data. Memory 902 can be temporary or persistent storage. The application programs stored in memory 902 may include one or more modules (not shown), each module including a series of computer-executable instructions for the computer device. Furthermore, processor 901 may be configured to communicate with memory 902 and execute the series of computer-executable instructions stored in memory 902 on the computer device. The computer device may also include one or more power supplies 903, one or more wired or wireless network interfaces 904, one or more input / output interfaces 905, one or more keyboards 906, etc.

[0181] In one specific embodiment, the computer device includes a memory and one or more programs, wherein the one or more programs are stored in the memory, and the one or more programs may include one or more modules, and each module may include a series of computer-executable instructions for use in the computer device, and is configured to be executed by one or more processors. The one or more programs include computer-executable instructions for performing the following:

[0182] A first subset of bounding boxes is obtained from a first set of candidate bounding boxes, and the actual bounding boxes corresponding to each first reference bounding box in the first subset of bounding boxes are obtained respectively; the first subset of bounding boxes includes a first specified number of first reference bounding boxes, and the first set of candidate bounding boxes is obtained by extracting the target region from the sample image dataset using a preset region of interest extraction model.

[0183] The first reference bounding box and the actual bounding box are input into the target detection model to be trained for iterative training until the result of this iteration meets the preset termination condition, thus obtaining the trained target detection model; wherein, the target detection model includes a bounding box prediction sub-model; the specific implementation of each model training is as follows:

[0184] For each first reference bounding box: the bounding box prediction sub-model predicts bounding boxes based on the first reference bounding boxes to obtain first predicted bounding boxes; based on the actual bounding boxes corresponding to the first reference bounding boxes and the first predicted bounding boxes corresponding to the first reference bounding boxes, a bounding box comparison result set is generated; the bounding box comparison result set includes a first comparison result characterizing the similarity of bounding box distributions and a second comparison result characterizing the overlap of bounding box coordinates; based on the first comparison result and the second comparison result corresponding to the first reference bounding boxes in the first bounding box subset, a bounding box regression loss value is determined; the parameters of the bounding box prediction sub-model are updated based on the bounding box regression loss value.

[0185] In another specific embodiment, the computer device includes a memory and one or more programs, wherein the one or more programs are stored in the memory, and the one or more programs may include one or more modules, and each module may include a series of computer-executable instructions for use in the computer device, and is configured to be executed by one or more processors. The one or more programs include computer-executable instructions for performing the following:

[0186] A subset of second bounding boxes corresponding to the image to be detected is obtained from the set of second candidate bounding boxes; the subset of second bounding boxes includes a third specified number of second reference bounding boxes, and the set of second candidate bounding boxes is obtained by extracting the target region of the image to be detected using a preset region of interest extraction model; the second reference bounding boxes are input into the target detection model for target detection to obtain the second predicted bounding box and the second category prediction result corresponding to each second reference bounding box; based on the second predicted bounding box and the second category prediction result corresponding to each second reference bounding box, the target detection result of the image to be detected is generated.

[0187] In the computer device of this application embodiment, during the model training phase, the bounding box prediction sub-model predicts a first predicted bounding box based on a first reference bounding box. Then, based on the first predicted bounding box and its corresponding actual bounding box, the target detection model to be trained continuously learns the bounding box distribution, making the predicted first predicted bounding box closer to the corresponding actual bounding box, thereby improving the bounding box prediction accuracy, model generalization, and data transferability of the trained target detection model. Furthermore, the set of comparison results used to determine the bounding box regression loss value includes not only the first comparison result representing the similarity of the bounding box distribution, but also the second comparison result representing the overlap of the bounding box coordinates. The bounding box regression loss value is then obtained based on the first and second comparison results corresponding to each first reference bounding box. This makes the bounding box regression loss value include the regression loss obtained based on the coarse-grained comparison dimension of the similarity of the bounding box distribution and the regression loss obtained based on the fine-grained comparison dimension of the overlap of the bounding box coordinates. This can improve the accuracy of the bounding box regression loss value, thereby further improving the accuracy of the model parameters updated based on the bounding box regression loss value.

[0188] It should be noted that the embodiments concerning computer devices in this application and the embodiments concerning model training methods in this application are based on the same inventive concept. Therefore, the specific implementation of this embodiment can be referred to the implementation of the corresponding model training method described above, and the repeated parts will not be described again.

[0189] Furthermore, corresponding to the above Figures 1 to 4b Based on the same technical concept, this application also provides a storage medium for storing computer-executable instructions. In one specific embodiment, the storage medium can be a USB flash drive, optical disc, hard disk, etc. When the computer-executable instructions stored in the storage medium are executed by a processor, they can achieve the following process:

[0190] A first subset of bounding boxes is obtained from a first set of candidate bounding boxes, and the actual bounding boxes corresponding to each first reference bounding box in the first subset of bounding boxes are obtained respectively; the first subset of bounding boxes includes a first specified number of first reference bounding boxes, and the first set of candidate bounding boxes is obtained by extracting target regions from the sample image dataset using a preset region of interest extraction model; the first reference bounding boxes and the actual bounding boxes are input into the target detection model to be trained for iterative training until the result of this iteration of training meets the preset termination condition of the iteration of training, thus obtaining the trained target detection model; wherein, the target detection model includes a bounding box prediction sub-model; the specific implementation of each model training is as follows:

[0191] For each first reference bounding box: the bounding box prediction sub-model predicts bounding boxes based on the first reference bounding boxes to obtain first predicted bounding boxes; based on the actual bounding boxes corresponding to the first reference bounding boxes and the first predicted bounding boxes corresponding to the first reference bounding boxes, a bounding box comparison result set is generated; the bounding box comparison result set includes a first comparison result characterizing the similarity of bounding box distributions and a second comparison result characterizing the overlap of bounding box coordinates; based on the first comparison result and the second comparison result corresponding to the first reference bounding boxes in the first bounding box subset, a bounding box regression loss value is determined; the parameters of the bounding box prediction sub-model are updated based on the bounding box regression loss value.

[0192] In another specific embodiment, the storage medium can be a USB flash drive, optical disc, hard disk, etc., and the computer-executable instructions stored on the storage medium can achieve the following process when executed by the processor:

[0193] A subset of second bounding boxes corresponding to the image to be detected is obtained from a set of second candidate bounding boxes. The second bounding box subset includes a third specified number of second reference bounding boxes. The set of second candidate bounding boxes is obtained by extracting the target region of the image to be detected using a preset region of interest extraction model. The second reference bounding boxes are input into a target detection model for target detection to obtain a second predicted bounding box and a second category prediction result corresponding to each second reference bounding box. Based on the second predicted bounding box and the second category prediction result corresponding to each second reference bounding box, a target detection result for the image to be detected is generated.

[0194] In the embodiments of this application, when the computer-executable instructions stored in the storage medium are executed by the processor, during the model training phase, the bounding box prediction sub-model predicts a first predicted bounding box based on a first reference bounding box. Then, based on the first predicted bounding box and its corresponding actual bounding box, the target detection model to be trained continuously learns the bounding box distribution, making the predicted first predicted bounding box closer to the corresponding actual bounding box, thereby improving the bounding box prediction accuracy, model generalization, and data transferability of the trained target detection model. Furthermore, the set of comparison results used to determine the bounding box regression loss value includes not only the first comparison result representing the similarity of the bounding box distribution, but also the second comparison result representing the overlap of the bounding box coordinates. Then, based on the first and second comparison results corresponding to each first reference bounding box, the bounding box regression loss value is obtained. This makes the bounding box regression loss value include the regression loss obtained based on the coarse-grained comparison dimension based on the similarity of the bounding box distribution, and the regression loss obtained based on the fine-grained comparison dimension based on the overlap of the bounding box coordinates. This can improve the accuracy of the bounding box regression loss value, thereby further improving the accuracy of the model parameters updated based on the bounding box regression loss value.

[0195] It should be noted that the embodiments concerning storage media in this application and the embodiments concerning model training methods in this application are based on the same inventive concept. Therefore, the specific implementation of this embodiment can be referred to the implementation of the corresponding model training methods mentioned above, and the repeated parts will not be described again.

[0196] The foregoing has described specific embodiments of this application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired results. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous. Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, embodiments of this application may take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Moreover, this application may take the form of a computer program product embodied on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing device to produce a machine, such that the instructions, which are executable by the processor of the computer or other programmable data processing device, produce instructions for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The functions specified in one or more boxes. These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable apparatus for implementing the process. Figure 1 One or more processes and / or boxes Figure 1The steps of the function specified in one or more boxes.

[0197] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory. Memory may include non-persistent storage in computer-readable media, random access memory (RAM), and / or non-volatile memory such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media. Computer-readable media includes both permanent and non-persistent, removable and non-removable media that can store information by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information that can be accessed by the computing device. As defined in this article, computer-readable media do not include transient media, such as modulated data signals and carrier waves.

[0198] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element. Embodiments of this application can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. One or more embodiments of this application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In a distributed computing environment, program modules can reside in local and remote computer storage media, including storage devices. The various embodiments in this application are described in a progressive manner; similar or identical parts between embodiments can be referred to mutually, and each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple, and relevant details can be found in the description of the method embodiments. The above descriptions are merely embodiments of this document and are not intended to limit this document. Various modifications and variations can be made to this document by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this document should be included within the scope of the claims of this document.

Claims

1. A model training method, characterized in that, The method includes: A first subset of bounding boxes is obtained from a first set of candidate bounding boxes, and the actual bounding boxes corresponding to each first reference bounding box in the first subset of bounding boxes are obtained respectively; the first subset of bounding boxes includes a first specified number of first reference bounding boxes, and the first set of candidate bounding boxes is obtained by extracting the target region from the sample image dataset using a preset region of interest extraction model. The first reference bounding box and the actual bounding box are input into the target detection model to be trained for iterative training until the result of this iteration meets the preset termination condition, thus obtaining the trained target detection model; wherein, the target detection model includes a bounding box prediction sub-model; the specific implementation of each model training is as follows: For each of the first reference bounding boxes: the bounding box prediction sub-model performs bounding box prediction based on the first reference bounding box to obtain a first predicted bounding box; based on the actual bounding box corresponding to the first reference bounding box and the first predicted bounding box corresponding to the first reference bounding box, a bounding box comparison result set is generated; the bounding box comparison result set includes a first comparison result characterizing the similarity of the bounding box distribution and a second comparison result characterizing the overlap of the bounding box coordinates; Based on the first alignment result and the second alignment result corresponding to the first reference bounding box in the first bounding box subset, the bounding box regression loss value is determined. The parameters of the bounding box prediction sub-model are updated based on the bounding box regression loss value.

2. The method according to claim 1, characterized in that, The step of generating a bounding box alignment result set based on the actual bounding box corresponding to the first reference bounding box and the first predicted bounding box corresponding to the first reference bounding box includes: Based on the actual bounding box and the first predicted bounding box corresponding to the first reference bounding box, the relative entropy KL divergence is calculated to obtain the first alignment result; based on the actual bounding box and the first predicted bounding box corresponding to the first reference bounding box, the bounding box intersection-union ratio loss is calculated to obtain the second alignment result.

3. The method according to claim 2, characterized in that, The step of determining the bounding box regression loss value based on the first alignment result and the second alignment result corresponding to the first reference bounding box in the first bounding box subset includes: The sub-regression loss values ​​corresponding to the first reference bounding boxes in the first bounding box subset are determined. The sub-regression loss value corresponding to each first reference bounding box is determined based on target information, which includes one or a combination of the following: the similarity of the bounding box distribution represented by the first comparison result corresponding to the first reference bounding box, and the degree of overlap of the bounding box coordinates represented by the second comparison result. The bounding box regression loss value is determined based on the sub-regression loss value corresponding to each of the first reference bounding boxes in the first bounding box subset.

4. The method according to claim 2, characterized in that, The step of calculating the relative entropy KL divergence based on the actual bounding box and the first predicted bounding box corresponding to the first reference bounding box to obtain the first alignment result includes: Determine a first probability distribution of the actual bounding box corresponding to the first reference bounding box, and determine a second probability distribution of the first predicted bounding box corresponding to the first reference bounding box; Calculate the KL divergence value between the first probability distribution and the second probability distribution; the KL divergence value is used to characterize the distribution similarity between the first predicted bounding box and the actual bounding box; Based on the KL divergence value, the first alignment result corresponding to the first reference bounding box is determined.

5. The method according to claim 2, characterized in that, The step of calculating the boundary box intersection-union ratio (IUU) loss based on the actual boundary box corresponding to the first reference boundary box and the first predicted boundary box to obtain the second alignment result includes: The first intersection-union ratio (IURR) loss is calculated by performing a boundary box intersection-union ratio (BUOR) loss on the actual boundary box corresponding to the first reference boundary box and the first predicted boundary box corresponding to the first reference boundary box. Based on the first intersection-union ratio loss, a second alignment result corresponding to the first reference bounding box is determined; the bounding box intersection-union ratio loss is used to characterize the degree of overlap of bounding box coordinates.

6. The method according to claim 5, characterized in that, The step of determining the second alignment result corresponding to the first reference bounding box based on the first intersection-union loss includes: A set of comparison bounding boxes is determined among the first predicted bounding boxes corresponding to the first specified number of first reference bounding boxes; the set of comparison bounding boxes includes other first predicted bounding boxes besides the first predicted bounding boxes corresponding to the first reference bounding boxes, or other first predicted bounding boxes that do not contain the target object enclosed by the first reference bounding boxes. The intersection-union ratio (IURR) loss is calculated for the actual bounding box corresponding to the first reference bounding box and the other first predicted bounding boxes to obtain the second IURR loss. Based on the first cross-union ratio loss and the second cross-union ratio loss, the second alignment result corresponding to the first reference bounding box is determined.

7. The method according to claim 3, characterized in that, The target detection model further includes a bounding box classification sub-model; the specific implementation of each model training also includes: the bounding box classification sub-model classifies the first reference bounding box or the first predicted bounding box to obtain the first category prediction result; The target information also includes a category matching result between the first predicted category represented by the first category prediction result corresponding to the first reference bounding box and the actual category of the first reference bounding box. If the category matching result is that the first predicted category does not match the actual category, then the sub-regression loss value corresponding to the first reference bounding box is zero. If the category matching result is that the first predicted category matches the actual category, then the sub-regression loss value corresponding to the first reference bounding box is a sub-regression loss value determined based on at least one of the first regression loss component corresponding to the similarity of the bounding box distribution and the second regression loss component corresponding to the overlap of the bounding box coordinates.

8. The method according to claim 1, characterized in that, The method further includes: The sample image dataset is input into a preset region of interest extraction model to extract the region of interest, resulting in a first set of candidate bounding boxes; the first set of candidate bounding boxes includes a second specified number of candidate bounding boxes; the second specified number is greater than the first specified number. Obtaining the first subset of bounding boxes from the first set of candidate bounding boxes includes: randomly selecting the first specified number of candidate bounding boxes from the second specified number of candidate bounding boxes as the first reference bounding boxes to obtain the first subset of bounding boxes.

9. A target detection method, characterized in that, The method includes: Obtain a subset of second bounding boxes corresponding to the image to be detected from the set of second candidate bounding boxes; the subset of second bounding boxes includes a third specified number of second reference bounding boxes, and the set of second candidate bounding boxes is obtained by extracting the target region of the image to be detected using a preset region of interest extraction model; The second reference bounding box is input into the target detection model for target detection, and the second predicted bounding box and the second category prediction result corresponding to each second reference bounding box are obtained. The target detection model is trained based on the model training method of any one of claims 1-8. Based on the second predicted bounding box and the second category prediction result corresponding to each second reference bounding box, the target detection result of the image to be detected is generated.

10. The method according to claim 9, characterized in that, The target detection model includes a bounding box prediction sub-model and a bounding box classification sub-model; For each second reference bounding box: during the target detection process, the bounding box prediction sub-model predicts bounding boxes based on the second reference bounding boxes to obtain the second predicted bounding box corresponding to the second reference bounding box; the bounding box classification sub-model classifies the second reference bounding box or the second predicted bounding box to obtain the second category prediction result corresponding to the second reference bounding box.

11. A model training device, characterized in that, The device includes: The bounding box acquisition module is configured to acquire a first bounding box subset from a first candidate bounding box set, and to acquire the actual bounding boxes corresponding to each first reference bounding box in the first bounding box subset; the first bounding box subset includes a first specified number of first reference bounding boxes, and the first candidate bounding box set is obtained by extracting the target region from the sample image dataset using a preset region of interest extraction model. The model training module is configured to input the first reference bounding box and the actual bounding box into the target detection model to be trained for iterative training until the result of this iteration meets the preset model iteration training termination condition, thus obtaining the trained target detection model; wherein, the target detection model includes a bounding box prediction sub-model; the specific implementation of each model training is as follows: For each first reference bounding box: the bounding box prediction sub-model predicts bounding boxes based on the first reference bounding boxes to obtain first predicted bounding boxes; based on the actual bounding boxes corresponding to the first reference bounding boxes and the first predicted bounding boxes corresponding to the first reference bounding boxes, a bounding box comparison result set is generated; the bounding box comparison result set includes a first comparison result characterizing the similarity of bounding box distributions and a second comparison result characterizing the overlap of bounding box coordinates; based on the first comparison result and the second comparison result corresponding to the first reference bounding boxes in the first bounding box subset, a bounding box regression loss value is determined; the parameters of the bounding box prediction sub-model are updated based on the bounding box regression loss value.

12. A target detection device, characterized in that, The device includes: The bounding box acquisition module is configured to acquire a second bounding box subset corresponding to the image to be detected from a second candidate bounding box set; the second bounding box subset includes a third specified number of second reference bounding boxes, and the second candidate bounding box set is obtained by extracting the target region of the image to be detected using a preset region of interest extraction model. The object detection module is configured to input the second reference bounding box into the object detection model for object detection, and obtain the second predicted bounding box and the second category prediction result corresponding to each second reference bounding box. The object detection model is trained based on the model training method of any one of claims 1-8. The detection result generation module is configured to generate target detection results for the image to be detected based on the second predicted bounding box corresponding to each second reference bounding box and the second category prediction result.

13. A computer device, characterized in that, The device includes: Processor; and A memory configured to store computer-executable instructions configured to be executed by the processor, the executable instructions including steps for performing the method as described in any one of claims 1-8 or 9-10.

14. A storage medium, characterized in that, The storage medium is used to store computer-executable instructions that cause the computer to perform the method as described in any one of claims 1-8 or 9-10.

Images