Model Training Method, Device, Electronic Device, Medium, and Program Product

Abstract

The present disclosure provides a model training method, apparatus, electronic device, medium, and program product, which relate to the field of data processing, and particularly to technical fields such as image understanding, computer vision, object detection, and deep learning. The specific implementation solution is as follows: Obtain training data, where the training data includes sample images, and each sample image includes at least one labeled GT box, and one GT box is used to frame a target object in the sample image; Input the sample image into an initial network model for prediction to obtain a prediction result, where the prediction result includes prediction boundary information; Train the initial network model based on the prediction boundary information and the true boundary information to obtain a target network model, and the target network model is a model for detecting target objects in images, and the true boundary information includes the true relative position information between the boundary of each GT box and the corresponding target center point. The present disclosure can improve the training effect of the model.

Description

Technical Field

[0001] The present disclosure relates to the field of data processing, and in particular, to technical fields such as image understanding, computer vision, object detection, and deep learning. Specifically, it relates to a model training method, apparatus, electronic device, medium, and program product. Background Art

[0002] In related technologies, object detection is an important research direction in the field of computer vision and is also the basis for other complex vision tasks. As the cornerstone of image understanding and computer vision, object detection is the basis for solving higher-level vision tasks such as segmentation, scene understanding, object tracking, image description, and event detection. Small object detection has long been a difficult point in object detection, which aims to accurately detect small objects with extremely few visual features in an image. Due to the small pixel ratio of small objects, they have basic characteristics such as small coverage area and little contained information. Therefore, it has always been very difficult to improve the accuracy of small object detection. In related technologies, the anchor box mechanism is usually used to mark the target objects in the image to complete the training process of the object detection model. Summary of the Invention

[0003] The present disclosure provides a model training method, apparatus, electronic device, medium, and program product.

[0004] According to a first aspect of the present disclosure, there is provided a model training method, including:

[0005] Obtain training data, where the training data includes sample images, and at least one labeled GT box is included in the sample images, and one GT box is used to frame one target object in the sample image;

[0006] Input the sample image into an initial network model for prediction to obtain a prediction result, where the prediction result includes prediction boundary information. Among them, the initial network model is used to determine multiple target center points in the sample image, and the initial network model is further used to determine the target center point corresponding to each GT box among the multiple target center points, and predict the prediction boundary information based on the target center point corresponding to the GT box. The prediction boundary information includes the predicted relative position information between the boundary of each GT box and the corresponding target center point. The target center point corresponding to the GT box includes: the target center point located inside the GT box, or the target center point located outside the GT box;

[0007] Train the initial network model based on the prediction boundary information and the true boundary information to obtain a target network model, where the target network model is a model for detecting target objects in an image, and the true boundary information includes the true relative position information between the boundary of each GT box and the corresponding target center point.

[0008] According to a second aspect of the present disclosure, there is provided a model training apparatus, including:

[0009] An acquisition module, configured to acquire training data, where the training data includes sample images, and at least one labeled GT box is included in the sample images, and one GT box is used to frame a target object in the sample images;

[0010] A prediction module, configured to input the sample images into an initial network model for prediction to obtain a prediction result, where the prediction result includes prediction boundary information. Among them, the initial network model is used to determine a plurality of target center points in the sample images, and the initial network model is further used to determine, among the plurality of target center points, the target center point corresponding to each GT box, and predict the prediction boundary information based on the target center point corresponding to the GT box. The prediction boundary information includes the predicted relative position information between the boundary of each GT box and the corresponding target center point. The target center point corresponding to the GT box includes: a target center point located inside the GT box, or a target center point located outside the GT box;

[0011] A training module, configured to train the initial network model based on the prediction boundary information and the true boundary information to obtain a target network model, where the target network model is a model for detecting target objects in images, and the true boundary information includes the true relative position information between the boundary of each GT box and the corresponding target center point.

[0012] According to a third aspect of the present disclosure, there is provided an electronic device, including:

[0013] At least one processor; and

[0014] A memory communicatively connected to the at least one processor; wherein,

[0015] The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method described in the first aspect above.

[0016] According to a fourth aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium storing computer instructions, where the computer instructions are used to cause the computer to execute the method described in the first aspect above.

[0017] According to a fifth aspect of the present disclosure, there is provided a computer program product, including a computer program, where the computer program implements the method described in the first aspect when executed by a processor.

[0018] In the embodiments of the present disclosure, since the target center point corresponding to the GT box can be either the target center point located inside the GT box or the target center point located outside the GT box, therefore, regardless of whether the target center point is included in the GT box, the target center point corresponding to the GT box can be determined, and then the predicted 0 boundary information of each GT box can be generated based on the target center point corresponding to the GT box. That is, for target objects of various sizes, their boundary information can be predicted, and

[0019] the model can be trained based on the prediction results. In this way, it is beneficial to improve the training effect of the model.

[0020] Description of the Drawings The drawings are used to better understand the solution and do not constitute a limitation to the present disclosure. Among them:

[0021] is a flowchart of a model training method provided by an embodiment of the present disclosure; Figure 1 is a schematic diagram showing that the target center point is located inside the first GT box in an embodiment of the present disclosure;

[0022] Figure 2 is a schematic diagram showing that the target center point is located outside the first GT box in an embodiment of the present disclosure;

[0023] Figure 3 is a schematic diagram showing that the target center point is located outside the first GT box, and a prior box is determined outside the first GT box in an embodiment of the present disclosure;

[0024] Figure 4 is one of the structural schematic diagrams of a model training device provided by an embodiment of the present disclosure;

[0025] is another structural schematic diagram of a model training device provided by an embodiment of the present disclosure;

[0026] Figure 5 is a block diagram of an electronic device for implementing the model training method provided by an embodiment of the present disclosure.

[0027] Figure 6

[0028] Figure 7 Specific Embodiments

[0029]

[0030] The following makes an illustration of exemplary embodiments of the present disclosure with reference to the drawings, including various details of the embodiments of the present disclosure to facilitate understanding, and they should be considered merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted below.​​​

[0031] Please refer to Figure 1 , Figure 1 which is a schematic flowchart of a model training method provided by an embodiment of the present disclosure. The model training method includes the following steps:

[0032] Step S101, obtain training data, where the training data includes sample images, and at least one annotation (Ground Truth, GT) box is included in the sample images. One GT box is used to frame one target object in the sample image;

[0033] Step S102, input the sample image into an initial network model for prediction to obtain a prediction result. The prediction result includes prediction boundary information. Among them, the initial network model is used to determine multiple target center points in the sample image, and the initial network model is further used to determine the target center point corresponding to each GT box among the multiple target center points, and predict the prediction boundary information based on the target center point corresponding to the GT box. The prediction boundary information includes the predicted relative position information between the boundary of each GT box and the corresponding target center point. The target center point corresponding to the GT box includes: the target center point located inside the GT box, or the target center point located outside the GT box;

[0034] Step S103, train the initial network model based on the prediction boundary information and the true boundary information to obtain a target network model. The target network model is a model used to detect target objects in images. The true boundary information includes the true relative position information between the boundary of each GT box and the corresponding target center point.

[0035] Among them, the above target network model can be a network model used for classifying or identifying target objects in various target recognition scenarios. Specifically, the target network model can serve as the basis for higher-level vision tasks such as image segmentation, scene understanding, target tracking, image description, and event detection.

[0036] The above sample images can be sample images obtained by pre-annotating target objects in images. Among them, the target objects can be objects to be recognized or classified in various scenarios. For example, in the autonomous driving scenario, the target objects can include various traffic elements such as pedestrians, vehicles, traffic lights, and obstacles.

[0037] It can be understood that before model training, a large number of images containing the target object can be obtained in advance. Then, the target object in the image is labeled with a GT box, that is, the position, size and other information of the target object in the image are marked through the GT box. In this way, during the subsequent model training process, the position of the target object in the sample image can be indicated by the GT box, so that the model can learn how to accurately identify the target object from the sample image.

[0038] The above initial network model can be the network backbone used to train the image recognition model in the related art. During the model training process, the initial network model can perform downsampling on the sample image, and the sampling points can be used as the target center points. For example, the initial network model can divide the sample image into grids and select the center point of each grid as the target center point. Among them, the features of the target center point can be used as the features in its grid.

[0039] In the related art, when training a target detection model, usually a box is marked in the image, and then the target center point is determined in the image. The distance between the target center point located inside the box and each side of the box is used to predict the loss, so as to complete the model training process. However, when the target object is a small target, the size of the corresponding box is quite small. In this case, there may be no target center point in the box, which leads to the inability to predict the boundary information of the small target, and further leads to the problem that the detection effect of the trained target detection model for small targets is poor.

[0040] In the embodiment of the present disclosure, since the target center point corresponding to the GT box can be either the target center point located inside the GT box or the target center point located outside the GT box, when the GT box does not include the target center point, the target center point corresponding to the GT box can be determined from outside the GT box, and the boundary information of the GT box can be predicted based on this target center point. In this way, during the training process, the initial network model can effectively learn to predict the boundary information of small targets, thereby improving the recognition effect of the trained target network model for small targets.

[0041] The above prediction boundary information may include: the relative position information between the target center point corresponding to the GT box predicted by the initial network model and each side of the GT box. The true boundary information may include: the true relative position information between the target center point corresponding to the GT box and each side of the GT box. In this way, a loss function can be constructed based on the prediction result and the true result of the initial network model, and the parameters of the initial network model can be optimized based on the constructed loss function to obtain the target network model. Among them, the true boundary information can be directly obtained according to the relative position between the target center point and the GT box in the sample image.

[0042] Specifically, after training the target network model, only the image to be recognized needs to be input into the target network model, and the target network model can accurately recognize the target object in the image to be recognized and predict the boundary information of the target object, and this boundary information can be used to restore the GT box corresponding to the target object. In this way, the process of recognizing the target object in the image based on the target network model is realized.

[0043] It should be noted that the specific prediction process of the above initial network model for predicting the prediction boundary information based on the target center point corresponding to the GT box is similar to the prediction process of the target detection model in the related art. For example, it can be similar to the boundary prediction process in the process of training the target detection model using the anchor box mechanism in the related art.

[0044] In this embodiment, since the target center point corresponding to the GT box: can be either the target center point located inside the GT box or the target center point located outside the GT box, therefore, whether the GT box contains the target center point or not, the target center point corresponding to the GT box can be determined, and then the prediction boundary information of each GT box can be generated based on the target center point corresponding to the GT box. That is, for target objects of various sizes, their boundary information can be predicted, and the model can be trained based on the prediction results. In this way, it is beneficial to improve the training effect of the model.

[0045] Optionally, when there is at least one target center point in the area enclosed by the first GT box, the initial network model is used to determine the at least one target center point as the target center point corresponding to the first GT box, and the first GT box is any one of the at least one GT box;

[0046] When the area enclosed by the first GT box does not include the target center point, the initial network model is used to determine a prior box corresponding to the first GT box in the sample image. The first GT box is located within the prior box, and the prior box includes at least one target center point. The initial network model is also used to determine at least one target center point within the prior box as the target center point corresponding to the first GT box.

[0047] Among them, in the process of determining the target center points corresponding to all GT boxes in each sample image, the same method as the above-mentioned first GT box can be used for determination.

[0048] Specifically, in the process of determining the target center point corresponding to the first GT box, first, it is possible to check whether there is a target center point within the first GT box. As Figure 2 shown, if the first GT box includes a target center point, the target center point within the first GT box is determined as the target center point corresponding to the first GT box. As Figures 3 to 4 shown, when the first GT box does not include a target center point, a prior box can be determined around the first GT box. Among them, the distance between the prior box and the first GT box can be relatively close, and the first GT box is located inside the prior box. At the same time, the prior box also includes at least one target center point. Since the distance between the prior box and the first GT box can be relatively close, the distance between the target center point inside the prior box and the first GT box is also relatively close. Therefore, the features at the target center point inside the prior box can be used to predict the boundary information of the first GT box, so that at least one target center point included in the prior box can be determined as the target center point corresponding to the first GT box.

[0049] It can be understood that the number of target center points corresponding to the first GT box can be more than one. When the first GT box corresponds to at least two target center points, the boundary information of the first GT information can be predicted based on each target center point to obtain at least two prediction results. Then, post-processing is performed on the at least two prediction results to obtain the final prediction result. Among them, the post-processing can be a conventional processing method in the related art. For example, the average value of the at least two prediction results can be calculated to obtain the final prediction result.

[0050] In this embodiment, when at least one target center point is included in the area framed by the first GT box, the initial network model is used to determine the at least one target center point as the target center point corresponding to the first GT box; when the target center point is not included in the area framed by the first GT box, at least one target center point within the prior box is determined as the target center point corresponding to the first GT box. In this way, regardless of whether the first GT box includes a target center point, the target center point corresponding to the first GT box can be determined, thereby realizing the process of determining the center points corresponding to each GT box.

[0051] Optionally, the first GT box is a rectangular box, and the prior box is a square box determined with the center point of the first GT box as the center and a target length as the side length, and the four sides of the first GT box are parallel to the four sides of the prior box respectively. Among them, the target length can be set according to the actual scenario.

[0052] In this embodiment, by determining the prior box with the center point of the first GT box as the center and a target length as the side length, and making the four sides of the first GT box parallel to the four sides of the prior box respectively, in this way, it can be ensured that the distances between the first GT box and each side of the prior box are relatively balanced, which is conducive to determining the target center point located outside the first GT box and relatively close to the first GT box as the target center point corresponding to the first GT box. In this way, since the determined target center point is relatively close to the first GT box, the features at the target center point are also closer to the features inside the first GT box, which is further conducive to improving the accuracy of predicting the boundary information of the first GT box based on the target center point.

[0053] Optionally, the training data is the training data in a preset training set, the training data set includes at least two different training data, and before inputting the sample image into the initial network model for prediction to obtain a prediction result, the method further includes:

[0054] Determine a regression range based on the side lengths of the GT boxes in each training data in the preset training set, where the minimum value in the regression range is less than 0, and the maximum value in the regression range is determined based on the target side length, where the target side length is: the longest boundary side length among the boundary side lengths of all GT boxes corresponding to the at least two different training data;

[0055] Among them, the initial network model is used to predict the predicted boundary information within the regression range based on the target center point corresponding to the GT box. The predicted boundary information includes one-dimensional regression vectors corresponding to each boundary in the GT box. The starting point of the regression vector is the target center point, the direction of the regression vector is perpendicular to the corresponding boundary, and the length of the regression vector is within the regression range.

[0056] The above regression range is the value range of the regression vector when the initial network model predicts the regression vector. In the embodiments of the present disclosure, in order to further optimize the regression range, the regression range is specifically reduced to improve the accuracy of regression.

[0057] Specifically, when the GT box is large, it usually includes the target center point inside. At this time, the distance from the target center point to each side of the GT box is less than the longest side length of the GT box. At this time, the regression distance will not exceed the above target side length. When the GT box is small, it may be necessary to select the target center point outside the small GT box. At this time, the distance from the target center point to a certain side of the small GT box may exceed the maximum side length of the small GT box. However, since the GT box itself is very small and the selected target center point is also near the GT box, the distance from the target center point to each side of the small GT box is relatively small and usually less than the above target side length. Therefore, the above target side length can be determined as the upper limit of the regression range. In addition, in order to further reduce the regression range, a value less than the target side length can also be taken as the upper limit of the regression range. For example, 95% of the target side length can be selected as the upper limit of the regression range. In this way, the prediction process of all GT boxes in the training dataset can also be realized. Correspondingly, please refer to Figure 4 , Figure 4 The four arrows in represent the regression vectors from the target center point to the four boundaries of the first GT box respectively. Since the upward arrow and the leftward arrow need to be reversed to point to the upper boundary and the left boundary of the first GT box respectively, the values of the regression vectors corresponding to the upper boundary and the left boundary of the first GT box need to be negative. Therefore, the lower limit of the above regression range needs to be negative. And when there is a situation as shown in Figure 4 The size of the first GT box itself is small, and the length of the corresponding regression vector is also relatively small. Therefore, a negative value with a relatively small absolute value can be taken as the lower limit of the regression range to achieve regression for smaller targets. For example, in an embodiment of the present disclosure, the value range of the minimum value of the regression range can be between -3 and -1. In an embodiment of the present disclosure, the regression range is [-2, reg_max]. Wherein, reg_max is 95% of the target side length.

[0058] It can be understood that the regression ranges corresponding to different preset training sets may be different. Before training the initial network model, the preset training set can be obtained first, and then the regression range is determined by the preset training set, and then the regression range parameter of the initial network model is adjusted to the determined regression range.

[0059] In this embodiment, the regression range is determined based on the side lengths of the GT boxes in each training data in the preset training set. In this way, it can be ensured that the determined regression range is more matched with the current preset training data set, so as to optimize the regression range, which is conducive to improving the accuracy of the prediction result.

[0060] Optionally, the regression range includes a plurality of regression distances. The initial network model predicts the target regression vector corresponding to the target boundary in the GT box, including: predicting a plurality of probability values corresponding one by one to the plurality of regression distances, and calculating the length of the target regression vector based on the plurality of probability values and the plurality of regression distances;

[0061] Wherein, the target boundary is any boundary in the GT box, and the probability value is used to represent the probability that the length of the target regression vector is the regression distance corresponding to the probability value.

[0062] Wherein, the above-mentioned regression range including a plurality of regression distances may mean: determining a plurality of regression distances within the regression range. Specifically, each integer value in the regression range can be determined as a regression distance. Then, when the initial network model predicts the regression vector from the target center point to the target boundary, it can respectively output the probability that the distance between the target center point and the target boundary is at each of the above regression distances, and then calculate the sum of probabilities to obtain the target regression vector.

[0063] For example, when the regression range is [-1, 3], this regression range includes 5 integer points in total. Therefore, 5 regression distances -1, 0, 1, 2, and 3 can be determined respectively. Among them, the negative sign represents the vector direction, and the range of the value is mm. Assuming that the initial network model makes a prediction and the probability values corresponding to these 5 regression distances are: 0, 0, 0, 0.1, and 0.9 respectively, then the target regression vector is: (-1×0)+(0×0)(1×0)+(2×0.1)+(3×0.9) = 2.9, that is, the target regression vector is 2.9 mm in the positive direction.

[0064] In this embodiment, by respectively predicting a plurality of probability values corresponding one by one to the plurality of regression distances and calculating the sum of probabilities of the plurality of probability values and the plurality of regression distances to obtain the length of the target regression vector, in this way, it is conducive to improving the accuracy of the prediction result.

[0065] Optionally, the initial network model is used to generate feature maps of at least two different sizes based on the sample image, and the feature maps are obtained after reducing the sample image according to a preset ratio.

[0066] The feature maps, and the two feature maps of different sizes correspond one-to-one to at least two different preset ratios; determining the regression range based on the side lengths of the GT boxes in each training data in the preset training set includes:

[0067] Reducing the target side length according to the at least two different preset ratios to obtain at least two candidate length values, and the at least two candidate length values correspond one-to-one to the at least two different preset ratios.

[0068] Same preset ratios;

[0069] Determine the length of the regression vector based on the maximum candidate length among the at least two candidate length values.

[0070] Specifically, during the process of the initial network model predicting the sample image, it is usually necessary to downsample the sample image to obtain the corresponding feature map. And the initial network model may include at least two sampling layers with different strides, and the at least two sampling layers correspond one-to-one to the at least two feature maps of different sizes, that is, the sampling layer is used to sample the sample image.

[0071] To obtain the corresponding feature map. Figure 1 One-to-one correspondence, that is, the sampling layer is used to sample the sample image.

[0072] To obtain the corresponding feature map.

[0073] Since during the downsampling process, the feature map will be reduced relative to the sample image. Correspondingly, the GT box in the sample image will also be reduced by the same ratio and then presented in the feature map.

[0074] Among them. And the initial network model makes predictions based on the feature map. Therefore, the upper limit of the above regression range also needs to be reduced accordingly. Therefore, the target side length can be reduced according to the at least two different preset ratios to obtain at least two candidate length values, and then, the length of the regression vector is determined based on the maximum candidate length among the at least two candidate length values. Specifically, 95% of the maximum candidate length among the at least two candidate length values can be used as the upper limit of the above regression range.

[0075] In this embodiment, by using the maximum candidate length among the at least two candidate length values.

[0076] 5

[0077] Determine the length of the regression vector, so as to ensure that the determined regression range can cover the regression ranges of each feature map, which is conducive to improving the accuracy of the determined regression range.

[0078] Optionally, the training data further includes the true class label of the target object, and the prediction result further includes the predicted class label of the target object. Training the initial network model based on the predicted boundary information and the true boundary information to obtain a target network model includes:

[0079] Training the initial network model based on the predicted boundary information, the true boundary information, the true class label and the predicted class label to obtain the target network model.

[0080] Specifically, a first loss function can be constructed based on the predicted boundary information and the true boundary information. At the same time, a second loss function is constructed based on the true class label and the predicted class label. Then, the parameters of the initial network model are optimized based on the first loss function and the second loss function respectively to obtain the target network model.

[0081] It can be understood that after training the target network model, only an image including the target object needs to be input into the target network model, and the target network model can identify the position of the target object in the image and can identify the class of the target object. Specifically, the target network model can frame the target object in the image in the form of a rectangular box, mark the class of each target object in the image, and then output.

[0082] In this embodiment, by training the initial network model based on the predicted boundary information, the true boundary information, the true class label and the predicted class label, the target network model is obtained. In this way, both the position of the target object in the image can be detected, and at the same time, the target object in the image can be classified.

[0083] The target network model trained by the model training method provided by the embodiments of the present disclosure can effectively improve the accuracy of small target detection. Specifically, through experimental verification, it can reach 38.29 mAP through matlab testing on the visdrone standard dataset. Compared with PP-YOLOE, the accuracy is improved by 1.16; compared with tph-yolov5, the accuracy is improved by 2.09. On the coco dataset, compared with ppyoloe_plus, mAP|AP50|AP-small increased by 0.1|0.3|1.9 respectively.

[0084] Please refer to

[0085] Please refer toFigure 5 , Figure 5 This is a schematic structural diagram of a model training device 500 provided by an embodiment of the present disclosure. The model training device 500 includes:

[0086] An acquisition module 501, configured to acquire training data, where the training data includes sample images, and each sample image includes at least one labeled GT box, and one GT box is used to frame a target object in the sample image;

[0087] A prediction module 502, configured to input the sample image into an initial network model for prediction to obtain a prediction result, where the prediction result includes predicted boundary information. The initial network model is used to determine multiple target center points in the sample image, and the initial network model is further used to determine, among the multiple target center points, the target center point corresponding to each GT box, and predict the predicted boundary information based on the target center point corresponding to the GT box. The predicted boundary information includes the predicted relative position information between the boundary of each GT box and the corresponding target center point. The target center point corresponding to the GT box includes: a target center point located inside the GT box, or a target center point located outside the GT box;

[0088] A training module 503, configured to train the initial network model based on the predicted boundary information and the true boundary information to obtain a target network model, where the target network model is a model for detecting target objects in images, and the true boundary information includes the true relative position information between the boundary of each GT box and the corresponding target center point.

[0089] Optionally, when at least one target center point is included in the area framed by the first GT box, the initial network model is configured to determine the at least one target center point as the target center point corresponding to the first GT box, and the first GT box is any one of the at least one GT box;

[0090] When the target center point is not included in the area framed by the first GT box, the initial network model is configured to determine a prior box corresponding to the first GT box in the sample image, the first GT box is located inside the prior box, and at least one target center point is included in the prior box, and the initial network model is further configured to determine the at least one target center point in the prior box as the target center point corresponding to the first GT box.

[0091] Optionally, the first GT box is a rectangular box, the prior box is a square box determined with the center point of the first GT box as the center and a target length as the side length, and the four sides of the first GT box are respectively parallel to the four sides of the prior box.

[0092] Optionally, the training data is the training data in a preset training set, the training data set includes at least two different training data, and the apparatus further includes:

[0093] A determination module 504, configured to determine a regression range based on the side lengths of the GT boxes in each training data in the preset training set, where the minimum value in the regression range is less than 0, and the maximum value in the regression range is determined based on the target side length, where the target side length is: the longest side length among all the boundary side lengths of the GT boxes corresponding to the at least two different training data;

[0094] Wherein, the initial network model is used to predict the predicted boundary information within the regression range based on the target center point corresponding to the GT box, the predicted boundary information includes a one-dimensional regression vector corresponding to each boundary in the GT box, the starting point of the regression vector is the target center point, the direction of the regression vector is perpendicular to the corresponding boundary, and the length of the regression vector is within the regression range.

[0095] Optionally, the regression range includes multiple regression distances, and the initial network model predicts the target regression vector corresponding to the target boundary in the GT box, including: predicting multiple probability values corresponding one-to-one to the multiple regression distances, and calculating the length of the target regression vector based on the multiple probability values and the multiple regression distances;

[0096] Wherein, the target boundary is any one of the boundaries in the GT box, and the probability value is used to represent the probability that the length of the target regression vector is the regression distance corresponding to the probability value.

[0097] Optionally, the initial network model includes a module for generating at least two feature maps with different sizes based on the sample image, the feature maps are feature maps obtained by shrinking the sample image according to a preset ratio, and the two feature maps with different sizes correspond one-to-one to at least two different preset ratios; the determination module 504 is specifically configured to shrink the target side length according to the at least two different preset ratios to obtain at least two candidate length values, and the at least two candidate length values correspond one-to-one to the at least two different preset ratios;

[0098] The determination module 504 is specifically further configured to determine the length of the regression vector based on the maximum candidate length among the at least two candidate length values.

[0099] Optionally, the training data further includes the true class label of the target object, and the prediction result further includes the predicted class label of the target object. The training module 503 is specifically configured to train the initial network model based on the predicted boundary information, the true boundary information, the true class label, and the predicted class label to obtain the target network model.

[0100] It should be noted that the model training device 500 provided in this embodiment can implement all the technical solutions of the above model training method embodiment, and thus can at least achieve all the above technical effects, which will not be elaborated here.

[0101] In the technical solution of the present disclosure, the acquisition, storage, and application of the user's personal information involved all comply with the provisions of relevant laws and regulations and do not violate public order and good customs.

[0102] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0103] Figure 7 A schematic block diagram of an exemplary electronic device 700 that can be used to implement the embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as, a laptop computer, a desktop computer, a workbench, a personal digital assistant, a server, a blade server, a mainframe computer, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as, a personal digital processor, a cellular phone, a smart phone, a wearable device, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely exemplary and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0104] As Figure 7 shown, the electronic device 700 includes a computing unit 701, which can execute various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 702 or the computer program loaded from the storage unit 708 into the random access memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the device 700 can also be stored. The computing unit 701, the ROM 702, and the RAM 703 are connected to each other through a bus 704. The input / output (I / O) interface 705 is also connected to the bus 704.

[0105] Multiple components in the electronic device 700 are connected to the I / O interface 705, including: an input unit 706, such as a keyboard, a mouse, etc.; an output unit 707, such as various types of displays, speakers, etc.; a storage unit 708, such as a magnetic disk, an optical disc, etc.; and a communication unit 709, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 709 allows the device 700 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0106] The computing unit 701 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 701 executes the various methods and processes described above, such as the model training method. For example, in some embodiments, the model training method can be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as the storage unit 708. In some embodiments, part or all of the computer program can be loaded and / or installed onto the device 700 via the ROM 702 and / or the communication unit 709. When the computer program is loaded into the RAM 703 and executed by the computing unit 701, one or more steps of the model training method described above are executed. Alternatively, in other embodiments, the computing unit 701 can be configured to execute the model training method in any other suitable way (e.g., by means of firmware).

[0107] The various embodiments of the systems and techniques described above in this document can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a dedicated or general-purpose programmable processor, and can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit the data and instructions to the storage system, the at least one input device, and the at least one output device.

[0108] The program code for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general purpose computer, a special purpose computer, or other programmable data processing device, such that the program codes, when executed by the processor or controller, cause the functions / operations specified in the flowchart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partly on the machine, as a stand-alone software package partly on the machine and partly on a remote machine, or entirely on the remote machine or server.

[0109] In the context of the present disclosure, a machine-readable medium may be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0110] In order to provide interaction with a user, the systems and techniques described herein may be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0111] The systems and techniques described herein can be implemented in a computing system including backend components (e.g., as a data server), or a computing system including middleware components (e.g., an application server), or a computing system including frontend components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described herein), or a computing system including any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected to each other by digital data communication in any form or medium (e.g., a communication network). Examples of communication networks include: local area network (LAN), wide area network (WAN), and the Internet.

[0112] A computer system can include a client and a server. The client and the server are generally far from each other and typically interact through a communication network. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, or a server of a distributed system, or a server incorporating blockchain.

[0113] It should be understood that various forms of the processes shown above can be used, steps can be reordered, added, or deleted. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved. There is no limitation herein.

[0114] The above specific embodiments do not constitute a limitation on the protection scope of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure shall be included within the protection scope of this disclosure.

Claims

1. A model training method, comprising: Obtaining training data, where the training data includes sample images, and each sample image includes at least one labeled GT box, and one GT box is used to enclose a target object in the sample image; Inputting the sample image into an initial network model for prediction to obtain a prediction result, where the prediction result includes prediction boundary information. Among them, the initial network model is used to determine multiple target center points in the sample image, and the initial network model is further used to determine the target center point corresponding to each GT box among the multiple target center points, and predict the prediction boundary information based on the target center point corresponding to the GT box. The prediction boundary information includes the predicted relative position information between the boundary of each GT box and the corresponding target center point. The target center point corresponding to the GT box includes: a target center point located inside the GT box, or a target center point located outside the GT box; Training the initial network model based on the prediction boundary information and the true boundary information to obtain a target network model, where the target network model is a model for detecting target objects in images, and the true boundary information includes the true relative position information between the boundary of each GT box and the corresponding target center point; When the area enclosed by the first GT box includes at least one target center point, the initial network model is used to determine the at least one target center point as the target center point corresponding to the first GT box, where the first GT box is any one of the at least one GT box; When the area enclosed by the first GT box does not include the target center point, the initial network model is used to determine a prior box corresponding to the first GT box in the sample image, the first GT box is located inside the prior box, and the prior box includes at least one target center point, and the initial network model is further used to determine the at least one target center point inside the prior box as the target center point corresponding to the first GT box.

2. The method according to claim 1, wherein, The first GT box is a rectangular box, and the prior box is a square box determined with the center point of the first GT box as the center and the target length as the side length, and the four sides of the first GT box are respectively parallel to the four sides of the prior box.

3. The method according to claim 1, wherein The training data is the training data in a preset training set, and the preset training set includes at least two different training data. Before inputting the sample image into the initial network model for prediction to obtain a prediction result, the method further includes: Determining a regression range based on the side lengths of the GT boxes in each training data in the preset training set, where the minimum value in the regression range is less than 0, and the maximum value in the regression range is determined based on the target side length, where the target side length is: the longest boundary side length among the boundary side lengths of all GT boxes corresponding to the at least two different training data; Among them, the initial network model is used to predict the predicted boundary information within the regression range based on the target center point corresponding to the GT box. The predicted boundary information includes one-dimensional regression vectors corresponding to each boundary in the GT box. The starting point of the regression vector is the target center point, the direction of the regression vector is perpendicular to the corresponding boundary, and the length of the regression vector is within the regression range.

4. The method according to claim 3, wherein, The regression range includes multiple regression distances. The initial network model predicts the target regression vector corresponding to the target boundary in the GT box, including: predicting multiple probability values corresponding one-to-one to the multiple regression distances, and calculating the length of the target regression vector based on the multiple probability values and the multiple regression distances; Among them, the target boundary is any one of the boundaries in the GT box, and the probability value is used to represent the probability that the length of the target regression vector is the regression distance corresponding to the probability value.

5. The method according to claim 3, wherein, The initial network model is used to generate at least two feature maps with different sizes based on the sample image. The feature map is a feature map obtained by reducing the sample image according to a preset ratio, and the two feature maps with different sizes correspond one-to-one to at least two different preset ratios; determining the regression range based on the side lengths of the GT boxes in each training data in the preset training set includes: Reducing the target side length according to the at least two different preset ratios to obtain at least two candidate length values, and the at least two candidate length values correspond one-to-one to the at least two different preset ratios; Determining the length of the regression vector based on the maximum candidate length among the at least two candidate length values.

6. The method according to claim 1, wherein The training data further includes the true class label of the target object, and the prediction result further includes the predicted class label of the target object. Training the initial network model based on the predicted boundary information and the true boundary information to obtain the target network model includes: Training the initial network model based on the predicted boundary information, the true boundary information, the true class label and the predicted class label to obtain the target network model.

7. A model training device, including: An acquisition module, configured to acquire training data, where the training data includes a sample image, and the sample image includes at least one labeled GT box, and one GT box is used to frame a target object in the sample image; A prediction module, configured to input the sample image into an initial network model for prediction to obtain a prediction result, where the prediction result includes prediction boundary information. The initial network model is used to determine a plurality of target center points in the sample image, and the initial network model is further used to determine, among the plurality of target center points, the target center point corresponding to each GT box, and predict the prediction boundary information based on the target center point corresponding to the GT box. The prediction boundary information includes the predicted relative position information between the boundary of each GT box and the corresponding target center point. The target center point corresponding to the GT box includes: a target center point located inside the GT box, or a target center point located outside the GT box; A training module, configured to train the initial network model based on the prediction boundary information and the ground truth boundary information to obtain a target network model. The target network model is a model for detecting target objects in an image. The ground truth boundary information includes the true relative position information between the boundary of each GT box and the corresponding target center point; When the area enclosed by the first GT box includes at least one target center point, the initial network model is used to determine the at least one target center point as the target center point corresponding to the first GT box, where the first GT box is any one of the at least one GT box; When the area enclosed by the first GT box does not include the target center point, the initial network model is used to determine a prior box corresponding to the first GT box in the sample image. The first GT box is located inside the prior box, and the prior box includes at least one target center point. The initial network model is further used to determine the at least one target center point inside the prior box as the target center point corresponding to the first GT box.

8. The apparatus according to claim 7, wherein The first GT box is a rectangular box, and the prior box is a square box centered at the center point of the first GT box and with a target length as the side length. The four sides of the first GT box are respectively parallel to the four sides of the prior box.

9. The apparatus according to claim 7, wherein, The training data is the training data in a preset training set. The preset training set includes at least two different training data. The apparatus further includes: A determination module, configured to determine a regression range based on the side lengths of the GT boxes in each training data in the preset training set. The minimum value in the regression range is less than 0, and the maximum value in the regression range is determined based on a target side length, where the target side length is the longest side length among the boundary side lengths of all GT boxes corresponding to the at least two different training data; Wherein, the initial network model is used to predict the prediction boundary information within the regression range based on the target center point corresponding to the GT box. The prediction boundary information includes a one-dimensional regression vector corresponding to each boundary in the GT box. The starting point of the regression vector is the target center point, the direction of the regression vector is perpendicular to the corresponding boundary, and the length of the regression vector is within the regression range.

10. The apparatus according to claim 9, wherein, The regression range includes multiple regression distances. The initial network model predicts the target regression vector corresponding to the target boundary in the GT box, including: predicting multiple probability values corresponding one by one to the multiple regression distances, and calculating the length of the target regression vector based on the multiple probability values and the multiple regression distances; wherein, the target boundary is any one of the boundaries in the GT box, and the probability value is used to represent the probability that the length of the target regression vector is the regression distance corresponding to the probability value.

11. The device according to claim 9, wherein The initial network model includes a module for generating at least two feature maps with different sizes based on the sample image. The feature map is a feature map obtained by shrinking the sample image according to a preset ratio, and the at least two feature maps with different sizes correspond one by one to at least two different preset ratios; the determining module is specifically configured to shrink the target side length according to the at least two different preset ratios to obtain at least two candidate length values, and the at least two candidate length values correspond one by one to the at least two different preset ratios; The determining module is specifically further configured to determine the length of the regression vector based on the maximum candidate length among the at least two candidate length values.

12. The device according to claim 7, wherein, The training data further includes the true class label of the target object, and the prediction result further includes the predicted class label of the target object. The training module is specifically configured to train the initial network model based on the predicted boundary information, the true boundary information, the true class label, and the predicted class label to obtain the target network model.

13. An electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein, the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the model training method according to any one of claims 1-6.

14. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to execute the model training method according to any one of claims 1-6.

15. A computer program product, comprising a computer program which, when executed by a processor, implements the model training method according to any one of claims 1-6.

Images