Target detection model training and target detection method, device and equipment

Abstract

The present disclosure provides a target detection model training and target detection method, device and equipment, relating to the technical field of artificial intelligence, and particularly relating to the technical field of intelligent transportation, computer vision, deep learning, large model and the like. The target detection model training method comprises: adopting a teacher model to process an image sample containing a target object to obtain a first image feature; the teacher model is obtained in advance by a UFO training manner; adopting a student model to process the image sample to obtain a second image feature; based on the first image feature and the second image feature, a total loss function is obtained; based on the total loss function, the model parameters of the student model are adjusted until a preset end condition is reached, and the student model when the end condition is reached is taken as a target detection model, and the target detection model is used to obtain structured information of the target object.

Description

Object detection model training and object detection methods, devices and equipment Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, specifically to the fields of intelligent transportation, computer vision, deep learning, and large models, and particularly to a target detection model training and target detection method, apparatus, and equipment. Background Technology

[0002] In roadside perception systems, target object structuring is a crucial function. In practical applications, large models are typically used for target object structuring to improve detection performance. However, these models are large, consume significant resources, and have long inference times. Summary of the Invention

[0003] This disclosure provides a target detection model training method, apparatus, and device for target detection.

[0004] According to one aspect of this disclosure, a method for training an object detection model is provided, comprising: using a teacher model to process image samples containing a target object to obtain a first image feature; the teacher model is pre-trained using a UFO method; using a student model to process the image samples to obtain a second image feature; obtaining a total loss function based on the first image feature and the second image feature; adjusting the model parameters of the student model based on the total loss function until a preset termination condition is reached, and using the student model that reaches the termination condition as the object detection model, wherein the object detection model is used to obtain the structured information of the target object.

[0005] According to another aspect of this disclosure, a target detection method is provided, comprising: acquiring a target image, the target image containing a target object; processing the target image using a pre-trained target detection model to obtain structured information of the target object; wherein the target detection model is trained using the method described in any of the preceding aspects.

[0006] According to another aspect of this disclosure, a target detection model training apparatus is provided, comprising: a first feature extraction module, used to process image samples containing target objects using a teacher model to obtain first image features; the teacher model is obtained in advance through UFO training; a second feature extraction module, used to process the image samples using a student model to obtain second image features; an acquisition module, used to acquire a total loss function based on the first image features and the second image features; and an adjustment module, used to adjust the model parameters of the student model based on the total loss function until a preset termination condition is reached, and the student model that reaches the termination condition is used as the target detection model, the target detection model being used to acquire the structured information of the target object.

[0007] According to another aspect of this disclosure, a target detection apparatus is provided, comprising: an acquisition module for acquiring a target image, the target image containing a target object; and a detection module for processing the target image using a pre-trained target detection model to obtain structured information of the target object; wherein the target detection model is trained using the method described in any of the preceding aspects.

[0008] According to another aspect of this disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to said at least one processor; wherein the memory stores instructions executable by said at least one processor, said instructions being executed by said at least one processor to enable said at least one processor to perform the method as described in any of the foregoing aspects.

[0009] According to another aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are configured to cause the computer to perform the method according to any of the preceding aspects.

[0010] According to another aspect of this disclosure, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the method according to any of the preceding aspects.

[0011] According to the technical solution disclosed herein, detection speed and accuracy can be improved with lower resource consumption.

[0012] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0013] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0014] Figure 1 is a schematic diagram according to a first embodiment of the present disclosure;

[0015] Figure 2 is a schematic diagram of an application scenario for implementing the target detection method of the present disclosure embodiment;

[0016] Figure 3 is a schematic diagram of a target detection model training process provided according to an embodiment of the present disclosure;

[0017] Figure 4 is a schematic diagram of another target detection model training process provided according to an embodiment of the present disclosure;

[0018] Figure 5 is a schematic diagram according to a second embodiment of the present disclosure;

[0019] Figure 6 is a schematic diagram according to a third embodiment of the present disclosure;

[0020] Figure 7 is a schematic diagram according to the fourth embodiment of the present disclosure;

[0021] Figure 8 is a schematic diagram according to the fifth embodiment of the present disclosure;

[0022] Figure 9 is a schematic diagram according to the sixth embodiment of the present disclosure;

[0023] Figure 10 is a schematic diagram of an electronic device used to implement the target detection model training method or target detection method of the present disclosure embodiments. Detailed Implementation

[0024] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0025] Target object structuring refers to obtaining the structured information of a target object. Taking a car as an example, the structured information of a car includes, for example, its model, color, and brand.

[0026] In related technologies, large models (such as the ResNet50 model) are typically used for object structuring. However, due to the large scale of these models, they consume a lot of hardware resources and have long inference times. Residual Networks (ResNet) are a type of deep convolutional neural network, and the ResNet50 model is a 50-layer ResNet model.

[0027] To save resources and improve detection speed, this disclosure provides the following embodiments.

[0028] Figure 1 is a schematic diagram according to a first embodiment of the present disclosure. This embodiment provides a method for training an object detection model, the method including:

[0029] 101. A teacher model is used to process image samples containing the target object to obtain first image features; the teacher model is obtained in advance through UFO training.

[0030] 102. Using a student model, the image samples are processed to obtain second image features.

[0031] 103. Based on the first image features and the second image features, obtain the total loss function.

[0032] 104. Based on the total loss function, adjust the model parameters of the student model until a preset termination condition is reached, and use the student model that reaches the termination condition as the target detection model. The target detection model is used to obtain the structured information of the target object.

[0033] Image samples can be obtained from existing datasets that contain the target object.

[0034] The target object refers to the object to be detected, such as motor vehicles or non-motor vehicles.

[0035] The teacher model is larger than the student model. For example, the teacher model is a ResNet50 model, while the student model is a ResNet18 model. The ResNet50 model and the ResNet18 model are 50-layer and 18-layer ResNet models, respectively. The ResNet model is a deep convolutional neural network model.

[0036] Unified Feature Optimization (UFO) is a training method that aims to benefit each task by pre-training all sub-tasks in a large model for multi-task learning. In roadside perception scenarios, it can perform multi-task detection of motor vehicles, non-motor vehicles, and pedestrians, and identify the corresponding structured information. The image samples corresponding to these multi-tasks can come from various sources such as bullet cameras and fisheye cameras.

[0037] Currently, UFO is based on the Vision Transformer (ViT) architecture. First, it trains a unified model (called supernet) on various depth representation tasks in a multi-task learning manner. Then, UFO learns to build a compressed supernet to obtain a subnet. The subnet is a path in the supernet and is used for some sub-tasks.

[0038] In this specific embodiment, after obtaining a unified model through UFO, a subnet suitable for the target object (such as a motor vehicle) is obtained through subnet search, and this subnet is used as the teacher model.

[0039] To distinguish them, the image features extracted by the teacher model are called the first image features, and the image features extracted by the student model are called the second image features. Then, the total loss function is obtained based on the first image features and the second image features.

[0040] Regarding the total loss function: a knowledge distillation loss function can be constructed based on the first and second image features; the predicted information output by the student model is obtained based on the second image features, and the student model loss function is constructed based on the predicted information and its ground truth value; then, the total loss function is constructed based on the knowledge distillation loss function and the student model loss function. Alternatively,

[0041] First, the first structured information of the target object output by the teacher model is obtained based on the first image features. Second, the second structured information of the target object output by the student model is obtained based on the second image features. A knowledge distillation loss function is constructed based on the first and second structured information. Predicted information output by the student model is obtained based on the second image features. A student model loss function is constructed based on the predicted information and its ground truth value. Finally, a total loss function is constructed based on the knowledge distillation loss function and the student model loss function.

[0042] After obtaining the total loss function, the model parameters of the student model are adjusted using the total loss function until a preset termination condition is reached. The termination condition may include, for example, reaching a preset number of iterations or model convergence. The student model that reaches the termination condition is used as the final object detection model, which is used to obtain the structured information of the target object.

[0043] Structured information is information used to describe a target object, such as the target object's attribute information. Taking a motor vehicle as an example, its structured information might include: vehicle model, color, brand, etc.

[0044] In this embodiment, the student model is trained using knowledge distillation, and the resulting student model is used as the object detection model. Because the student model is small in scale and has learned from the teacher model, it achieves high detection accuracy with a relatively small scale, thus improving detection speed and accuracy with lower resource overhead. Furthermore, the teacher model is trained using a UFO (Underlying First-Order) method, which, compared to single-task training, improves the accuracy of the teacher model, thereby enhancing the accuracy of the object detection model and the accuracy of acquiring structured information about the target object.

[0045] To better understand the embodiments of this disclosure, the application scenarios to which the embodiments of this disclosure may be applied are described.

[0046] Figure 2 is a schematic diagram of an application scenario for implementing the target detection method of the present disclosure.

[0047] As shown in Figure 2, an application (APP) can be installed on the user terminal 201. This APP interacts with the server, which is deployed on server 202. The server can be a local server or a cloud server, and can be a single server or a server cluster. The user terminal and the server can be connected via a wired network and / or a wireless network.

[0048] Regarding the detection process: Users can send a target image containing the target object to the server via the app. The server inputs the target image into the target detection model and outputs the target detection result, which specifically includes the structured information of the target object. The server can then feed this target detection result back to the app, which displays it to the user. This embodiment uses server-side processing as an example; it can be understood that if the user terminal has the corresponding capabilities, the relevant process can also be executed locally on the user terminal.

[0049] The object detection model described above is pre-trained. The training process of the object detection model can be found in the following example.

[0050] Figure 3 is a schematic diagram of the training process of an object detection model according to an embodiment of the present disclosure. Figure 4 is a schematic diagram of the training process of another object detection model according to an embodiment of the present disclosure. In this embodiment, the object detection model is trained using a knowledge distillation method.

[0051] The training process includes a teacher model and a student model. The teacher model is a pre-trained large model, while the student model is a small model to be trained. Specifically, the teacher model can be a ResNet50 model, and the student model can be a ResNet18 model. During training, the model parameters of the student model are adjusted using the total loss function until a preset termination condition is reached, such as reaching a preset number of iterations. The student model that reaches the termination condition is then used as the object detection model, which is used to obtain the structured information of the target object.

[0052] In the knowledge distillation process, as shown in Figure 3, a knowledge distillation loss function can be constructed based on image features; or, as shown in Figure 4, a knowledge distillation loss function can be constructed based on structured information.

[0053] Both the teacher model and the student model can include an image feature extraction network and an output network, respectively. The image feature extraction network of the teacher model is represented by ViT UFO, and the image feature extraction network of the student model is represented by the backbone network of Convolutional Neural Networks (CNN).

[0054] As shown in Figure 3, for the same image sample, the image sample is input into the image feature extraction network (ViT UFO) of the teacher model and the image feature extraction network (CNN backbone network) of the student model, and the outputs are the first image feature and the second image feature, respectively.

[0055] Then, a knowledge distillation loss function is constructed based on the first image features and the second image features, and is represented by the first loss function.

[0056] In this embodiment, a first loss function is constructed based on the first image features and the second image features, which can achieve feature-level knowledge distillation and improve the model's versatility.

[0057] Furthermore, the first image features can be normalized to obtain a first normalized feature; the second image features can be normalized to obtain a second normalized feature; and the first loss function can be constructed based on the first normalized feature and the second normalized feature.

[0058] Normalization refers to mapping the first image features and the second image features to a specific interval. In this embodiment, the specific formula for normalization can be:

[0059]

[0060] in, F is the normalized feature, μ is the original feature, and σ is the mean and standard deviation of the original feature, respectively.

[0061] After normalization, amplitude information can be removed, allowing the student model to learn feature similarity from the teacher model rather than amplitude information, thereby improving model performance.

[0062] The features obtained by normalizing the first image features are called the first normalized features, and the features obtained by normalizing the second image features are called the second normalized features. Then, a first loss function is constructed based on the first and second normalized features. The first loss function is a function representing distance, such as the smooth L1 loss function. The smooth L1 loss function is a loss function that combines the advantages of the L1 and L2 loss functions. The L1 loss function is the absolute error function, and the L2 loss function is the least squares error function.

[0063] In this embodiment, by normalizing the first image features and the second image features, two sets of normalized features are obtained. The first loss function is calculated based on the two sets of normalized features, which can improve the accuracy of the first loss function and thus improve the accuracy of the model and target detection.

[0064] Furthermore, both the first image feature and the second image feature are multi-scale image features; for each scale, a sub-loss function is constructed based on the first image feature and the second image feature at each scale; and the first loss function is constructed based on the sub-loss function at each scale.

[0065] For example, if both the first and second image features include image features at three scales, then a first sub-loss function can be constructed for the first scale, a second sub-loss function for the second scale, and a third sub-loss function for the third scale. These three sub-loss functions can then be added together to obtain the first loss function. Taking the first scale as an example, the first sub-loss function can be constructed based on the first and second image features at the first scale. For instance, these two types of image features at the first scale can be normalized, and the smooth L1 value can be calculated using the normalized features as the first sub-loss function.

[0066] In this embodiment, a knowledge distillation loss function is constructed based on multi-scale image features, which can consider feature information at multiple levels and improve model performance.

[0067] The second image feature is also input into the output network of the student model to output predicted information. Specifically, the output network includes a classification network and a regression network. The classification network is used to obtain the structured information of the target object, and the regression network is used to obtain the location information of the target object. Therefore, the predicted information can specifically include predicted structured information and predicted location information.

[0068] Next, a student model loss function, or second loss function, is constructed based on the predicted information and its corresponding ground truth. For example, a classification loss function is constructed based on the predicted structured information and the true structured information of the target object in the image sample, and a regression loss function is constructed based on the predicted location information and the true location information of the target object. The second loss function is obtained by adding the classification loss function and the regression loss function.

[0069] After obtaining the first loss function and the second loss function, a total loss function is constructed based on the first loss function and the second loss function. For example, the total loss function is obtained by weighted summation of the first loss function and the second loss function. The first weight corresponding to the first loss function and the second weight corresponding to the second loss function are set values, and the sum of the first weight and the second weight is 1.

[0070] After obtaining the total loss function, the model parameters of the student model are adjusted using the total loss function, specifically including adjusting the model parameters of the CNN backbone network and the output network.

[0071] The above example of constructing a knowledge distillation loss function based on image features can also be used to construct a knowledge distillation loss function based on the structured information output by the model.

[0072] As shown in Figure 4, for the same image sample, the image sample is input into the image feature extraction network (ViT UFO) of the teacher model and the image feature extraction network (CNN backbone network) of the student model, and the outputs are the first image feature and the second image feature, respectively.

[0073] The teacher model and student model also include an output network. The output network of the teacher model includes a classification network, which is represented by a first fully connected layer. The first image feature is input into the first fully connected layer, and the output is the first structured information.

[0074] The output network of the student model includes a classification network and a regression network. The classification network of the student network is represented by a second fully connected layer, and the regression network is represented by a centernet. The second image features are input into the second fully connected layer and the centernet, respectively, and the outputs are the second structured information and the location information.

[0075] Structured information is the attribute information of a target object. For example, if the target object is a motor vehicle, the structured information may include: vehicle model, color, brand, etc.

[0076] Next, a knowledge distillation loss function is constructed based on the first and second structured information, and represented by a third loss function. Specifically, the third loss function can be the L1 loss function.

[0077] In this embodiment, a knowledge distillation loss function is constructed based on the first and second structured information, which can achieve knowledge distillation at the output level without needing to focus on the internal structure of the model, thus achieving a simpler and more efficient result.

[0078] In addition, a student model loss function is constructed based on the prediction information output by the student model. For example, as shown in Figure 4, the prediction information includes prediction structured information (i.e., second structured information) and prediction location information. Then, the student model loss function is constructed using the prediction information and its corresponding ground truth, which is represented by the fourth loss function.

[0079] For example, a classification loss function is constructed based on the predicted structured information and the real structured information of the target object in the image sample. A regression loss function is constructed based on the predicted location information and the real location information of the target object. The fourth loss function is obtained by adding the classification loss function and the regression loss function.

[0080] After obtaining the third and fourth loss functions, a total loss function is constructed based on the third and fourth loss functions. For example, the total loss function is obtained by weighted summation of the third and fourth loss functions. The first weight corresponding to the third loss function and the second weight corresponding to the fourth loss function are set values, and the sum of the first weight and the second weight is 1.

[0081] After obtaining the total loss function, the model parameters of the student model are adjusted using the total loss function. Specifically, this includes adjusting the model parameters of the CNN backbone network and the model parameters of the output network (the second fully connected layer and the center point network).

[0082] In this embodiment, the network used by the student model to obtain location information is a centernet. Centernet is an anchor-free target detection network, which has advantages in both speed and accuracy, thereby improving the speed and accuracy of target detection.

[0083] In conjunction with the above application scenarios, this disclosure also provides the following embodiments.

[0084] Figure 5 is a schematic diagram according to a second embodiment of the present disclosure. This embodiment provides a method for training an object detection model, taking the construction of a knowledge distillation loss function based on image features as an example. Referring to Figure 3, the method includes:

[0085] 501. A teacher model is used to process image samples containing the target object to obtain first image features; the teacher model is obtained in advance through UFO training.

[0086] For example, referring to Figure 3, a teacher model image feature extraction network, such as ViT UFO, can be used to perform feature extraction processing on the input image samples to obtain the first image features.

[0087] 502. Using a student model, the image samples are processed to obtain second image features.

[0088] For example, referring to Figure 3, a student model image feature extraction network, such as a CNN backbone network, can be used to extract features from the input image samples to obtain the second image features.

[0089] 503. Construct a first loss function based on the first image features and the second image features.

[0090] For example, the first image features and the second image features can be normalized first and second normalized features respectively to obtain the first normalized features and the second normalized features, and then the first loss function can be constructed using the first normalized features and the second normalized features.

[0091] 504. Obtain the prediction information output by the student model based on the second image features, and construct a second loss function based on the prediction information and its corresponding ground truth.

[0092] For example, the predicted information includes classification information (predicted structured information) and regression information (predicted location information). A classification loss function is then constructed based on the predicted structured information and the actual structured information, and a regression loss function is constructed based on the predicted location information and the actual location information. The classification loss function and the regression loss function are added together to obtain the second loss function.

[0093] 505. Based on the first loss function and the second loss function, construct the total loss function.

[0094] For example, the first loss function and the second loss function are weighted and summed based on preset weights to obtain the total loss function.

[0095] 506. Based on the total loss function, adjust the model parameters of the student model until a preset termination condition is reached, and use the student model that reaches the termination condition as the target detection model. The target detection model is used to obtain the structured information of the target object.

[0096] In this embodiment, a knowledge distillation loss function (first loss function) is constructed based on the first image features and the second image features, which can realize feature-level knowledge distillation and improve the generality of the model.

[0097] Figure 6 is a schematic diagram according to a third embodiment of the present disclosure. This embodiment provides a method for training an object detection model, taking the construction of a knowledge distillation loss function based on structured information as an example. Referring to Figure 4, the method includes:

[0098] 601. A teacher model is used to process image samples containing the target object to obtain first image features; the teacher model is obtained in advance through UFO training.

[0099] For example, referring to Figure 4, a teacher model image feature extraction network, such as ViT UFO, can be used to perform feature extraction processing on the input image samples to obtain the first image features.

[0100] 602. Using a student model, the image samples are processed to obtain second image features.

[0101] For example, referring to Figure 4, a student model image feature extraction network, such as a CNN backbone network, can be used to extract features from the input image samples to obtain the second image features.

[0102] 603. Based on the first image features, obtain the first structured information of the target object output by the teacher model.

[0103] For example, referring to Figure 4, the first fully connected network processes the first image features input and outputs the first structured information.

[0104] 604. Based on the second image features, obtain the prediction information output by the student model, wherein the prediction information includes the second structured information of the target object.

[0105] For example, referring to Figure 4, the prediction information includes: second structured information and prediction location information; after the second fully connected network processes the input first image features, it outputs the second structured information, and after the center point network processes the input first image features, it outputs the prediction location information.

[0106] 605. Construct a third loss function based on the first structured information and the second structured information.

[0107] If the third loss function is, for example, the L1 loss function, then the first and second structured information can be used to construct the L1 loss function as the third loss function.

[0108] 606. Construct a fourth loss function based on the predicted information and its corresponding true value.

[0109] For example, a classification loss function is constructed based on the second structured information and the real structured information, and a regression loss function is constructed based on the predicted location information and the real location information. The classification loss function and the regression loss function are added together to obtain the fourth loss function.

[0110] 607. Construct the total loss function based on the third loss function and the fourth loss function.

[0111] For example, the third and fourth loss functions are weighted and summed based on preset weights to obtain the total loss function.

[0112] 608. Based on the total loss function, adjust the model parameters of the student model until a preset termination condition is reached, and use the student model that reaches the termination condition as the target detection model, which is used to obtain the structured information of the target object.

[0113] In this embodiment, a knowledge distillation loss function (third loss function) is constructed based on the first and second structured information, which can realize output-level knowledge distillation and improve processing efficiency.

[0114] Figure 7 is a schematic diagram according to a fourth embodiment of the present disclosure, which provides a target detection method, the method comprising:

[0115] 701. Obtain the target image, wherein the target image contains the target object.

[0116] 702. A pre-trained target detection model is used to process the target image to obtain the structured information of the target object.

[0117] The object detection model was trained using any of the above embodiments.

[0118] For example, if the target object is a motor vehicle, the target detection model can process the information such as the vehicle model, color, and brand.

[0119] In this embodiment, since the target detection model is small in size but high in performance, using this model for target detection can quickly and accurately obtain the structured information of the target object at a low cost.

[0120] Figure 8 is a schematic diagram according to the fifth embodiment of the present disclosure. This embodiment provides a target detection model training device 800, which includes: a first feature extraction module 801, a second feature extraction module 802, an acquisition module 803, and an adjustment module 804.

[0121] The first feature extraction module 801 is used to process image samples containing target objects using a teacher model to obtain first image features; the teacher model is obtained in advance through UFO training. The second feature extraction module 802 is used to process the image samples using a student model to obtain second image features. The acquisition module 803 is used to acquire a total loss function based on the first image features and the second image features. The adjustment module 804 is used to adjust the model parameters of the student model based on the total loss function until a preset termination condition is reached, and the student model that reaches the termination condition is used as the target detection model, which is used to acquire the structured information of the target object.

[0122] In this embodiment, the student model is trained using knowledge distillation, and the resulting student model is used as the object detection model. Because the student model is small in scale and has learned from the teacher model, it achieves high detection accuracy with a relatively small scale, thus improving detection speed and accuracy with lower resource overhead. Furthermore, the teacher model is trained using a UFO (Underlying First-Order) method, which, compared to single-task training, improves the accuracy of the teacher model, thereby enhancing the accuracy of the object detection model and the accuracy of acquiring structured information about the target object.

[0123] In some embodiments, the acquisition module 803 is further configured to:

[0124] A first loss function is constructed based on the first image features and the second image features;

[0125] Based on the second image features, the predicted information output by the student model is obtained, and a second loss function is constructed based on the predicted information and its corresponding ground truth.

[0126] The total loss function is constructed based on the first loss function and the second loss function.

[0127] In this embodiment, a knowledge distillation loss function (first loss function) is constructed based on the first image features and the second image features, which can realize feature-level knowledge distillation and improve the generality of the model.

[0128] In some embodiments, the acquisition module 803 is further configured to:

[0129] The first image features are normalized to obtain the first normalized features;

[0130] The second image feature is normalized to obtain the second normalized feature;

[0131] The first loss function is constructed based on the first normalized feature and the second normalized feature.

[0132] In this embodiment, by normalizing the first image features and the second image features, two sets of normalized features are obtained. The first loss function is calculated based on the two sets of normalized features, which can improve the accuracy of the first loss function and thus improve the accuracy of the model and target detection.

[0133] In some embodiments, both the first image feature and the second image feature are multi-scale image features; the acquisition module 803 is further used for:

[0134] For each scale, a sub-loss function is constructed based on the first and second image features of each scale.

[0135] The first loss function is constructed based on the sub-loss functions at each scale.

[0136] In this embodiment, a knowledge distillation loss function is constructed based on multi-scale image features, which can consider feature information at multiple levels and improve model performance.

[0137] In some embodiments, the acquisition module 803 is further configured to:

[0138] Based on the first image features, the first structured information of the target object output by the teacher model is obtained;

[0139] Based on the second image features, the predicted information output by the student model is obtained, and the predicted information includes the second structured information of the target object;

[0140] A third loss function is constructed based on the first structured information and the second structured information;

[0141] A fourth loss function is constructed based on the predicted information and its corresponding true values;

[0142] The total loss function is constructed based on the third loss function and the fourth loss function.

[0143] In this embodiment, a knowledge distillation loss function (third loss function) is constructed based on the first and second structured information, which can realize output-level knowledge distillation and improve processing efficiency.

[0144] Figure 9 is a schematic diagram according to a sixth embodiment of the present disclosure. This embodiment provides a target detection device 900, which includes an acquisition module 901 and a detection module 902.

[0145] The acquisition module 901 is used to acquire a target image, which contains a target object; the detection module 902 is used to process the target image using a pre-trained target detection model to obtain the structured information of the target object.

[0146] The object detection model was trained using any of the above embodiments.

[0147] For example, if the target object is a motor vehicle, the target detection model can process the information such as the vehicle model, color, and brand.

[0148] In this embodiment, since the target detection model is small in size but high in performance, using this model for target detection can quickly and accurately obtain the structured information of the target object at a low cost.

[0149] It is understood that the same or similar content in different embodiments of this disclosure can be referred to each other.

[0150] It is understood that the terms "first" and "second" in the embodiments of this disclosure are only used for distinction and do not indicate the degree of importance or the order of events.

[0151] The collection, storage, use, processing, transmission, provision, and disclosure of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0152] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0153] Figure 10 illustrates a schematic block diagram of an example electronic device 1000 that can be used to implement embodiments of the present disclosure. The electronic device 1000 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, servers, blade servers, mainframe computers, and other suitable computers. The electronic device 1000 may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0154] As shown in Figure 10, the electronic device 1000 includes a computing unit 1001, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 1002 or a computer program loaded from a storage unit 1008 into a random access memory (RAM) 1003. The RAM 1003 may also store various programs and data required for the operation of the electronic device 1000. The computing unit 1001, ROM 1002, and RAM 1003 are interconnected via a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.

[0155] Multiple components in electronic device 1000 are connected to I / O interface 1005, including: input unit 1006, such as keyboard, mouse, etc.; output unit 1007, such as various types of displays, speakers, etc.; storage unit 1008, such as disk, optical disk, etc.; and communication unit 1009, such as network card, modem, wireless transceiver, etc. Communication unit 1009 allows electronic device 1000 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0156] The computing unit 1001 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 1001 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 1001 performs the various methods and processes described above, such as object detection model training methods or object detection methods. For example, in some embodiments, the object detection model training method or object detection method can be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 1008. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 1000 via ROM 1002 and / or communication unit 1009. When the computer program is loaded into RAM 1003 and executed by the computing unit 1001, one or more steps of the object detection model training method or object detection method described above can be performed. Alternatively, in other embodiments, the computing unit 1001 may be configured to perform an object detection model training method or an object detection method by any other suitable means (e.g., by means of firmware).

[0157] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0158] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0159] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0160] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0161] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0162] Computer systems can include clients and servers. Clients and servers are generally geographically separated and typically interact via communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. A server can be a cloud server, also known as a cloud computing server or cloud host, a hosting product within the cloud computing service ecosystem, addressing the shortcomings of traditional physical hosts and VPS (Virtual Private Server, or simply "VPS") services, such as high management difficulty and weak business scalability. Servers can also be servers for distributed systems or servers incorporating blockchain technology.

[0163] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0164] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A method for training an object detection model, comprising: A teacher model is used to process image samples containing the target object to obtain the first image features; The teacher model is obtained in advance through UFO training; the teacher model is a subnet in the unified model that is applicable to the target object; the unified model is obtained by training with image samples corresponding to multiple tasks from multiple sources. The image samples are processed using a student model to obtain second image features; Based on the first image features and the second image features, the total loss function is obtained, including: constructing a first loss function based on the first image features and the second image features; The predicted information output by the student model is obtained based on the second image features, and a second loss function is constructed based on the predicted information and its corresponding ground truth. The total loss function is constructed based on the first and second loss functions. Both the first and second image features are multi-scale image features. The first loss function is constructed based on sub-loss functions at each scale, and each sub-loss function at each scale is constructed based on the first and second image features at each scale. Based on the total loss function, the model parameters of the student model are adjusted until a preset termination condition is reached. The student model that reaches the termination condition is used as the target detection model, which is used to obtain the structured information of the target object. The output network of the teacher model includes a classification network, and the output network of the student model includes a classification network and a regression network. The classification network is used to output structured information, and the regression network is used to output location information. The total loss function is constructed based on the predicted information output by the student model, and the predicted information includes predicted structured information and predicted location information.

2. The method according to claim 1, wherein, The step of constructing a first loss function based on the first image features and the second image features includes: normalizing the first image features to obtain a first normalized feature; normalizing the second image features to obtain a second normalized feature; and constructing the first loss function based on the first normalized feature and the second normalized feature.

3. The method according to claim 1, wherein, The step of obtaining the total loss function based on the first image features and the second image features includes: obtaining the first structured information of the target object output by the teacher model based on the first image features; obtaining the prediction information output by the student model based on the second image features, wherein the prediction information includes the second structured information of the target object; constructing a third loss function based on the first structured information and the second structured information; constructing a fourth loss function based on the prediction information and its corresponding ground truth; and constructing the total loss function based on the third loss function and the fourth loss function.

4. A target detection method, comprising: Acquire a target image, wherein the target image contains a target object; The target image is processed using a pre-trained target detection model to obtain the structured information of the target object; wherein the target detection model is trained using the method described in any one of claims 1-3.

5. A target detection model training device, comprising: The first feature extraction module is used to process image samples containing target objects using a teacher model to obtain first image features; The teacher model is obtained in advance through UFO training; the teacher model is a subnet in the unified model that is applicable to the target object; the unified model is obtained by training with image samples corresponding to multiple tasks from multiple sources. The second feature extraction module is used to process the image samples using a student model to obtain second image features; An acquisition module is used to acquire a total loss function based on the first image features and the second image features, including: constructing a first loss function based on the first image features and the second image features; acquiring the prediction information output by the student model based on the second image features, and constructing a second loss function based on the prediction information and its corresponding ground truth; constructing the total loss function based on the first loss function and the second loss function; the first image features and the second image features are both multi-scale image features, the first loss function is constructed based on sub-loss functions at each scale, and the sub-loss functions at each scale are constructed based on the first image features and the second image features at each scale; an adjustment module is used to adjust the model parameters of the student model based on the total loss function until a preset termination condition is reached, and the student model that reaches the termination condition is used as the target detection model, the target detection model is used to acquire the structured information of the target object; the output network of the teacher model includes a classification network, the output network of the student model includes a classification network and a regression network, the classification network is used to output structured information, and the regression network is used to output location information; the total loss function is constructed based on the prediction information output by the student model, the prediction information includes predicted structured information and predicted location information.

6. The apparatus according to claim 5, wherein, The acquisition module is further configured to: normalize the first image features to obtain the first normalized features; The second image feature is normalized to obtain the second normalized feature; The first loss function is constructed based on the first normalized feature and the second normalized feature.

7. The apparatus according to claim 5, wherein, The acquisition module is further configured to: acquire first structured information of the target object output by the teacher model based on the first image features; acquire prediction information output by the student model based on the second image features, the prediction information including second structured information of the target object; construct a third loss function based on the first structured information and the second structured information; construct a fourth loss function based on the prediction information and its corresponding ground truth; and construct the total loss function based on the third loss function and the fourth loss function.

8. A target detection device, comprising: The acquisition module is used to acquire a target image, wherein the target image contains a target object; A detection module is used to process the target image using a pre-trained target detection model to obtain structured information of the target object; wherein the target detection model is trained using the method described in any one of claims 1-3.

9. An electronic device, comprising: At least one processor; And a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-4.

10. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-4.

11. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-4.

Images