A training method, device and equipment of a multitask target detection model
Patent Information
- Application Number
- CN202510471204.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-15
- Publication Date
- 2026-10-23
AI Technical Summary
针对不同的障碍物类别分别开发专属的目标检测模型，可以满足下游对于视觉感知的精度要求，但是车载自动驾驶芯片的计算能力是有限的，会造成车载芯片的算力不足
[0044]本申请实施例获取多个目标检测任务的待训练图像以及标签，将其处理为批训练数据。构建多任务目标检测模型，多任务目标检测模型包括骨干网络、颈部网络以及多个头部网络，每个头部网络对应于一个目标检测任务。将批训练数据输入多任务目标检测模型，得到多个头部网络输出的检测结果，以生成每个目标检测任务对应的损失值。从而根据损失值对多任务目标检测模型中的骨干网络、颈部网络以及多个头部网络进行训练。利用一个多任务目标检测模型，通过不同的头部网络实现多个目标检测任务的分解。将不同目标检测任务集成到了多任务目标检测模型框架中，保证了对不同的障碍物类别的识别精度，且节省算力，提高了目标检测模型的开发及迭代效率。
Smart Images
Figure CN122967231A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of neural network technology, specifically to a training method, apparatus, and device for a multi-task object detection model. Background Technology
[0002] In the field of autonomous driving, object detection algorithms are a key core technology. Obstacles in the real environment vary in size and type. To ensure safety, vehicles need to be able to accurately and promptly perceive dynamic or static obstacles in their surroundings while driving or parking.
[0003] In existing technologies, obstacles can be detected using object detection models. As autonomous driving systems become increasingly complex, the accuracy requirements for object detection models will also increase, and the types of obstacles these models need to monitor will continue to expand. Developing dedicated object detection models for different obstacle types can meet the downstream requirements for visual perception accuracy; however, the computing power of onboard autonomous driving chips is limited, leading to insufficient computational resources. Summary of the Invention
[0004] In view of this, embodiments of this application provide a training method, apparatus, and device for a multi-task target detection model, which achieves a target detection model with the highest possible accuracy under limited computing power constraints.
[0005] To address the above problems, the technical solutions provided in this application are as follows:
[0006] In a first aspect, embodiments of this application provide a training method for a multi-task object detection model, the method comprising:
[0007] Acquire training images for multiple object detection tasks and the labels of the training images;
[0008] The images to be trained and their labels are processed into batch training data.
[0009] The batch training data is input into a multi-task target detection model, which includes a backbone network, a neck network, and multiple head networks. The backbone network is connected to the neck network, and the neck network is connected to multiple head networks. Each head network corresponds to one target detection task.
[0010] Based on the detection results output by the multiple head networks, a loss value corresponding to each target detection task is generated, and the backbone network, the neck network, and the multiple head networks are trained based on the loss value.
[0011] Repeatedly input the batch training data into the multi-task object detection model and subsequent steps until the preset conditions are met.
[0012] In one possible implementation, processing the image to be trained and its labels into batch training data includes:
[0013] The training images and their labels for each object detection task are processed into batch training data corresponding to each object detection task.
[0014] In one possible implementation, processing the image to be trained and its labels into batch training data includes:
[0015] Based on the labels of the images to be trained, a corresponding batch label is added to each image to be trained. The batch label corresponding to each image to be trained includes the labels of each of the target detection tasks arranged in order. The labels of each target detection task include the label corresponding to its own target detection task and the labels corresponding to other target detection tasks. The labels corresponding to the other target detection tasks are zero.
[0016] The training images for each of the target detection tasks, along with the corresponding batch labels for each training image, are combined into batch training data.
[0017] In one possible implementation, generating a loss value corresponding to each target detection task based on the detection results output by the multiple head networks, and training the backbone network, the neck network, and the multiple head networks based on the loss values, includes:
[0018] Based on the detection results output by the multiple head networks, a loss value corresponding to each target detection task is generated;
[0019] Backpropagation is performed on the loss value corresponding to each of the object detection tasks to obtain the gradient corresponding to each of the object detection tasks;
[0020] Multiply the gradient corresponding to each of the target detection tasks by the weight coefficient corresponding to each of the target detection tasks to obtain the weighted gradient value corresponding to each of the target detection tasks.
[0021] The weighted gradient values corresponding to each of the target detection tasks are summed to obtain the first comprehensive gradient value;
[0022] The backbone network and the neck network are trained based on the first comprehensive gradient value, and the head network corresponding to each target detection task is trained based on the weighted gradient value corresponding to each target detection task.
[0023] In one possible implementation, generating a loss value corresponding to each target detection task based on the detection results output by the multiple head networks, and training the backbone network, the neck network, and the multiple head networks based on the loss values, includes:
[0024] Based on the detection results output by the multiple head networks, a loss value corresponding to each target detection task is generated;
[0025] Multiply the loss value corresponding to each target detection task by the label mask corresponding to each target detection task to obtain the update loss value corresponding to each target detection task.
[0026] The update loss value corresponding to each of the target detection tasks is multiplied by the weight coefficient corresponding to each of the target detection tasks and then summed to obtain the comprehensive loss value.
[0027] The combined loss value is backpropagated to obtain the second combined gradient value;
[0028] The backbone network, the neck network, and the multiple head networks are trained based on the second comprehensive gradient value.
[0029] In one possible implementation, the method further includes:
[0030] The weight coefficients for each target detection task are determined using a weight optimization algorithm.
[0031] In one possible implementation, the method further includes:
[0032] The batch training data corresponding to each target detection task is sampled according to a preset rule to obtain the sampled batch training data corresponding to each target detection task.
[0033] The step of inputting the batch training data into the multi-task object detection model includes:
[0034] The sampled batch training data corresponding to each of the object detection tasks is input into the multi-task object detection model.
[0035] Secondly, embodiments of this application provide a training apparatus for a multi-task object detection model, the apparatus comprising:
[0036] An acquisition unit is used to acquire training images for multiple object detection tasks and the labels of the training images;
[0037] The preprocessing unit is used to process the image to be trained and the label of the image to be trained into batch training data;
[0038] The input unit is used to input the batch training data into the multi-task target detection model. The multi-task target detection model includes a backbone network, a neck network, and multiple head networks. The backbone network is connected to the neck network, and the neck network is connected to multiple head networks. Each head network corresponds to one target detection task.
[0039] The training unit is used to generate a loss value corresponding to each target detection task based on the detection results output by the multiple head networks, and to train the backbone network, the neck network and the multiple head networks based on the loss value;
[0040] The triggering unit is used to trigger the repeated execution of the input unit inputting the batch training data into the multi-task target detection model and the training unit generating a loss value corresponding to each target detection task based on the detection results output by multiple head networks, and training the backbone network, the neck network and multiple head networks based on the loss value until a preset condition is reached.
[0041] Thirdly, embodiments of this application provide a training device for a multi-task target detection model, characterized in that it includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, it implements the training method for the multi-task target detection model as described in any of the above claims.
[0042] Fourthly, embodiments of this application provide a computer-readable storage medium, characterized in that the computer-readable storage medium stores instructions that, when executed on a terminal device, cause the terminal device to perform the training method of the multi-task target detection model as described in any of the preceding claims.
[0043] Therefore, the embodiments of this application have the following beneficial effects:
[0044] This application embodiment acquires training images and labels for multiple object detection tasks and processes them into batch training data. A multi-task object detection model is constructed, comprising a backbone network, a neck network, and multiple head networks, each head network corresponding to one object detection task. The batch training data is input into the multi-task object detection model to obtain the detection results output by multiple head networks, generating a loss value corresponding to each object detection task. The backbone network, neck network, and multiple head networks in the multi-task object detection model are then trained based on the loss values. By utilizing a single multi-task object detection model and different head networks, multiple object detection tasks are decomposed. Integrating different object detection tasks into the multi-task object detection model framework ensures the recognition accuracy for different obstacle categories, saves computational power, and improves the development and iteration efficiency of the object detection model. Attached Figure Description
[0045] Figure 1 A flowchart illustrating a training method for a multi-task object detection model provided in this application embodiment;
[0046] Figure 2 This is a schematic diagram of the dataset module based on gradient accumulation training in an embodiment of this application;
[0047] Figure 3 This is a schematic diagram of the dataset module based on hybrid gradient training in an embodiment of this application;
[0048] Figure 4 This is a schematic diagram of the multi-task target detection model in the embodiments of this application;
[0049] Figure 5 This is a schematic diagram of a gradient accumulation-based training strategy in an embodiment of this application;
[0050] Figure 6 This is a schematic diagram of a training strategy based on hybrid gradients in an embodiment of this application;
[0051] Figure 7 This is a schematic diagram illustrating the training of the multi-task object detection model in an embodiment of this application;
[0052] Figure 8 This is a schematic diagram of a training device for a multi-task target detection model provided in an embodiment of this application. Detailed Implementation
[0053] To make the above-mentioned objectives, features and advantages of the embodiments of this application more apparent and understandable, the embodiments of this application will be further described in detail below with reference to the accompanying drawings and specific implementation methods.
[0054] To facilitate understanding and explanation of the technical solutions provided in the embodiments of this application, the background technology of the embodiments of this application will be described first below.
[0055] Object detection algorithms belong to the visual perception module in the field of autonomous driving. Autonomous driving systems need to perceive dynamic and static obstacles around the vehicle in real time, such as vehicles, pedestrians, cyclists, and traffic cones. However, during data acquisition, the distribution of different types of obstacles is uneven. For example, for vehicles, large vehicles such as buses and trucks are more difficult to detect than cars; pedestrians, cyclists, and traffic cones are more difficult to detect than vehicles. If a traditional single-task object detection model is used, the category of each object appearing in every training image must be labeled. This leads to easily labeled categories occupying most of the training data, causing redundancy and increasing the cost of data labeling and storage, while rare categories are difficult to obtain.
[0056] Furthermore, as autonomous driving systems become increasingly complex, the requirements for visual perception modules will also increase, and the types of obstacles that perception models need to pay attention to will continue to increase. If dedicated target detection models are developed for different obstacle types, the downstream requirements for visual perception accuracy can be met. However, the computing power of onboard autonomous driving chips is limited, which will result in insufficient computing power of onboard chips.
[0057] Therefore, how to develop target detection models that meet performance and accuracy requirements under limited computing power constraints, and how to efficiently optimize these models, has become a key aspect of the iteration of autonomous driving technology.
[0058] Based on this, embodiments of this application provide a training method, apparatus, and device for a multi-task object detection model, which realizes object detection of different obstacles based on a multi-task framework. The input data of the multi-task object detection model can be decoupled according to the categories of object detection tasks, improving the efficiency of training data acquisition and annotation. Simultaneously, the detection head (head network) of the object detection model is decomposed according to different object detection tasks. Under the multi-task framework, different single-task object detection models are aggregated into a multi-task object detection model, reducing model computational consumption, saving computational power, and improving the efficiency of model development and iteration.
[0059] To facilitate understanding of the embodiments of this application, the training method of a multi-task object detection model provided by the embodiments of this application will be described below with reference to the accompanying drawings.
[0060] See Figure 1 As shown, this figure is a flowchart of a training method for a multi-task object detection model provided in an embodiment of this application. Figure 1 As shown, the method may include S101-S105:
[0061] S101: Obtain the training images and labels of the training images for multiple object detection tasks.
[0062] First, we obtain the training images and their corresponding labels for different object detection tasks. For a specific object detection task, the training images can include positive sample data and negative sample data. Positive sample data carries a positive sample (True) label, for example, a label value of 1, and negative sample data carries a negative sample (False) label, for example, a label value of 0. The positive sample images contain the target to be detected, while the negative sample images do not. For example, in the object detection task of pedestrian detection, the positive sample images contain pedestrians, while the negative sample images do not.
[0063] S102: Process the images to be trained and their labels into batch training data.
[0064] After obtaining the images to be trained and their labels, they can be preprocessed into batch training data to provide corresponding training data for subsequent multi-task object detection model training.
[0065] The S202 preprocessing steps vary depending on the single task type or detection category. Depending on the training strategy, it can be based on gradient accumulation training or hybrid gradient training. In practical applications, S202 can be implemented using the dataset module.
[0066] For preprocessing based on gradient accumulation training, in one possible implementation, the specific implementation of S102 processing the image to be trained and its labels into batch training data may include:
[0067] The training images and their labels for each object detection task are processed into batch training data corresponding to each object detection task.
[0068] In this approach, the training images and corresponding labels for each object detection task are processed separately. The processing involves batch training data for each object detection task, which includes a batch of training images and corresponding labels for that object detection task.
[0069] See Figure 2The diagram illustrates a dataset module for gradient accumulation training. This dataset module can include multiple sub-dataset modules, forming a collection of sub-dataset modules, each corresponding to an object detection task. Assuming the current object detection task includes Task A, Task B, and Task C, the sub-dataset modules correspond to the dataset modules for each object detection task, and their data preprocessing flow remains consistent with that of a single object detection task. Data from different object detection tasks (including training images and their corresponding labels) are input into the corresponding sub-dataset modules. These sub-dataset modules operate independently and can process data in parallel, outputting batches of training images (img batch) and their corresponding labels (GT batch).
[0070] In one possible implementation of preprocessing based on hybrid gradient training, S102, which processes the images to be trained and their labels into batch training data, may include:
[0071] A1: Based on the labels of the images to be trained, add corresponding batch labels for each image to be trained. The batch labels for each image to be trained include the labels of each object detection task arranged in order. The labels of each object detection task include the labels corresponding to its own object detection task and the labels corresponding to other object detection tasks. The labels corresponding to other object detection tasks are zero.
[0072] A2: Combine the images to be trained for each object detection task with the corresponding batch labels to form batch training data.
[0073] In this approach, training images for various object detection tasks can be mixed, and a corresponding batch label is added to each training image. The order of the object detection tasks corresponding to the batch labels of each training image is consistent. The batch label includes the label corresponding to the object detection task to which the training image belongs, and other label positions are filled with 0. For example, each training image corresponds to labels for tasks A, B, and C. If the training image is positive sample data for task A, then the batch label corresponding to the training image includes the ground truth label for task A, and the label positions for tasks B and C are filled with zero. Thus, the training images for each object detection task and their corresponding batch labels are combined to form batch training data.
[0074] See Figure 3The diagram illustrates a dataset module based on hybrid gradient training. This module includes data preprocessing procedures for various object detection tasks. All data from single-object detection tasks are randomly (or selectively) fed into the dataset module for processing. For data with corresponding ground truth labels for the object detection task, these labels are added, and zeros are used to fill other label positions. For data without corresponding ground truth labels, zeros are used to fill the labels, and a mask is used to indicate the label absence for each data point.
[0075] For example, inputting data0 from task A, data1 and data2 from task B, and data3 from task C into the dataset module directly outputs batch training data. In the batch training data, each training image corresponds to a label for task A, task B, and task C. Specifically, the training image img0 corresponding to data0 has the label for task A (GT 0), and the labels for tasks B and C are 0; the training image img1 corresponding to data1 has the label for task B (GT 1), and the labels for tasks A and C are 0; the training image img2 corresponding to data2 has the label for task B (GT 2), and the labels for tasks A and C are 0; the training image img3 corresponding to data3 has the label for task C (GT 3), and the labels for tasks A and B are 0. Here, GT 0, GT 1, GT 2, and GT 3 can be understood as the ground truth labels for the corresponding object detection tasks.
[0076] In the embodiments of this application, corresponding data preprocessing can be performed for different target detection task categories to decouple the data of different target detection categories.
[0077] S103: Input the batch training data into the multi-task object detection model. The multi-task object detection model includes a backbone network, a neck network, and multiple head networks. The backbone network is connected to the neck network, and the neck network is connected to multiple head networks. Each head network corresponds to an object detection task.
[0078] This application's embodiments construct a multi-task object detection model, inputting batch training data into the model to train it. The multi-task object detection model adopts a backbone-neck-head structure, where all object detection tasks share the parameters of the backbone and neck networks. Under the premise of meeting computational limitations, the shared backbone network should ideally be a network with a large number of parameters to ensure that useful representational features can be extracted for all tasks. The detection heads (head networks) for each object detection task are independent, and their model parameters are not shared to meet the requirements of different outputs between different tasks.
[0079] See Figure 4 The diagram illustrates a multi-task object detection model in an embodiment of this application. Each task corresponds to a branch. For example, there are three different single-task object detection tasks: vehicle 2 / 3D detection, pedestrian 2 / 3D detection, and cone 2 / 3D detection and cone keypoint detection. The backbone and neck share all parameters, allowing the three different tasks to be aggregated into an independent multi-task object detection, thereby reducing the number of model parameters and computational load. Tasks with the same data source are on the same branch, while tasks with different data sources are on different branches, ensuring the independence of data annotation for each single task.
[0080] For the batch training data obtained from preprocessing based on gradient accumulation training, data sampling can be performed before inputting it into the multi-task object detection model.
[0081] In one possible implementation, it may also include:
[0082] The batch training data corresponding to each object detection task is sampled according to the preset rules to obtain the sampled batch training data corresponding to each object detection task.
[0083] Then S103 inputs the batch training data into the multi-task object detection model, including:
[0084] The sampled batch training data for each object detection task is input into the multi-task object detection model.
[0085] In practical applications, training data is sampled for each iteration according to a preset sampling rule, resulting in sampled batch training data that is then input into the multi-task object detection model. Currently available samplers include SyncTaskSampler, RepeatTaskSampler, RollTaskSampler, and CircleTaskSample. Specifically, SyncTaskSampler samples batch training data from all object detection tasks simultaneously in each iteration; RepeatTaskSampler repeatedly samples a specific object detection task according to a preset number of repetitions in each iteration; RollTaskSampler samples only one object detection task from all object detection tasks in each iteration; and CircleTaskSampler samples only one object detection task from all object detection tasks throughout an entire epoch. This allows for the determination of the single object detection task participating in model training for each iteration, enabling more targeted model training.
[0086] S104: Based on the detection results output by multiple head networks, generate the loss value corresponding to each target detection task, and train the backbone network, neck network, and multiple head networks based on the loss value.
[0087] By inputting batch training data into the multi-task object detection model, the detection results output by the head network corresponding to different object detection tasks can be obtained, and the loss values of the different head network outputs can be obtained to update the model parameters for training the multi-task object detection model.
[0088] In the embodiments of this application, based on the preprocessing of gradient accumulation training described above, a gradient accumulation-based training strategy can be provided; based on the preprocessing of hybrid gradient training described above, a hybrid gradient-based training strategy can be provided. For details, please refer to the descriptions of the following embodiments, which will not be repeated here.
[0089] S105: Repeatedly execute the batch training data input into the multi-task object detection model and subsequent steps until the preset conditions are met.
[0090] Repeat steps S103-S104 to iteratively train the multi-task object detection model until preset conditions are met, such as reaching a preset threshold for the number of training epochs or a preset threshold for the loss value, thus obtaining the trained multi-task object detection model. The multi-task object detection model can perform multi-task object detection. By inputting the image to be detected into the multi-task object detection model, the detection results of multiple object detection tasks can be obtained. For example, it can simultaneously obtain detection results for whether vehicles, pedestrians, and traffic cones exist in the image.
[0091] This application embodiment acquires training images and labels for multiple object detection tasks and processes them into batch training data. A multi-task object detection model is constructed, comprising a backbone network, a neck network, and multiple head networks, each head network corresponding to one object detection task. The batch training data is input into the multi-task object detection model to obtain the detection results output by multiple head networks, generating a loss value corresponding to each object detection task. The backbone network, neck network, and multiple head networks in the multi-task object detection model are then trained based on the loss values. By utilizing a single multi-task object detection model and different head networks, multiple object detection tasks are decomposed. Integrating different object detection tasks into the multi-task object detection model framework ensures the recognition accuracy for different obstacle categories, saves computational power, and improves the development and iteration efficiency of the object detection model.
[0092] The following describes the gradient accumulation-based training strategy in the embodiments of this application.
[0093] In one possible implementation, S104 generates a loss value for each target detection task based on the detection results output by multiple head networks. The specific implementation of training the backbone network, neck network, and multiple head networks based on the loss values may include:
[0094] B1: Generate the loss value for each target detection task based on the detection results output by multiple head networks.
[0095] B2: Backpropagate the loss value corresponding to each object detection task to obtain the gradient corresponding to each object detection task.
[0096] B3: Multiply the gradient corresponding to each object detection task by the weight coefficient corresponding to each object detection task to obtain the weighted gradient value corresponding to each object detection task.
[0097] B4: Sum the weighted gradient values corresponding to each object detection task to obtain the first comprehensive gradient value.
[0098] B5: Train the backbone network and neck network based on the first comprehensive gradient value, and train the head network corresponding to each object detection task based on the weighted gradient value corresponding to each object detection task.
[0099] See Figure 5 The diagram shows a schematic of a gradient accumulation-based training strategy in an embodiment of this application.
[0100] During the training phase, the multi-task object detection model is input with batch training data output from the dataset module, which is trained based on gradient accumulation. Assuming the current multi-task object detection model consists of three single-object detection tasks, the training process in each iteration is as follows:
[0101] The batch training data corresponding to the three object detection tasks are sequentially fed into the multi-task object detection model for forward inference. Each set of data generates a feature map through the backbone and neck. i However, it will only use the specific detection head (head network) for each task. The loss value for each target detection task is obtained through the loss function specific to that task. i Loss value for each target detection task i Perform backpropagation to obtain the gradient Grad for the current object detection task. i Grad for each group of gradients i Multiply by a corresponding weighting coefficient GradScale i Once all the detection heads have been traversed, all Grades will be... i Multiply by GradScale i The values are accumulated to obtain the first comprehensive gradient value, Grad.
[0102] At this point, in the multi-object detection model, shared modules (such as backbone and neck) accumulate gradients for different tasks, while non-shared modules (such as head) maintain independent gradients for each task. The parameters of all modules are updated using the first combined gradient value, Grad, thus completing one iteration of training.
[0103] The following describes the training strategy based on hybrid gradients in the embodiments of this application.
[0104] In one possible implementation, S104 generates a loss value for each target detection task based on the detection results output by multiple head networks. The specific implementation of training the backbone network, neck network, and multiple head networks based on the loss values may include:
[0105] C1: Generate the loss value for each target detection task based on the detection results output by multiple head networks.
[0106] C2: Multiply the loss value corresponding to each object detection task by the label mask corresponding to each object detection task to obtain the updated loss value corresponding to each object detection task.
[0107] C3: Multiply the update loss value corresponding to each object detection task by the weight coefficient corresponding to each object detection task and sum them to obtain the comprehensive loss value.
[0108] C4: Backpropagate the combined loss value to obtain the second combined gradient value.
[0109] C5: Train the backbone network, neck network, and multiple head networks based on the second integrated gradient value.
[0110] See Figure 6 The diagram illustrates a training strategy based on hybrid gradients in an embodiment of this application. During the training phase, the multi-task object detection model is input with batch training data output from the dataset module trained using hybrid gradients. Assuming the current multi-task object detection model consists of three single-object detection tasks, the training process in each iteration is as follows:
[0111] The batch training data is fed into the multi-task object detection model for forward inference. After passing through the backbone, neck, and task-specific detection heads (head networks), the loss values for each of the three detection heads for the respective object detection task are obtained. i Each loss value is multiplied by a label mask consisting of 0s and 1s to ensure that each object detection task only calculates the loss corresponding to the ground truth value, thus obtaining the updated loss value. The label mask is the label mask of the same object detection task corresponding to a batch of training images in the batch training data. The updated loss value for each object detection task is multiplied by the corresponding weight coefficient GradScale. i The summation yields the final comprehensive loss value, Loss, for the multi-task object detection model. Backpropagation is then performed on the comprehensive loss value, Loss, to obtain the second comprehensive gradient values for all model parameters. Finally, the parameters of all modules are updated, completing one iteration of training.
[0112] The multi-object detection task model in this embodiment supports adding or removing object detection tasks as needed, integrating single tasks of different object detection categories into the multi-task object detection model framework. This framework supports different multi-task training strategies and weight optimization algorithms, which can be adjusted according to different actual tasks, improving development efficiency.
[0113] One possible implementation may also include: using a weight optimization algorithm to determine the weight coefficients corresponding to each object detection task.
[0114] Both of the aforementioned training strategies utilize the GradScale weight coefficient, which plays a crucial role in the performance of the final multi-task object detection model. This embodiment integrates various weight optimization algorithms, including high-performance algorithms such as Aligned_MTL, GradNorm, PCGrad, and MGDA. These can be configured as needed during the training of the multi-object detection model.
[0115] See Figure 7 The diagram illustrates the training of a multi-task object detection model in an embodiment of this application. Taking three object detection tasks as an example, the diagram integrates the aforementioned modules to construct a multi-task object detection training process encompassing data processing, model building, and training strategies. This framework allows for adjustments to the training configuration and the addition or removal of model tasks based on actual conditions. It is understood that a gradient-accumulation-based training strategy and a hybrid gradient-based training strategy are selected for execution.
[0116] In this embodiment, ground truth data for various object detection tasks can be labeled separately, which not only improves development efficiency but also reduces labeling costs. Aggregating multiple single-task models into a multi-object detection task model with shared parameters effectively reduces the computational load of the model, meeting the computational requirements of low-performance platforms. The multi-object detection task model has a simple structure; by adding or removing detection heads based on the original single-task model structure, it can be rapidly iterated and deployed without requiring additional post-processing steps. The proposed data processing and training strategies are flexibly configurable, effectively reducing the accuracy gap between the multi-object detection task model and the single-task model.
[0117] Based on the training method for a multi-task target detection model provided in the above-described embodiments, this application also provides a training device for a multi-task target detection model, which will be described below with reference to the accompanying drawings.
[0118] See Figure 8 As shown in the figure, this is a schematic diagram of the structure of a training device for a multi-task object detection model provided in an embodiment of this application. Figure 8 As shown, the training device for this multi-task object detection model includes:
[0119] The acquisition unit 801 is used to acquire training images for multiple object detection tasks and the labels of the training images;
[0120] The preprocessing unit 802 is used to process the image to be trained and the label of the image to be trained into batch training data;
[0121] Input unit 803 is used to input the batch training data into a multi-task target detection model. The multi-task target detection model includes a backbone network, a neck network, and multiple head networks. The backbone network is connected to the neck network, and the neck network is connected to multiple head networks. Each head network corresponds to one target detection task.
[0122] Training unit 804 is used to generate a loss value corresponding to each target detection task based on the detection results output by the multiple head networks, and to train the backbone network, the neck network and the multiple head networks based on the loss value;
[0123] Triggering unit 805 is used to trigger repeated execution of the input unit inputting the batch training data into the multi-task target detection model and the training unit generating a loss value corresponding to each target detection task based on the detection results output by multiple head networks, and training the backbone network, the neck network and multiple head networks based on the loss value until a preset condition is reached.
[0124] In one possible implementation, the preprocessing unit is specifically used for:
[0125] The training images and their labels for each object detection task are processed into batch training data corresponding to each object detection task.
[0126] In one possible implementation, the preprocessing unit is specifically used for:
[0127] Based on the labels of the images to be trained, a corresponding batch label is added to each image to be trained. The batch label corresponding to each image to be trained includes the labels of each of the target detection tasks arranged in order. The labels of each target detection task include the label corresponding to its own target detection task and the labels corresponding to other target detection tasks. The labels corresponding to the other target detection tasks are zero.
[0128] The training images for each of the target detection tasks, along with the corresponding batch labels for each training image, are combined into batch training data.
[0129] In one possible implementation, the training unit is specifically used for:
[0130] Based on the detection results output by the multiple head networks, a loss value corresponding to each target detection task is generated;
[0131] Backpropagation is performed on the loss value corresponding to each of the object detection tasks to obtain the gradient corresponding to each of the object detection tasks;
[0132] Multiply the gradient corresponding to each of the target detection tasks by the weight coefficient corresponding to each of the target detection tasks to obtain the weighted gradient value corresponding to each of the target detection tasks.
[0133] The weighted gradient values corresponding to each of the target detection tasks are summed to obtain the first comprehensive gradient value;
[0134] The backbone network and the neck network are trained based on the first comprehensive gradient value, and the head network corresponding to each target detection task is trained based on the weighted gradient value corresponding to each target detection task.
[0135] In one possible implementation, the training unit is specifically used for:
[0136] Based on the detection results output by the multiple head networks, a loss value corresponding to each target detection task is generated;
[0137] Multiply the loss value corresponding to each target detection task by the label mask corresponding to each target detection task to obtain the update loss value corresponding to each target detection task.
[0138] The update loss value corresponding to each of the target detection tasks is multiplied by the weight coefficient corresponding to each of the target detection tasks and then summed to obtain the comprehensive loss value.
[0139] The combined loss value is backpropagated to obtain the second combined gradient value;
[0140] The backbone network, the neck network, and the multiple head networks are trained based on the second comprehensive gradient value.
[0141] In one possible implementation, the device further includes:
[0142] The determining unit is used to determine the weight coefficients corresponding to each of the target detection tasks using a weight optimization algorithm.
[0143] In one possible implementation, the device further includes:
[0144] The sampling unit is used to sample the batch training data corresponding to each target detection task according to a preset rule, so as to obtain the sampled batch training data corresponding to each target detection task.
[0145] In one possible implementation, the input unit is specifically used for:
[0146] The sampled batch training data corresponding to each of the object detection tasks is input into the multi-task object detection model.
[0147] In addition, this application embodiment also provides a training device for a multi-task target detection model, including: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the training method for the multi-task target detection model as described in any of the above claims.
[0148] This application also provides a computer-readable storage medium storing instructions that, when executed on a terminal device, cause the terminal device to perform the training method for the multi-task object detection model as described in any of the preceding claims.
[0149] This application also provides a computer program product, including computer program instructions, which, when executed on a computer, cause the computer to perform the training method for the multi-task object detection model as described in any of the preceding claims.
[0150] This application embodiment acquires training images and labels for multiple object detection tasks and processes them into batch training data. A multi-task object detection model is constructed, comprising a backbone network, a neck network, and multiple head networks, each head network corresponding to one object detection task. The batch training data is input into the multi-task object detection model to obtain the detection results output by multiple head networks, generating a loss value corresponding to each object detection task. The backbone network, neck network, and multiple head networks in the multi-task object detection model are then trained based on the loss values. By utilizing a single multi-task object detection model and different head networks, multiple object detection tasks are decomposed. Integrating different object detection tasks into the multi-task object detection model framework ensures the recognition accuracy for different obstacle categories, saves computational power, and improves the development and iteration efficiency of the object detection model.
[0151] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the systems or apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple, and relevant parts can be referred to the method section.
[0152] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.
[0153] It should also be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
[0154] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.
[0155] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A training method for a multi-task object detection model, characterized in that, The method includes: Acquire training images for multiple object detection tasks and the labels of the training images; The images to be trained and their labels are processed into batch training data. The batch training data is input into a multi-task target detection model, which includes a backbone network, a neck network, and multiple head networks. The backbone network is connected to the neck network, and the neck network is connected to multiple head networks. Each head network corresponds to one target detection task. Based on the detection results output by the multiple head networks, a loss value corresponding to each target detection task is generated, and the backbone network, the neck network, and the multiple head networks are trained based on the loss value. Repeatedly input the batch training data into the multi-task object detection model and subsequent steps until the preset conditions are met.
2. The method according to claim 1, characterized in that, The step of processing the image to be trained and its labels into batch training data includes: The training images and their labels for each object detection task are processed into batch training data corresponding to each object detection task.
3. The method according to claim 1, characterized in that, The step of processing the image to be trained and its labels into batch training data includes: Based on the labels of the images to be trained, a corresponding batch label is added to each image to be trained. The batch label corresponding to each image to be trained includes the labels of each of the target detection tasks arranged in order. The labels of each target detection task include the label corresponding to its own target detection task and the labels corresponding to other target detection tasks. The labels corresponding to the other target detection tasks are zero. The training images for each of the target detection tasks, along with the corresponding batch labels for each training image, are combined into batch training data.
4. The method according to claim 2, characterized in that, The step of generating a loss value for each target detection task based on the detection results output by the multiple head networks, and training the backbone network, the neck network, and the multiple head networks based on the loss values includes: Based on the detection results output by the multiple head networks, a loss value corresponding to each target detection task is generated; Backpropagation is performed on the loss value corresponding to each of the object detection tasks to obtain the gradient corresponding to each of the object detection tasks; Multiply the gradient corresponding to each of the target detection tasks by the weight coefficient corresponding to each of the target detection tasks to obtain the weighted gradient value corresponding to each of the target detection tasks. The weighted gradient values corresponding to each of the target detection tasks are summed to obtain the first comprehensive gradient value; The backbone network and the neck network are trained based on the first comprehensive gradient value, and the head network corresponding to each target detection task is trained based on the weighted gradient value corresponding to each target detection task.
5. The method according to claim 3, characterized in that, The step of generating a loss value for each target detection task based on the detection results output by the multiple head networks, and training the backbone network, the neck network, and the multiple head networks based on the loss values includes: Based on the detection results output by the multiple head networks, a loss value corresponding to each target detection task is generated; Multiply the loss value corresponding to each target detection task by the label mask corresponding to each target detection task to obtain the update loss value corresponding to each target detection task. The update loss value corresponding to each of the target detection tasks is multiplied by the weight coefficient corresponding to each of the target detection tasks and then summed to obtain the comprehensive loss value. The combined loss value is backpropagated to obtain the second combined gradient value; The backbone network, the neck network, and the multiple head networks are trained based on the second comprehensive gradient value.
6. The method according to claim 4 or 5, characterized in that, The method further includes: The weight coefficients for each target detection task are determined using a weight optimization algorithm.
7. The method according to claim 2, characterized in that, The method further includes: The batch training data corresponding to each target detection task is sampled according to a preset rule to obtain the sampled batch training data corresponding to each target detection task. The step of inputting the batch training data into the multi-task object detection model includes: The sampled batch training data corresponding to each of the object detection tasks is input into the multi-task object detection model.
8. A training device for a multi-task target detection model, characterized in that, The device includes: An acquisition unit is used to acquire training images for multiple object detection tasks and the labels of the training images; The preprocessing unit is used to process the image to be trained and the label of the image to be trained into batch training data; The input unit is used to input the batch training data into the multi-task target detection model. The multi-task target detection model includes a backbone network, a neck network, and multiple head networks. The backbone network is connected to the neck network, and the neck network is connected to multiple head networks. Each head network corresponds to one target detection task. The training unit is used to generate a loss value corresponding to each target detection task based on the detection results output by the multiple head networks, and to train the backbone network, the neck network and the multiple head networks based on the loss value; The triggering unit is used to trigger the repeated execution of the input unit inputting the batch training data into the multi-task target detection model and the training unit generating a loss value corresponding to each target detection task based on the detection results output by multiple head networks, and training the backbone network, the neck network and multiple head networks based on the loss value until a preset condition is reached.
9. A training device for a multi-task object detection model, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the training method for the multi-task object detection model as described in any one of claims 1-7.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed on a terminal device, cause the terminal device to perform the training method for the multi-task object detection model as described in any one of claims 1-7.