Multi-task target detection method, device, autonomous driving system and storage medium

By training a multi-task object detection model using a step-by-step method, the problems of task conflict and slow convergence speed under heterogeneous datasets are solved, and efficient and reliable multi-task object detection is achieved.

CN114821269BActive Publication Date: 2024-11-26安徽蔚来智驾科技有限公司
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202210507817.2
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-05-10
Publication Date
2024-11-26
Estimated Expiration
2042-05-10

AI Technical Summary

Technical Problem

In autonomous driving perception scenarios, heterogeneous multi-task datasets lead to inter-task conflicts, slow convergence speed, and poor model training results, affecting the reliability of multi-task target detection.

Method used

A step-by-step training method is adopted for multi-task object detection models. The model parameters are gradually optimized through a step-by-step training structure consisting of a main backbone network, branch backbone networks, and a head network. This includes a three-stage training process: initial training, fixing the main backbone network parameters, and releasing the network parameters, which is adapted to training on heterogeneous datasets.

Benefits of technology

It effectively prevents task conflicts, improves convergence speed, shortens training time, and enhances model training performance and the reliability of object detection.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114821269B_ABST
    Figure CN114821269B_ABST
Patent Text Reader

Abstract

The present invention provides a multi-task target detection method, device, automatic driving system and storage medium, including selecting some tasks in each task group to perform first-stage training on a to-be-trained model to obtain an initial training model, thereby preventing task conflicts and improving convergence speed; fixing the main backbone network parameters of the initial training model, using all tasks of each task group to perform second-stage training on the initial training model to obtain an intermediate training model, thereby realizing training of different tasks in homologous data sets or heterologous data sets without interfering with each other, thereby shortening the training time; releasing the main backbone network parameters of the intermediate training model, using all tasks of all task groups to perform third-stage training on the intermediate training model to obtain a multi-task target detection model, inputting an image to be detected containing a target detection object into the multi-task target detection model, detecting and identifying the target detection object, thereby improving the reliability of multi-task target detection.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of artificial neural network technology, and specifically provides a multi-task target detection method, equipment, automatic driving system and storage medium. Background Art

[0002] As one of the basic tasks in the field of computer vision, multi-task target detection has broad application prospects. In the perception scenario of autonomous driving, the recognition and detection of multi-task targets are required. If each task target uses a single-task learning model, the superimposed delay, computing resources and deployment difficulty are unbearable. The multi-task learning model shares the same backbone network to extract deep features. Each task is connected to a head network after the deep features output by the backbone network, which effectively solves the computing resources and reduces the delay.

[0003] However, in the actual implementation of autonomous driving perception, due to data cost issues, most multi-task data sets are heterogeneous, that is, each data sample has only one task annotation, and the distribution of data for different tasks is very different. In addition, the number of tasks is generally large, which is very different from the multi-task learning methods currently explored in the public domain, which use homologous data sets and have a small number of tasks. When using heterogeneous data to train multi-task models, it is easy to cause conflicts between tasks, slow convergence, and poor model training effects, which in turn affects the reliability of multi-task target detection. Summary of the invention

[0004] In order to overcome the above-mentioned defects, the present invention is proposed to provide a multi-task target detection method, device, autonomous driving system and storage medium that solve or at least partially solve the technical problem that heterogeneous multi-data easily lead to conflicts between tasks, slow convergence speed, poor model training effect, and thus affect the reliability of multi-task target detection.

[0005] In a first aspect, the present invention provides a multi-task target detection method, comprising:

[0006] Acquire an image to be detected containing at least one target detection object;

[0007] Inputting the image to be detected into a multi-task target detection model trained based on a step-by-step method to detect and identify the target detection object;

[0008] The multi-task target detection model is obtained by training the pre-built model to be trained step by step; the model to be trained includes a main backbone network shared by all tasks, multiple branch backbone networks for different task groups, and a head network used separately for each task. The pre-built model to be trained is trained step by step, including:

[0009] Selecting the traffic scene training images corresponding to some tasks in each task group to perform the first stage training on the model to be trained, to obtain an initial training model;

[0010] The main backbone network parameters of the initial training model are fixed, and the initial training model is trained in the second stage using the traffic scene training images corresponding to all tasks of each task group to obtain an intermediate training model;

[0011] The main backbone network parameters of the intermediate training model are released, and the intermediate training model is trained in the third stage using the traffic scene training images corresponding to all tasks of all task groups to obtain the multi-task target detection model.

[0012] Furthermore, in the above-mentioned multi-task target detection method, the intermediate training model is trained in the third stage using the traffic scene training images corresponding to all tasks of all task groups to obtain the multi-task target detection model, including:

[0013] Get the learning rate during the second stage of training;

[0014] Adjusting the learning rate during the second stage of training based on a preset learning rate reduction parameter to obtain a reduced learning rate;

[0015] Based on the reduced learning rate, the intermediate training model is trained in the third stage using traffic scene training images corresponding to all tasks in all task groups to obtain the multi-task target detection model.

[0016] Furthermore, in the multi-task target detection method described above, each of the branch backbone networks is respectively connected to the main backbone network and the head network of each task in the corresponding task group to form a network path corresponding to each task;

[0017] In at least one training phase, training is performed according to the following steps, wherein the training phase includes the first phase training, the second phase training, and the third phase training:

[0018] Based on the traffic scene training images corresponding to multiple tasks in the current training phase, the corresponding network paths are respectively forward trained and reverse trained to obtain the gradient of the main backbone network, the gradient of the branch backbone network and the gradient of the head network on each network path;

[0019] Based on the gradient of the main backbone network, the gradient of the branch backbone network and the gradient of the head network on each network path, the parameters of the model in the current training stage are updated until the convergence conditions are met to obtain the model that has completed the training in the current training stage.

[0020] Furthermore, in the above-mentioned multi-task target detection method, the convergence condition includes:

[0021] The loss value of each network path no longer decreases, and after testing the model in the current training stage using the traffic scene training images corresponding to the validation set, the accuracy of the test results no longer increases.

[0022] Furthermore, in the multi-task target detection method described above, forward training and reverse training are performed on the corresponding network path based on the traffic scene training image corresponding to the task in the current training phase to obtain the gradient of the main backbone network, the gradient of the branch backbone network and the gradient of the head network on each network path, including:

[0023] Based on the forward propagation algorithm, the corresponding network path is forward trained using the traffic scene training images corresponding to the task in the current training phase to obtain the loss value of each network path;

[0024] Based on the back propagation algorithm, the corresponding network path is reversely trained using the loss value to determine the gradient of the main backbone network, the gradient of the branch backbone network and the gradient of the head network on each network path.

[0025] Furthermore, in the above-mentioned multi-task target detection method, based on the gradient of the main backbone network, the gradient of the branch backbone network and the gradient of the head network on each network path, the parameters of the model in the current training stage are updated, including:

[0026] Superimposing the gradients of the main backbone network on all network paths to obtain the gradient superposition value of the main backbone network; and superimposing the gradients of the branch backbone network on each network path corresponding to each branch backbone network to obtain the gradient superposition value of each branch backbone network;

[0027] Based on the gradient superposition value of the main backbone network, the parameters of the main backbone network are updated, based on the gradient superposition of each branch backbone network, the parameters of each main backbone network are updated, and based on the gradient of the head network on each network path, the parameters of each head network are updated.

[0028] Furthermore, in the above-mentioned multi-task target detection method, traffic scene training images corresponding to some tasks in each task group are selected to perform the first stage training on the model to be trained, and the initial training model is obtained, including:

[0029] The traffic scene training images corresponding to the tasks with the largest data volume in each task group are respectively selected to perform the first stage training on the model to be trained to obtain the initial training model.

[0030] In a third aspect, a multi-task target detection device is provided, comprising a processor and a storage device, wherein the storage device is suitable for storing a plurality of program codes, and the program codes are suitable for being loaded and run by the processor to execute any of the multi-task target detection methods described above.

[0031] In a fourth aspect, an autonomous driving system is provided, comprising the multi-task target detection device as described above.

[0032] In a fifth aspect, a computer-readable storage medium is provided, wherein a plurality of program codes are stored therein, wherein the program codes are suitable for being loaded and run by a processor to execute the multi-task target detection method described in any one of the above technical solutions.

[0033] The above one or more technical solutions of the present invention have at least one or more of the following beneficial effects:

[0034] In the technical solution for implementing the present invention, the model to be trained is designed as a structure in which a main backbone network shared by all tasks, a branch backbone network shared by different task groups, and a head network used separately by each task are connected in sequence. During model training, traffic scene training images corresponding to some tasks in each task group are selected to perform the first stage training on the model to be trained to obtain an initial training model, and the main backbone network parameters of the initial training model are fixed. The initial training model is trained in the second stage using traffic scene training images corresponding to all tasks of each task group. After obtaining the intermediate training model, the main backbone network parameters of the intermediate training model are released. The intermediate training model is trained in the third stage using traffic scene training images corresponding to all tasks of all task groups, and finally the multi-task target detection model is obtained. In this way, during the first stage of training, since only part of the tasks are trained in each group, the amount of data is reduced, task conflicts are effectively prevented, the convergence speed is improved, and the training time is shortened. During the second stage of training, the tasks on the paths of the branch backbone networks after the main backbone network can be trained without interfering with each other while the parameters of the main backbone network remain unchanged, the head network parameters are optimized, and the training time is shortened. Moreover, it is not limited to whether the tasks come from the same source data set, and the tasks for the heterogeneous data sets can also be trained without interfering with each other. During the third stage of training, since the models of the first two stages are already in a convergence state, at this time, when all tasks are trained together, not only can the parameters of the main backbone network be further optimized, but also convergence can be completed faster to obtain the required multi-task target detection model. In this way, the conflict between tasks can be reduced, the convergence speed can be improved, and the model training effect can be improved. Finally, when the image to be detected is input into the multi-task target detection model trained based on the step-by-step method, the reliability of detecting and identifying the target detection object is higher.

[0035] Furthermore, in the first stage of training, the task with the largest amount of data in each task group is selected for training, which can avoid the problem that tasks with small amounts of data are difficult to converge when all tasks are trained together. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The disclosure of the present invention will become more easily understood with reference to the accompanying drawings. It is easy for those skilled in the art to understand that these drawings are only for illustrative purposes and are not intended to limit the scope of protection of the present invention. In addition, similar numbers in the figures are used to represent similar components, among which:

[0037] Figure 1 is a schematic flow chart of main steps of a multi-task target detection method according to an embodiment of the present invention;

[0038] Figure 2 is a structural block diagram of a model to be trained according to an embodiment of the present invention;

[0039] Figure 3 1 is a flow chart of main steps of a method for training a multi-task target detection model according to an embodiment of the present invention;

[0040] Figure 4 It is a main structural block diagram of a multi-task target detection device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0041] Some embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood by those skilled in the art that these embodiments are only used to explain the technical principles of the present invention and are not intended to limit the protection scope of the present invention.

[0042] In the description of the present invention, "module" and "processor" may include hardware, software or a combination of the two. A module may include hardware circuits, various suitable sensors, communication ports, and memories, and may also include software parts, such as program codes, or a combination of software and hardware. The processor may be a central processing unit, a microprocessor, an image processor, a digital signal processor, or any other suitable processor. The processor has data and / or signal processing functions. The processor may be implemented in software, hardware, or a combination of the two. Non-temporary computer-readable storage media include any suitable medium that can store program codes, such as a magnetic disk, a hard disk, an optical disk, a flash memory, a read-only memory, a random access memory, and the like. The term "A and / or B" means all possible combinations of A and B, such as only A, only B, or A and B. The term "at least one A or B" or "at least one of A and B" has a similar meaning to "A and / or B", and may include only A, only B, or A and B. The singular terms "one" and "the" may also include plural forms.

[0043] In the process of multi-task target detection, due to data cost issues, most multi-task data sets are heterogeneous, that is, each data sample has only one task annotation, and the distribution of data for different tasks is very different. In addition, the number of tasks is generally large, which is quite different from the multi-task learning methods currently explored in the public domain, which use homologous data sets and have a small number of tasks. As a result, when using heterogeneous data to train multi-task models, it is easy to cause conflicts between tasks, slow convergence, and poor model training effects, which in turn affects the reliability of multi-task target detection.

[0044] Therefore, in order to solve the above technical problems, the present invention provides the following technical solutions.

[0045] See attached Figure 1 , Figure 1 1 is a flowchart of the main steps of a multi-task target detection method according to an embodiment of the present invention. Figure 2 is a structural block diagram of a model to be trained according to an embodiment of the present invention, such as Figure 2 As shown, the model to be trained may include a main backbone network shared by all tasks, multiple branch backbone networks respectively used for different task groups, and a head network used separately for each task, and each branch backbone network is respectively connected to the main backbone network and the head network of each task in the corresponding task group to form a network path corresponding to each task. Among them, each branch backbone network can be divided according to the characteristics of each task, for example, the characteristics of the task can include semantic segmentation, target detection, etc.

[0046] Figure 2In the example, a network with two branch backbone networks, each of which is connected to three head networks of different tasks, is taken as an example. The main backbone network can be recorded as shared networks1, and the two branch backbone networks are recorded as shared networks2 and shared networks3 respectively. The head networks corresponding to the six tasks are recorded as task head1-task head6 in sequence. The first branch backbone network shared networks2 is connected to the first head network task head1-the third head network task head3, and the second branch backbone network shared networks3 is connected to the first head network task head4-the third head network task head6. The main backbone network shared networks1 is connected to the input data Data. shared networks1-shared networks2-task head1 is the first network path, and its corresponding loss function can be recorded as loss1; shared networks1-shared networks2-task head2 is the second network path, and its corresponding loss function can be recorded as loss2, and so on. No more examples are given here.

[0047] like Figure 1 As shown, the multi-task target detection method in the embodiment of the present invention mainly includes the following steps 101-102.

[0048] Step 101: Acquire an image to be detected containing at least one target detection object;

[0049] In a specific implementation process, the target detection object may include but is not limited to obstacles, pedestrians, traffic lights, and lane lines. The image to be detected containing the target detection object may be obtained through a vehicle-mounted camera, a millimeter radar, and the like.

[0050] Step 102: input the image to be detected into a multi-task target detection model trained based on a step-by-step method to detect and identify the target detection object;

[0051] In a specific implementation process, after the image to be detected is input into a multi-task target detection model trained based on the step-by-step method, the multi-task target detection model first divides the image to be detected according to the task characteristics corresponding to the image to be detected, and then inputs the divided image to be detected into the corresponding network path in turn for detection and identification, thereby obtaining the target detection object.

[0052] In a specific implementation process, the pre-built model to be trained can be trained step by step to obtain the training of the multi-task target detection model. The specific training process is as follows: Figure 3 As shown, Figure 3 FIG. 1 is a flow chart showing the main steps of a method for training a multi-task target detection model according to an embodiment of the present invention. Figure 3 As shown, the training method of the multi-task target detection model of this embodiment mainly includes steps 301 to 303;

[0053] Step 301, respectively selecting traffic scene training images corresponding to some tasks in each task group to perform first-stage training on the model to be trained to obtain an initial training model;

[0054] In a specific implementation process, the data volume of traffic scene training images corresponding to different tasks in each task is different. When all tasks are trained together, tasks with smaller data volume are more difficult to converge. If the resampling method is used for tasks with smaller data volume, the training convergence will be slow and the training time will be greatly extended. Therefore, in this embodiment, only the traffic scene training images corresponding to some tasks in each task group can be selected to perform the first stage training on the model to be trained, so as to update the main backbone network parameters of the model to be trained, the branch backbone network parameters of the model to be trained and the head network parameters corresponding to the tasks participating in the training in the model to be trained to obtain an initial training model.

[0055] In a specific implementation process, the traffic scene training images corresponding to the task with the largest data volume in each task group can be selected to perform the first stage training on the model to be trained to obtain the initial training model. In this way, the traffic scene training images corresponding to the task with a smaller data volume are no longer used to train the model to be trained, thereby preventing the problem of long training time caused by the difficulty in converging tasks with a smaller data volume. When the task with the largest data volume is used for training, the data volume is larger and no resampling is required. Moreover, the training results are more accurate than those of tasks with a smaller data volume.

[0056] In the first stage of training, training can be performed according to the following steps:

[0057] (1) Based on the traffic scene training images corresponding to multiple tasks in the current training stage, the corresponding network paths are respectively forward trained and reverse trained to obtain the gradient of the main backbone network, the gradient of the branch backbone network, and the gradient of the head network on each network path;

[0058] Specifically, based on the forward propagation algorithm, the corresponding network path can be forward trained using the traffic scene training images corresponding to the task in the current training phase to obtain the loss value of each network path; based on the back propagation algorithm, the corresponding network path can be reversely trained using the loss value to determine the gradient of the main backbone network, the gradient of the branch backbone network, and the gradient of the head network on each network path.

[0059] (2) Based on the gradient of the main backbone network, the gradient of the branch backbone network, and the gradient of the head network on each network path, the parameters of the model in the current training stage are updated until the convergence conditions are met to obtain the model that has completed the training in the current training stage.

[0060] Specifically, the gradients of the main backbone network on all network paths can be superimposed to obtain the gradient superposition value of the main backbone network; and the gradients of the branch backbone networks on each network path corresponding to each branch backbone network can be superimposed to obtain the gradient superposition value of each branch backbone network; based on the gradient superposition value of the main backbone network, the main backbone network parameters are updated, based on the gradient superposition of each branch backbone network, the parameters of each main backbone network are updated, and based on the gradient of the head network on each network path, the parameters of each head network are updated.

[0061] It should be noted that after completing one iteration of training, the next iteration of training is carried out directly. After reaching a certain number, a test on the validation set is performed to check the model accuracy of the current training stage until

[0062] The training can be stopped when the loss value of each network path no longer decreases and the model of the current training stage is tested using the traffic scene training images corresponding to the validation set, and the accuracy of the test results no longer increases.

[0063] Step 302: fix the main backbone network parameters of the initial training model, and perform second-stage training on the initial training model using the traffic scene training images corresponding to all tasks of each task group to obtain an intermediate training model;

[0064] In a specific implementation process, since only traffic scene training images corresponding to some tasks participate in model training in the first stage of training, the main backbone network parameters of the initial training model, the branch main backbone network parameters of the initial training model and the parameters of each head network in the initial training model are not optimal parameters. The branch network corresponding to each task group is not obtained by all tasks in the task group. Therefore, after obtaining the initial training model, the second stage of training is required.

[0065] In a specific implementation process, since all tasks share a main backbone network, and the main backbone network of the model to be trained has been updated after the first training stage, a certain training effect has been achieved. In order to shorten the training time, the main backbone network parameters of the initial training model can be fixed to remain unchanged, and the initial training model can be trained in the second stage using the traffic scene training images corresponding to all tasks of each task group, and the branch backbone network parameters of the initial training model and all the head network parameters in the initial training model are updated to obtain an intermediate training model.

[0066] In a specific implementation process, the second stage training process can be trained with reference to the first stage training process, which will not be described in detail here.

[0067] It should be noted that during the second stage of training, since the main backbone network parameters of the initial training model are fixed, even if the gradient of the main backbone network on each network path is obtained, it will not affect the main backbone network parameters of the initial training model. In this way, the branch main backbone network on each network path and the head network connected to the branch main backbone network can be trained simultaneously on different processors, thereby shortening the training time. It is not limited to whether the task comes from the same source data set. Tasks for heterogeneous data sets can also be trained without interfering with each other.

[0068] Step 303: Release the main backbone network parameters of the intermediate training model, and perform the third stage training on the intermediate training model using the traffic scene training images corresponding to all tasks of all task groups to obtain the multi-task target detection model.

[0069] In a specific implementation process, since only some tasks participate in the training of the main backbone network parameters during the first and second stage training, the main backbone network parameters are not the optimal parameters. Therefore, after obtaining the intermediate training model, the main backbone network parameters of the intermediate training model can be released. In this way, the intermediate training model is trained in the third stage using the traffic scene training images corresponding to all tasks of all task groups. The main backbone network parameters of the intermediate training model, the branch backbone network parameters of the intermediate training model, and the head network parameters corresponding to all tasks in the intermediate training model are updated again to obtain the multi-task target detection model.

[0070] In a specific implementation process, the third stage training process can also be trained with reference to the first stage training process, which will not be repeated here.

[0071] It should be noted that, since the main backbone network parameters have been preliminarily optimized after the first stage of training, the branch backbone network parameters and the head network parameters corresponding to all tasks in the second stage of training, that is, when the third stage of training is carried out, the training is carried out under the condition that the first two stages of training have completed convergence. Even if all tasks are trained together, it can complete convergence faster, and the main backbone network parameters of the multi-task target detection model are related to all tasks, and its accuracy is higher.

[0072] In addition, since the parameters of the main backbone network have been preliminarily optimized after the first stage of training, this step can be completed at a lower learning rate, thereby shortening the entire training time.

[0073] Specifically, the learning rate during the second stage of training can be obtained; the learning rate during the second stage of training is adjusted based on a preset learning rate reduction parameter to obtain a reduced learning rate; based on the reduced learning rate, the intermediate training model is trained in the third stage using all tasks of all task groups to obtain the multi-task target detection model. Typically, the preset learning rate reduction parameter can be adjusted to 1 / 10 of the learning rate of the previous training stage.

[0074] The multi-task target detection method of this embodiment designs the model to be trained as a structure in which a main backbone network shared by all tasks, a branch backbone network shared by different task groups, and a head network used separately by each task are connected in sequence. During model training, traffic scene training images corresponding to some tasks in each task group are selected to perform the first stage training on the model to be trained to obtain an initial training model, and the main backbone network parameters of the initial training model are fixed. The initial training model is trained in the second stage using the traffic scene training images corresponding to all tasks of each task group. After obtaining the intermediate training model, the main backbone network parameters of the intermediate training model are released. The intermediate training model is trained in the third stage using the traffic scene training images corresponding to all tasks of all task groups to finally obtain the multi-task target detection model. In this way, during the first stage of training, since only part of the tasks are trained in each group, the amount of data is reduced, task conflicts are effectively prevented, the convergence speed is improved, and the training time is shortened. During the second stage of training, the tasks on the paths of the branch backbone networks after the main backbone network can be trained without interfering with each other while the parameters of the main backbone network remain unchanged, the head network parameters are optimized, and the training time is shortened. Moreover, it is not limited to whether the tasks come from the same source data set, and the tasks for the heterogeneous data sets can also be trained without interfering with each other. During the third stage of training, since the models of the first two stages are already in a convergence state, at this time, when all tasks are trained together, not only can the parameters of the main backbone network be further optimized, but also convergence can be completed faster to obtain the required multi-task target detection model. In this way, the conflict between tasks can be reduced, the convergence speed can be improved, and the model training effect can be improved. Finally, when the image to be detected is input into the multi-task target detection model trained based on the step-by-step method, the reliability of detecting and identifying the target detection object is higher.

[0075] It should be pointed out that although the various steps in the above embodiments are described in a specific order, those skilled in the art can understand that in order to achieve the effects of the present invention, different steps do not have to be performed in such an order. They can be performed simultaneously (in parallel) or in other orders. These changes are within the scope of protection of the present invention.

[0076] It is understood by those skilled in the art that the present invention implements all or part of the processes in the method of the above embodiment, and can also be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable storage medium may include: any entity or device, medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory, random access memory, electric carrier signal, telecommunication signal and software distribution medium that can carry the computer program code. It should be noted that the content contained in the computer-readable storage medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable storage media do not include electric carrier signals and telecommunication signals.

[0077] Furthermore, the present invention also provides a multi-task target detection device.

[0078] Figure 4 FIG. 1 is a main structural block diagram of a multi-task target detection device according to an embodiment of the present invention. Figure 4 As shown, in an embodiment of a control device according to the present invention, a multi-task target detection device includes a processor 40 and a storage device 41. The storage device 41 can be configured to store a program for executing the multi-task target detection method of the above method embodiment, and the processor 40 can be configured to execute the program in the storage device 41, which includes but is not limited to a program for executing the control method of the smart home device of the above method embodiment. For ease of explanation, only the parts related to the embodiment of the present invention are shown. For specific technical details not disclosed, please refer to the method part of the embodiment of the present invention. The multi-task target detection device can be a control device formed by various electronic devices.

[0079] Furthermore, the present invention also provides an autonomous driving system, which may include the multi-task target detection device of the above embodiment.

[0080] Furthermore, the present invention also provides a computer-readable storage medium. In a computer-readable storage medium embodiment according to the present invention, the computer-readable storage medium can be configured to store a program for executing the multi-task target detection method of the above-mentioned method embodiment, and the program can be loaded and run by the processor to implement the above-mentioned multi-task target detection method. For ease of explanation, only the parts related to the embodiment of the present invention are shown. For specific technical details not disclosed, please refer to the method part of the embodiment of the present invention. The computer-readable storage medium can be a storage device formed by various electronic devices. Optionally, the computer-readable storage medium in the embodiment of the present invention is a non-temporary computer-readable storage medium.

[0081] Further, it should be understood that since the setting of each module is only for illustrating the functional units of the device of the present invention, the physical devices corresponding to these modules may be the processor itself, or a part of the software in the processor, a part of the hardware, or a part of the combination of software and hardware. Therefore, the number of each module in the figure is only schematic.

[0082] Those skilled in the art will appreciate that the modules in the device can be adaptively split or merged. Such splitting or merging of specific modules will not cause the technical solution to deviate from the principle of the present invention, and therefore, the technical solutions after splitting or merging will fall within the protection scope of the present invention.

[0083] So far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it is easy for those skilled in the art to understand that the protection scope of the present invention is obviously not limited to these specific embodiments. Without departing from the principle of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present invention.

Claims

1. A multi-task object detection method, It is characterized in that Acquire an image to be detected containing at least one target detection object; Inputting the image to be detected into a multi-task target detection model trained based on a step-by-step method to detect and identify the target detection object; The multi-task target detection model is obtained by training the pre-built model to be trained step by step; the model to be trained includes a main backbone network shared by all tasks, multiple branch backbone networks for different task groups, and a head network used separately for each task. The pre-built model to be trained is trained step by step, including: Selecting the traffic scene training images corresponding to some tasks in each task group to perform the first stage training on the model to be trained, so as to obtain an initial training model; The main backbone network parameters of the initial training model are fixed, and the initial training model is trained in the second stage using the traffic scene training images corresponding to all tasks of each task group to obtain an intermediate training model; The main backbone network parameters of the intermediate training model are released, and the intermediate training model is trained in the third stage using the traffic scene training images corresponding to all tasks of all task groups to obtain the multi-task target detection model.

2. The multi-task target detection method according to claim 1, It is characterized in that The intermediate training model is trained in the third stage using the traffic scene training images corresponding to all tasks of all task groups to obtain the multi-task target detection model, including: Get the learning rate during the second stage of training; Adjusting the learning rate during the second stage of training based on a preset learning rate reduction parameter to obtain a reduced learning rate; Based on the reduced learning rate, the intermediate training model is trained in the third stage using traffic scene training images corresponding to all tasks in all task groups to obtain the multi-task target detection model.

3. The multi-task target detection method according to claim 1, It is characterized in that Each of the branch backbone networks is respectively connected to the main backbone network and the head network of each task in the corresponding task group to form a network path corresponding to each task; In at least one training phase, training is performed according to the following steps, wherein the training phase includes the first phase training, the second phase training, and the third phase training: Based on the traffic scene training images corresponding to multiple tasks in the current training phase, the corresponding network paths are respectively forward trained and reverse trained to obtain the gradient of the main backbone network, the gradient of the branch backbone network and the gradient of the head network on each network path; Based on the gradient of the main backbone network, the gradient of the branch backbone network and the gradient of the head network on each network path, the parameters of the model in the current training stage are updated until the convergence conditions are met to obtain the model that has completed the training in the current training stage.

4. The multi-task target detection method according to claim 3, It is characterized in that The convergence conditions include: The loss value of each network path no longer decreases, and after testing the model in the current training stage using the traffic scene training images corresponding to the validation set, the accuracy of the test results no longer increases.

5. The multi-task target detection method according to claim 3, It is characterized in that Based on the traffic scene training images corresponding to the tasks in the current training phase, the corresponding network paths are forward trained and reverse trained to obtain the gradients of the main backbone network, the branch backbone network, and the head network on each network path, including: Based on the forward propagation algorithm, the corresponding network path is forward trained using the traffic scene training images corresponding to the task in the current training phase to obtain the loss value of each network path; Based on the back propagation algorithm, the corresponding network path is reversely trained using the loss value to determine the gradient of the main backbone network, the gradient of the branch backbone network and the gradient of the head network on each network path.

6. The multi-task target detection method according to claim 3, It is characterized in that Based on the gradient of the main backbone network, the gradient of the branch backbone network, and the gradient of the head network on each network path, the parameters of the model in the current training phase are updated, including: Superimposing the gradients of the main backbone network on all network paths to obtain the gradient superposition value of the main backbone network; and superimposing the gradients of the branch backbone network on each network path corresponding to each branch backbone network to obtain the gradient superposition value of each branch backbone network; Based on the gradient superposition value of the main backbone network, the parameters of the main backbone network are updated, based on the gradient superposition of each branch backbone network, the parameters of each main backbone network are updated, and based on the gradient of the head network on each network path, the parameters of each head network are updated.

7. The multi-task target detection method according to claim 1, It is characterized in that The traffic scene training images corresponding to some tasks in each task group are selected to perform the first stage training on the model to be trained, and the initial training model obtained includes: The traffic scene training images corresponding to the tasks with the largest amount of data in each task group are respectively selected to perform the first stage training on the model to be trained to obtain the initial training model.

8. A multi-task target detection device, comprising a processor and a storage device, wherein the storage device is suitable for storing a plurality of program codes, It is characterized in that The program code is suitable for being loaded and run by the processor to execute the multi-task target detection method according to any one of claims 1 to 7.

9. An autonomous driving system, It is characterized in that Comprising the multi-task target detection device as described in claim 8.

10. A computer-readable storage medium having a plurality of program codes stored therein, It is characterized in that The program code is suitable for being loaded and run by a processor to execute the multi-task target detection method according to any one of claims 1 to 7.