Multi-task neural network training method, multi-task processing method and device

Abstract

The present disclosure relates to the field of computer vision technology, and specifically provides a multi-task neural network training method, a multi-task processing method and a device. A multi-task neural network training method includes: inputting a pre-acquired image data set into a multi-task neural network to be trained, and determining the loss value of the image detection result of each subtask according to the output image detection result; based on the loss value of each subtask, adjusting the network parameters of the first sub-network shared by all sub-task networks in the multi-task neural network; for any associated task group included in the multi-task neural network, based on the loss value of each sub-task included in the associated task group, adjusting the network parameters of the second sub-network shared by each sub-task network in the associated task group until the convergence condition is met. The embodiments of the present disclosure improve the effect and efficiency of multi-task image detection.

Description

Technical Field

[0001] The present disclosure relates to the field of computer vision technology, and in particular to a multi-task neural network training method, a multi-task processing method and a device. Background Art

[0002] A multi-task neural network is a neural network that integrates multiple computer vision tasks into the same network and performs unified network training. Multi-task neural networks are a relatively new and practical direction in the field of deep learning, and have important applications in many fields, such as human detection and tracking, crowd behavior analysis, target attribute analysis, and abnormal behavior detection.

[0003] In the related art, when a multi-task neural network is trained, multiple tasks may restrict each other, resulting in poor network performance. Summary of the invention

[0004] In order to improve the image detection task performance of a multi-task neural network, the embodiments of the present disclosure provide a multi-task neural network training method and device, a multi-task processing method and device, an electronic device, and a storage medium.

[0005] In a first aspect, an embodiment of the present disclosure provides a multi-task neural network training method, comprising:

[0006] Inputting a pre-acquired image data set into a multi-task neural network to be trained, obtaining an image detection result of each subtask output by the multi-task neural network, and determining a loss value of the image detection result of each subtask according to the image detection result;

[0007] Based on the loss value of each subtask, adjusting the network parameters of the first sub-network shared by all sub-task networks in the multi-task neural network;

[0008] For any associated task group included in the multi-task neural network, based on the loss values ​​of each subtask included in the associated task group, the network parameters of the second subnetwork shared by each subtask network in the associated task group are adjusted until the convergence conditions are met; wherein the multi-task neural network includes at least two associated task groups, each of the associated task groups includes at least one subtask, and each subtask is used to perform an image detection task.

[0009] In some embodiments, the training method further comprises:

[0010] Using the image data set to perform network training on the image detection network to be trained, and obtaining the performance correlation between the various subtasks of the image detection network during the network training process; the image detection network includes the various subtasks of the multi-task neural network;

[0011] According to the performance correlation, each of the associated task groups included in the multi-task neural network is determined.

[0012] In some implementations, adjusting the network parameters of the first sub-network shared by all sub-task networks in the multi-task neural network based on the loss value of each sub-task includes:

[0013] Get the first weight parameter corresponding to each subtask;

[0014] According to the loss value of each subtask and the first weight parameter, the network parameters of the first subnetwork are adjusted.

[0015] In some implementations, adjusting the network parameters of the second sub-network shared by each sub-task network in the associated task group based on the loss value of each sub-task included in the associated task group includes:

[0016] Obtaining a second weight parameter corresponding to each subtask included in the associated task group;

[0017] The network parameters of the second sub-network are adjusted according to the loss value of each sub-task included in the associated task group and the second weight parameter.

[0018] In some embodiments, the image data set includes a sample image subset corresponding to each subtask; and inputting the pre-acquired image data set into the multi-task neural network to be trained to obtain the image detection result of each subtask output by the multi-task neural network includes:

[0019] According to the first sub-network of the multi-task neural network and the second sub-network corresponding to each sub-task, a training sub-network corresponding to each sub-task is established;

[0020] For the training sub-network corresponding to each sub-task, the sample image subset corresponding to the sub-task is input into the training sub-network to obtain the image detection result of the sub-task output by the training sub-network.

[0021] In some implementations, determining the loss value of the image detection result of each subtask according to the image detection result includes:

[0022] Obtain the image detection result of each subtask and label data of the image dataset;

[0023] The loss value is obtained according to the difference between the image detection result and the label data.

[0024] In a second aspect, an embodiment of the present disclosure provides a multi-tasking method, including:

[0025] Get the image to be processed;

[0026] The image to be processed is input into a pre-trained multi-task neural network to obtain an image detection result of each subtask output by the multi-task neural network; wherein the multi-task neural network is obtained according to the training method described in any embodiment of the first aspect.

[0027] In some implementations, inputting the image to be processed into a pre-trained multi-task neural network to obtain the image detection result of each subtask output by the multi-task neural network includes:

[0028] Inputting the image to be processed into the first sub-network of the multi-task neural network to obtain feature information of the image to be processed;

[0029] The feature information is respectively input into the second sub-network corresponding to each sub-task to obtain the image detection result of each sub-task.

[0030] In a third aspect, the present disclosure provides a multi-task neural network training device, including:

[0031] An input module is configured to input a pre-acquired image data set into the multi-task neural network to be trained, obtain an image detection result of each subtask output by the multi-task neural network, and determine a loss value of the image detection result of each subtask according to the image detection result;

[0032] A first adjustment module is configured to adjust network parameters of a first sub-network shared by all sub-task networks in the multi-task neural network based on the loss value of each sub-task;

[0033] The second adjustment module is configured to adjust the network parameters of the second sub-network shared by each sub-task network in any associated task group included in the multi-task neural network based on the loss values ​​of each sub-task included in the associated task group until the convergence condition is met; wherein the multi-task neural network includes at least two associated task groups, each of the associated task groups includes at least one sub-task, and each sub-task is used to perform an image detection task.

[0034] In some embodiments, the training device further comprises:

[0035] A correlation module is configured to perform network training on an image detection network to be trained using the image data set to obtain performance correlations between subtasks of the image detection network during the network training process; the image detection network includes subtasks of the multi-task neural network;

[0036] The grouping module is configured to determine each of the associated task groups included in the multi-task neural network according to the performance correlation.

[0037] In some embodiments, the first adjustment module is configured to:

[0038] Get the first weight parameter corresponding to each subtask;

[0039] According to the loss value of each subtask and the first weight parameter, the network parameters of the first subnetwork are adjusted.

[0040] In some embodiments, the second adjustment module is configured to:

[0041] Obtaining a second weight parameter corresponding to each subtask included in the associated task group;

[0042] The network parameters of the second sub-network are adjusted according to the loss value of each sub-task included in the associated task group and the second weight parameter.

[0043] In some implementations, the image dataset includes a sample image subset corresponding to each subtask; and the input module is configured to:

[0044] According to the first sub-network of the multi-task neural network and the second sub-network corresponding to each sub-task, a training sub-network corresponding to each sub-task is established;

[0045] For the training sub-network corresponding to each sub-task, the sample image subset corresponding to the sub-task is input into the training sub-network to obtain the image detection result of the sub-task output by the training sub-network.

[0046] In some embodiments, the input module is configured to:

[0047] Obtain the image detection result of each subtask and label data of the image dataset;

[0048] The loss value is obtained according to the difference between the image detection result and the label data.

[0049] In a fourth aspect, an embodiment of the present disclosure provides a multi-task processing device, including:

[0050] An acquisition module, configured to acquire an image to be processed;

[0051] The image detection module is configured to input the image to be processed into a pre-trained multi-task neural network to obtain the image detection result of each subtask output by the multi-task neural network; wherein the multi-task neural network is obtained according to the training method described in any embodiment of the first aspect.

[0052] In some embodiments, the image detection module is configured to:

[0053] Inputting the image to be processed into the first sub-network of the multi-task neural network to obtain feature information of the image to be processed;

[0054] The feature information is respectively input into the second sub-network corresponding to each sub-task to obtain the image detection result of each sub-task.

[0055] In a fifth aspect, an embodiment of the present disclosure provides an electronic device, including:

[0056] Processor; and

[0057] A memory storing computer instructions, wherein the computer instructions are used to enable a processor to execute the method described in any embodiment of the first aspect or the second aspect.

[0058] In a sixth aspect, an embodiment of the present disclosure provides a storage medium storing computer instructions, wherein the computer instructions are used to enable a computer to execute the method described in any embodiment of the first aspect or the second aspect.

[0059] The training method of the disclosed embodiment uses an image data set to train a first sub-network shared by all image detection tasks. At the same time, the image detection tasks are divided into multiple related task groups based on performance correlation, and the multiple sub-task loss values ​​of the same related task group are used to train the second sub-network corresponding to the related task group, thereby reducing the mutual constraints between different image detection tasks, taking into account the performance of all tasks, improving the multi-task image detection effect, and realizing the integration of more tasks, broadening the application scenarios of multi-task neural networks. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In order to more clearly illustrate the specific embodiments of the present disclosure or the technical solutions in the prior art, the drawings required for use in the specific embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0061] Figure 1 It is the network structure of the traditional multi-task neural network in related technologies.

[0062] Figure 2 This is the network structure of a multi-task neural network in some embodiments of the present disclosure.

[0063] Figure 3 This is the network structure of a multi-task neural network in some embodiments of the present disclosure.

[0064] Figure 4 is a flowchart of a method for training a multi-task neural network according to some embodiments of the present disclosure.

[0065] Figure 5 is a flowchart of a method for training a multi-task neural network according to some embodiments of the present disclosure.

[0066] Figure 6 It is a schematic diagram of a training method for a multi-task neural network according to some embodiments of the present disclosure.

[0067] Figure 7 is a flowchart of a method for training a multi-task neural network according to some embodiments of the present disclosure.

[0068] Figure 8 is a flowchart of a method for training a multi-task neural network according to some embodiments of the present disclosure.

[0069] Fig. 9 is a flowchart of a method for training a multi-task neural network according to some embodiments of the present disclosure.

[0070] Fig.10 It is a schematic diagram of a training method for a multi-task neural network according to some embodiments of the present disclosure.

[0071] Fig.11 is a flowchart of a multi-tasking method according to some embodiments of the present disclosure.

[0072] Fig.12 It is a structural block diagram of a multi-task neural network training device according to some embodiments of the present disclosure.

[0073] Fig.13 It is a structural block diagram of a multi-task neural network training device according to some embodiments of the present disclosure.

[0074] Fig.14 It is a structural block diagram of a multi-tasking processing device in some embodiments of the present disclosure.

[0075] Fig.15 It is a structural block diagram of an electronic device according to some implementations of the present disclosure. DETAILED DESCRIPTION

[0076] The technical solution of the present disclosure will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described implementation is a part of the implementation of the present disclosure, rather than all the implementations. Based on the implementation in the present disclosure, all other implementations obtained by ordinary technicians in the field without creative work are within the scope of protection of the present disclosure. In addition, the technical features involved in the different implementations of the present disclosure described below can be combined with each other as long as they do not conflict with each other.

[0077] Multi-task neural network is a relatively new and practical direction in the field of deep learning. Multi-task neural network refers to a neural network that integrates multiple visual tasks that use similar information into the same neural network and is obtained through unified network training. Multi-task neural network has important applications in many scenarios, such as human / face detection and tracking, crowd behavior analysis, human attribute analysis, and abnormal behavior detection.

[0078] In a multi-task neural network, the performance of each task needs to be optimized as much as possible. However, in a multi-task scenario, although the image features used by different tasks have a certain correlation, there may be mutual constraints between tasks, resulting in poor performance of the multi-task neural network.

[0079] The inventor of this case found through research that this is because, for multi-task neural networks, the receptive field sizes required for different tasks are different. For example, the receptive field sizes of image features required for face detection tasks and human behavior detection tasks are different, resulting in a mutually restrictive relationship between the learning of the two tasks. That is, as the learning performance of one task gradually strengthens, the effect of the other task degrades. Especially for scenarios with more tasks (such as seven or eight tasks), the multi-task neural network in the relevant technology is difficult to meet the use requirements.

[0080] Figure 1 The structure diagram of a multi-task neural network in the related art is shown. A multi-task neural network generally includes a multi-task shared part structure and a private part structure for each task.

[0081] For example Figure 1 As shown, the network structure of the part shared by multiple tasks mainly includes a backbone network 100 and a decoding network 200. The input of the backbone network 100 is the image to be processed, which is mainly used to perform deep feature extraction on the image to be processed to obtain a feature map of the image to be processed. The decoding network 200 is used to decode the feature map output by the backbone network 100, for example, up-sampling the feature map to obtain feature information related to the task that depends on a large-scale receptive field.

[0082] The private network structure of each task mainly includes a plurality of subtask networks 300 respectively connected after the decoding network 200, for example Figure 1 The example includes three subtask networks, that is, the multi-task network can process three tasks at the same time. The input of each subtask network 300 is the feature map output by the decoding network 200, and then each subtask network 300 predicts the output result corresponding to the respective task based on the feature map.

[0083] The inventors of this case have found that for the above-mentioned multi-task neural network, the receptive field size required for each task is different. For example, "subtask network 1" is a network for detecting key points of facial features, while "subtask network 2" is a network for detecting human body. Since the key point detection task of facial features focuses more on the regional features of the face, while the human body detection task focuses more on the regional features of the human body, the receptive field sizes required for the two are different. When training the multi-task neural network, it is necessary to train the shared part (that is, the backbone network 100 and the decoding network 200) according to the loss values ​​of all subtasks, and train the private part (that is, each subtask network) according to the loss value of each subtask. For the backbone network 100 and the decoding network 200 of the shared part of the network, when the learning performance of a certain task is enhanced, the performance of another one or more tasks will often degrade, making it difficult to take into account the performance of multiple tasks. It is also for this reason that the multi-task neural network in the related art can often only integrate 2 or 3 tasks at the same time, and it is difficult to achieve the integration of more (for example, 8 or 10) tasks.

[0084] It can be seen from this that the multi-task neural network in related technologies has poor performance, and it is difficult to take into account the performance of multiple tasks, and it is also difficult to integrate a larger number of tasks.

[0085] Based on the problems mentioned in the above-mentioned related technologies, the embodiments of the present disclosure provide a multi-task neural network training method, a multi-task detection method and device, an electronic device, and a storage medium, aiming to take into account multi-task performance at the same time, ensure that each task has a good effect, and at the same time realize network integration of a larger number of tasks.

[0086] In some embodiments, the present disclosure provides a multi-task neural network training method, which can be applied to an electronic device, and the electronic device performs one or more of the following method steps.

[0087] In the embodiments of the present disclosure, the electronic device may be any type of device with computing capabilities, such as handheld mobile terminals, desktop terminal devices, vehicle-mounted terminal devices, wearable devices, servers, and cloud platforms, etc., and the present disclosure does not impose any restrictions on this.

[0088] In the embodiments of the present disclosure, the multi-task neural network refers to a neural network that can perform multiple image detection tasks, such as face detection, body detection, human behavior detection, etc., and integrate these image detection tasks into the same neural network, which is the multi-task neural network described in the present disclosure.

[0089] Figure 2 The schematic diagram of the structure of the multi-task neural network in some embodiments of the present disclosure is shown below. Figure 2 The structure and principle of the multi-task neural network disclosed in the present invention are explained.

[0090] like Figure 2 As shown, the multi-task neural network of the embodiment of the present disclosure includes a first sub-network, a second sub-network and a sub-task network.

[0091] The first sub-network is the backbone network shared by all sub-tasks. The first sub-network is used to perform deep feature extraction on the image to be processed and obtain the first feature map of the image to be processed. For example, in an example, considering the lightweight of the network, the first sub-network can use a network based on Mobilenet V2, and its structure is as follows: Figure 3 Specifically, the first sub-network includes a batch normalization convolution layer (BN layer, Batch Normalization), which is used to preliminarily extract the feature information of the image to be processed, and then perform deep feature extraction on the image to be processed after upscaling the dimension. The scale of feature extraction by multiple reverse residual modules is getting smaller and smaller, and the dimension is getting higher and higher.

[0092] The second subnetwork is an intermediate layer network (Neck), which is used to decode the first feature map output by the first subnetwork, thereby upsampling the first feature map of a smaller image scale to obtain a second feature map with larger scale feature information, thereby providing the feature information required for tasks that rely on a large-scale receptive field.

[0093] In some embodiments, not all subtasks may share one second sub-network, but all subtasks may be divided into a plurality of related task groups according to relevance, and the subtasks in each related task group may share one second sub-network.

[0094] For example Figure 2 As shown in the figure, the multi-task neural network integrates subtasks 1 to subtask m, a total of m subtasks, and each subtask corresponds to its own private subtask network, that is, Figure 2 Subtask network 1 to subtask network m shown in .

[0095] In the embodiment of the present disclosure, when constructing a multi-task neural network, it is necessary to group multiple subtasks in advance according to the correlation between each subtask in all subtasks to obtain a related task group. Figure 2 As shown, the m subtasks are divided into n associated task groups, wherein associated task group 1 includes subtasks 1 to a1, associated task group 2 includes subtasks a1+1 to a2, ... associated task group n includes subtasks az+1 to m.

[0096] After the associated task groups are obtained, a second sub-network is set for each associated task group, so that for associated task groups 1 to n, the following are set: Figure 2 The second sub-networks 1 to n are shown, and there are n second sub-networks in total.

[0097] It can be understood that in the implementation manner of the present disclosure, the number of second sub-networks depends on the number of associated task groups divided by multiple sub-tasks. After determining the associated task groups of multiple sub-tasks, it is only necessary to copy the same number of second sub-networks. The specific number can be selected according to different application scenarios, and the present disclosure does not impose any restrictions on this.

[0098] After obtaining n second sub-networks, for each second sub-network, the private sub-task network of each sub-task can be connected to the back of the second sub-network according to the sub-tasks included in the corresponding associated task group. Taking the second sub-network 1 as an example, the associated task group 1 corresponding to the second sub-network 1 includes a total of a1 sub-tasks, namely, sub-tasks 1 to a1, so the sub-task network of the a1 tasks can be connected to the back of the second sub-network 1.

[0099] In the disclosed embodiment, the associated task group refers to the association between multiple subtasks, and the association can be understood as a positive correlation effect of mutual promotion between subtasks.

[0100] For example, for the two subtasks of "mask wearing detection" and "glasses wearing detection", both focus on the human facial area, and their requirements for receptive field and feature information are very similar. When the performance of one of them improves, the performance of the other also improves accordingly. By defining these two types of subtasks as related subtasks, the two types of tasks can be divided into the same related task group.

[0101] For another example, for the two subtasks of "mask detection" and "human body detection", the mask detection task focuses more on the human face area and requires a smaller receptive field, while the human body detection task focuses more on the human body area and requires a larger receptive field. When the performance of one task is improved, the performance of the other task may degrade. These two types of subtasks are defined as unrelated subtasks, and the two types of tasks cannot be classified into the same related task group.

[0102] In the following implementation of the present disclosure, the process of dividing multiple subtasks into associated task groups will be explained, which will not be expanded here.

[0103] Continue to see Figure 2 In the network structure shown in the figure, the second sub-network can adopt an upsampling module based on cubic convolution and bilinear interpolation, and use the upsampling module to upsample the first feature map output by the first sub-network, that is, to gradually decode the first feature map to obtain a second feature map with a larger scale and richer semantic information, so that the subsequent sub-task network can obtain more features and information of the required scale, so as to better complete the corresponding sub-task processing.

[0104] The subtask network is a head network (Head) private to each subtask, whose input is the second feature map output by the corresponding second subnetwork, and whose output is the output result corresponding to the subtask. For example, in one example, each subtask network includes three convolution layers, where the first layer is a convolution layer with a convolution kernel size of 1, which is used to reduce the dimension of the second feature map, the second layer is a convolution layer with a convolution kernel size of 3, which is used to add nonlinear transformation to each task, and the third layer is an output layer, which is used to output the output result of the subtask.

[0105] Based on the above Figure 2 The multi-task neural network shown in the figure, the network training process is described in the following embodiments of the present disclosure.

[0106] First of all, it is worth explaining that the training method of the present disclosure embodiment is to solve the following problems: Figure 1 In the traditional multi-task neural network shown in the figure, the performance of multiple image detection subtasks is mutually restricted and difficult to improve at the same time. This problem is caused by the fact that image detection subtasks with different image receptive fields share the same image features, which is in line with the laws of nature and is an objective technical problem in the field of multi-task image detection.

[0107] In the embodiment of the present disclosure, by finding relevant image detection tasks, dividing these tasks with positively correlated performance into the same associated task group, and separately training the second sub-network corresponding to the associated task group, it is possible to make the sub-tasks with similar image receptive fields share one second sub-network, while the sub-tasks with large differences in image receptive fields are respectively set in two associated task groups. Obviously, the scheme of the embodiment of the present disclosure is also a technical means that conforms to the laws of nature and is closely related to multi-task image detection.

[0108] Secondly, see Figure 4 As shown, in some embodiments, the training method of the multi-task neural network of the present disclosure example includes:

[0109] S410, inputting the pre-acquired image data set into the multi-task neural network to be trained, obtaining the image detection result of each subtask output by the multi-task neural network, and determining the loss value of the image detection result of each subtask according to the image detection result.

[0110] In the embodiments of the present disclosure, an image dataset refers to a sample image dataset used to train a multi-task neural network. The image dataset includes a plurality of sample data, each of which includes a sample image and label data of a corresponding task.

[0111] In some implementations, one dataset may be used for training multiple subtasks, that is, one sample data in an image dataset includes a sample image and label data for all tasks.

[0112] In other embodiments, a corresponding data set may be set for each subtask, that is, the image data set described in the present disclosure includes the same number of sample image subsets as the number of subtasks, and each sample image subset represents a sample data set of a subtask, that is, for a sample data in the sample image subset, it includes the sample image and the label data of the subtask. In the embodiments of the present disclosure, each task can be trained in parallel using the corresponding sample image subsets to improve the subtask performance and training efficiency. The following embodiments of the present disclosure are explained and will not be described in detail here.

[0113] Take a sample data in the image dataset as an example, input the sample data into Figure 2 In the multi-task neural network shown, the image detection result of each subtask output by the multi-task neural network is obtained.

[0114] It can be understood that the image detection result represents the prediction result of the multi-task neural network for the subtask, and the label data of the sample data represents the true value (Groundtruth). Based on the pre-set loss function, the loss value between the image detection result and the label data can be calculated. The loss value represents the difference between the network prediction result and the true result. The larger the loss value, the worse the network performance. Conversely, the smaller the loss value, the better the network performance. The goal of network training is to make the loss value of multiple iterations converge to a better range.

[0115] S420: Based on the loss value of each subtask, adjust the network parameters of the first subnetwork shared by all subtask networks in the multi-task neural network.

[0116] Still taking the iterative process in the above example as an example, after obtaining the difference between the image detection result of each subtask and the label data, that is, the loss value, the network parameters of the multi-task neural network can be optimized and adjusted using back propagation based on the loss value.

[0117] Specifically, in the embodiment of the present disclosure, the process of optimizing and adjusting the network parameters of the multi-task neural network can be divided into three parts: optimizing the first sub-network shared by all sub-task networks, optimizing the second sub-network shared by the associated task group, and optimizing the sub-task network private to each sub-task.

[0118] First, the optimization process of the subtask network private to each subtask is described. Figure 2 Taking the subtask network 1 shown as an example, after obtaining the image detection result 1 output by the subtask network 1, the loss value of the subtask 1 can be determined according to the image detection result 1 and the label data of the subtask 1, and then the network parameters of the subtask network 1 are optimized and adjusted according to the back propagation of the loss value. The same is true for each other subtask network, and this disclosure will not elaborate on this.

[0119] Secondly, the optimization process of the first sub-network shared by all sub-tasks is explained.

[0120] like Figure 2 As shown in the figure, after obtaining the image detection results output by each subtask network, the loss value of each subtask is obtained according to the image detection results and the corresponding label data. Since the first sub-network is a shared part of all sub-tasks, the first sub-network should learn the features required by all sub-tasks. When optimizing the first sub-network, the loss values ​​of each sub-task can be integrated, and the network parameters of the first sub-network can be optimized and adjusted using the loss values ​​of all sub-tasks.

[0121] S430. For any associated task group included in the multi-task neural network, based on the loss values ​​of each subtask included in the associated task group, adjust the network parameters of the second sub-network shared by each sub-task network in the associated task group until the convergence condition is met.

[0122] like Figure 2 As shown, combined with the foregoing, each second sub-network represents an intermediate layer network shared by related sub-tasks. Therefore, in the embodiments of the present disclosure, when training each second sub-network, it is necessary to train according to the loss values ​​of each sub-task included in the second sub-network.

[0123] Taking the second sub-network 1 as an example, after obtaining the loss values ​​corresponding to sub-tasks 1 to a1, the loss values ​​of sub-tasks 1 to a1 can be fused, and the fused loss values ​​can be used to optimize and adjust the network parameters of the second sub-network 1. Similarly, for the second sub-networks 2 to n, the optimization is also completed according to the above process.

[0124] The above describes the process of performing an iterative training using one sample data. For multiple sample data in the image data set, the above process can be repeated until the loss value of each network meets the convergence condition, thus completing the training of the above multi-task neural network.

[0125] Understandably, combined Figure 2 In the network structure shown, for the trained multi-task neural network, the multi-task feature information learned by each second sub-network is the feature information required by each sub-task in the associated task group corresponding to the second sub-network, and each sub-task in the associated task group is a task with a mutually reinforcing effect. Therefore, for each second sub-network after training, each sub-task included therein is a task with a mutually reinforcing effect, and there is no mutual restriction, so the performance of each task can be optimized, and the performance of all tasks can be effectively taken into account.

[0126] Furthermore, since the multi-task neural network of the disclosed embodiment can reduce the mutual constraints of different tasks as much as possible, it can be compatible with a larger number of tasks and realize the integration of several or even dozens of tasks. In general scenarios involving human body detection and analysis, the multi-task neural network of the disclosed embodiment can be compatible with more than 8 task detections.

[0127] From the above, it can be seen that in the embodiments of the present disclosure, by dividing the related image detection tasks into multiple related task groups and separately training the second sub-network shared by each related task group, the mutual constraints between different image detection tasks are reduced, while taking into account the performance of all tasks, improving the multi-task image detection effect, and realizing the integration of more tasks, broadening the application scenarios of multi-task neural networks.

[0128] In combination with the above, it can be seen that in the embodiments of the present disclosure, when constructing a multi-task neural network, it is necessary to divide the multiple subtasks included in the multi-task neural network according to their relevance to obtain multiple associated task groups. Figure 5 The implementation method describes the process of obtaining the associated task group.

[0129] like Figure 5 As shown, in some embodiments, the training method of the multi-task neural network of the disclosed example, the process of dividing multiple subtasks to obtain multiple associated task groups includes:

[0130] S510, using the image data set to perform network training on the image detection network to be trained, and obtaining the performance correlation between each subtask of the image detection network during the network training process.

[0131] S520: Determine various associated task groups included in the multi-task neural network according to performance correlation.

[0132] It can be understood that in the embodiments of the present disclosure, the purpose of grouping multiple subtasks is to find subtasks that have a mutually reinforcing effect among the multiple subtasks and divide the subtasks that have a mutually reinforcing effect into an associated task group.

[0133] To meet this goal, the image data set can be used to train the image detection network including the above-mentioned multiple subtasks, and the performance changes of each subtask during the network training process can be used to determine whether there is a correlation between the subtasks.

[0134] For example, in an example, during the training process of the image detection network, two subtasks show a pattern of simultaneous improvement or simultaneous decrease in performance, which means that the two subtasks are positively correlated and have relevance.

[0135] For example, in another example, during the training of the image detection network, if the performance of one of the two subtasks improves or decreases while the performance of the other is not affected, it means that the two subtasks are not related.

[0136] For example, in another example, during the training of the image detection network, two subtasks show a pattern in which the performance of one improves while the performance of the other decreases. This means that the two subtasks are negatively correlated, which means that they are not only unrelated but also conflicting.

[0137] Based on the above rules, subtasks with correlation among multiple subtasks can be divided into the same associated task group, and subtasks with conflict or no correlation can be divided into different associated task groups.

[0138] In some implementations, in order to ensure that the division of the associated task groups has a good reference, the same network structure as the multi-task neural network can be used as the image detection network, but in the image detection network, only one second sub-network is set, that is, all sub-tasks share the same second sub-network, for example Figure 6 In this way, by optimizing the second sub-network, the performance change of the loss value of each sub-task along with the optimization process of the second sub-network can be obtained, and the performance correlation between the sub-tasks can be obtained through the performance changes of multiple sub-tasks.

[0139] After determining the performance correlation between the subtasks, these subtasks can be grouped according to the performance correlation. That is, according to the rules described above, the subtasks with correlation are divided into the same associated task group, and the subtasks with conflicting or unrelated correlation are divided into different associated task groups, so as to obtain multiple associated task groups that are finally determined.

[0140] It can be understood that in the embodiments of the present disclosure, the multi-task neural network can be compatible with multiple image detection subtasks. When the multiple subtasks are grouped according to their association, they can be divided into at least two associated task groups. Of course, they can also be divided into more associated task groups. Those skilled in the art can complete the division of associated task groups through experiments according to the requirements of specific scenarios, and the present disclosure does not impose any restrictions on this.

[0141] After completing the division of the associated task groups of multiple subtasks, a second sub-network can be set for each associated task group, so that the constructed multi-task neural network can be as follows: Figure 2 As shown, no further elaboration is given.

[0142] From the above, it can be seen that in the embodiment of the present disclosure, by dividing multiple image detection subtasks into multiple related task groups, the subtasks with mutually reinforcing effects in each related task group share the same second sub-network, thereby greatly reducing the mutual constraints between tasks during training and improving the performance of the network for each task.

[0143] In some embodiments, considering that label data of different tasks are inconsistent during network training, the descent gradients are also different, and thus the learning rates of different tasks are also inconsistent, resulting in the performance bias of the first subnetwork or the second subnetwork of the shared part for different tasks.

[0144] Therefore, in the embodiment of the present disclosure, in order to balance the learning rates of different subtasks, in the process of optimizing and adjusting the network parameters of the first subnetwork and the second subnetwork by integrating the loss value of each subtask, a weight parameter can be set for each subtask, and the weight parameter can be used to balance the gradient and learning rate of each subtask. Figure 7 and Figure 8 The implementation method is described.

[0145] like Figure 7 As shown, in some embodiments, the training method for the task neural network of the example disclosed herein, the process of adjusting the network parameters of the first sub-network includes:

[0146] S710: Obtain a first weight parameter corresponding to each subtask.

[0147] S720: Adjust the network parameters of the first sub-network according to the loss value of each sub-task and the first weight parameter.

[0148] Combination Figure 2As shown in the figure, we still take one iteration process of a sample data as an example to illustrate. The sample image of the sample data is input into the multi-task neural network to obtain the image detection result of each subtask output by the multi-task neural network, and then the loss value of each subtask is obtained according to the difference between the image detection result of each subtask and the label data.

[0149] When fusing the loss values ​​of each subtask to perform back-propagation optimization on the first sub-network, it is also necessary to perform weighted balance on the loss values ​​based on the pre-set first weight parameter of each subtask.

[0150] Specifically, the corresponding hyperparameters, i.e., the first weight parameters, can be set for each image detection subtask in advance according to the different descent gradients of each subtask. After obtaining the loss value of each subtask, the loss value of each subtask can be multiplied by its corresponding first weight parameter, and then the loss values ​​are added and back-propagated to optimize and adjust the network parameters of the first subnetwork.

[0151] In some embodiments, for a multi-task neural network, the goal of network training is to make the first sub-network have a relatively balanced bias for each task, that is, the bias of each sub-task is maintained at a consistent level. Thus, a smaller first weight parameter can be set for a task with a larger gradient to reduce its learning rate; and a larger first weight parameter can be set for a task with a smaller gradient to increase its learning rate. This keeps the learning rate of each sub-task at a substantially consistent level.

[0152] In other embodiments, for a multi-task neural network, the goal of network training is to make the first sub-network have more bias towards high-priority target tasks, and have lower bias towards low-priority auxiliary tasks. Thus, a larger first weight parameter can be set for high-priority target tasks to increase their learning rate; and a smaller first weight parameter can be set for low-priority tasks to reduce their learning rate. This makes the multi-task network have more bias towards high-priority target tasks than low-priority auxiliary tasks.

[0153] In the implementation manner of the present disclosure, those skilled in the art may select a specific value of the first weight parameter according to a specific application scenario, and the present disclosure does not impose any limitation on this.

[0154] It can be seen from the above that in the embodiments of the present disclosure, the bias of the network towards each task can be adjusted through the first weight parameter to balance the learning rate of each subtask.

[0155] like Figure 8 As shown, in some embodiments, the training method for the task neural network of the example disclosed herein, the process of adjusting the network parameters of the second sub-network includes:

[0156] S810: Obtain a second weight parameter corresponding to each subtask included in the associated task group.

[0157] S820: Adjust the network parameters of the second sub-network according to the loss value and the second weight parameter of each sub-task included in the associated task group.

[0158] Combination Figure 2 As shown in the figure, an iterative process of a sample data is still used as an example for explanation. The image to be processed of the sample data is input into the multi-task neural network, and the image detection result of each subtask output by the multi-task neural network is obtained. Then, according to the difference between the image detection result of each subtask and the label data, the loss value of each subtask is obtained.

[0159] Taking the training process of the second sub-network 1 as an example, the corresponding associated task group 1 includes Figure 2 The image detection subtasks 1 to a1 are shown. When the loss values ​​of the subtasks 1 to a1 are integrated to optimize and adjust the network parameters of the second subnetwork 1, it is also necessary to perform weighted balance on the loss values ​​based on the second weight parameters of each subtask that are preset.

[0160] Specifically, the corresponding hyperparameters, i.e., the second weight parameters, can be set for each subtask in advance according to the different descent gradients of subtasks 1 to a1. After obtaining the loss values ​​of subtasks 1 to a1, the loss value of each subtask can be multiplied by its corresponding second weight parameter, and then the network parameters of the second subnetwork 1 can be optimized and adjusted by adding the loss values ​​and back-propagating.

[0161] Similarly, in some embodiments, for a multi-task neural network, the goal of network training is to make the second sub-network have a relatively balanced bias for each task, that is, the bias of each sub-task in the associated task group is maintained at a consistent level. Thus, a smaller second weight parameter can be set for a task with a larger gradient in the associated task group to reduce its learning rate; and a larger second weight parameter can be set for a task with a smaller gradient in the associated task group to increase its learning rate. The learning rate of each sub-task in the associated task group is maintained at a substantially consistent level.

[0162] In other embodiments, for a multi-task neural network, the goal of network training is to make the second sub-network have more bias towards high-priority target tasks, and have lower bias towards low-priority auxiliary tasks. Thus, a larger second weight parameter can be set for high-priority target tasks in the associated task group to increase their learning rate; and a smaller second weight parameter can be set for low-priority tasks in the associated task group to reduce their learning rate. This makes the multi-task network have more bias towards high-priority target tasks than low-priority auxiliary tasks.

[0163] In the implementation manner of the present disclosure, those skilled in the art may select a specific value of the second weight parameter according to a specific application scenario, and the present disclosure does not impose any limitation on this.

[0164] It can be seen from the above that in the embodiments of the present disclosure, the bias of the network towards each task can be adjusted by the second weight parameter to balance the learning rate of each subtask.

[0165] In such Figure 1 In the multi-task neural network in the related art shown, due to the mutual constraints between different tasks, it can only integrate a small number of similar tasks, such as facial key point detection, facial wearing object detection and other tasks that use similar information. Therefore, when training the network, it is only necessary to use the same training sample set to uniformly train multiple subtasks.

[0166] In the embodiments of the present disclosure, by grouping multiple subtasks into related task groups, the mutual constraints between different subtasks can be reduced, so that more subtasks with weak correlation can be integrated together. In this case, the training sample sets required for different subtasks may be quite different, and it is difficult to apply the same training sample set. Therefore, in some embodiments of the present disclosure, a corresponding sample image subset is set for each subtask, and the data of the sample image subset is used to train each subtask in parallel, so as to improve the performance of each subtask without increasing the training time.

[0167] Specifically, the image data set mentioned above in the present disclosure includes a sample image subset corresponding to each subtask, that is, each subtask has a corresponding training sample set. Of course, those skilled in the art will appreciate that for subtasks with very similar task objectives, two or more subtasks may also use the data of the same sample image subset, and the present disclosure does not limit this. Fig. 9 and Fig.10 The implementation method describes the training process.

[0168] like Fig. 9As shown, in some embodiments, the training method of the multi-task neural network of the present disclosure example, the process of inputting an image data set into the multi-task neural network to be trained, and obtaining the image detection result of each subtask output by the multi-task neural network, includes:

[0169] S910: Establish a training subnetwork corresponding to each subtask according to the first subnetwork of the multi-task neural network and the second subnetwork corresponding to each subtask.

[0170] S920: For each training sub-network corresponding to a sub-task, a sample image subset corresponding to the sub-task is input into the training sub-network to obtain an image detection result of the sub-task output by the training sub-network.

[0171] In the disclosed embodiment, since each subtask corresponds to its own subset of sample images, in order to achieve simultaneous parallel training of multiple tasks during the network training process, it is necessary to initialize a training subnetwork for each task in advance.

[0172] See also Figure 2 The multi-task neural network structure shown in the embodiment of the present disclosure initializes the training subnetwork of each subtask, including a first subnetwork, a second subnetwork corresponding to the subtask, and a subtask network.

[0173] For example Fig.10 The network structure of the training sub-network corresponding to the sub-tasks 1 to a1 included in the associated task group 1 is shown. Figure 2 and Fig.10 , for subtask 1, its corresponding training subnetwork includes the first subnetwork, the second subnetwork 1, and the subtask network 1. For subtask 2, its corresponding training subnetwork includes the first subnetwork, the second subnetwork 1, and the subtask network 2. Similarly, for subtask a1, its corresponding training subnetwork includes the first subnetwork, the second subnetwork 1, and the subtask network a1.

[0174] Of course, it is understandable. Fig.10 Due to limited space, only the training sub-networks of each sub-task corresponding to the relevant task group 1 are shown, and for each sub-task included in other relevant task groups, the corresponding training sub-networks are initialized according to the above process.

[0175] For example, for subtask a2, its corresponding training subnetwork includes the first subnetwork, the second subnetwork 2, and the subtask network a2. For subtask m, its corresponding training subnetwork includes the first subnetwork, the second subnetwork m, and the subtask network m.

[0176] In summary, in the embodiments of the present disclosure, a training subnetwork is established based on the first subnetwork and the second subnetwork corresponding to each subtask, and a private subtask network. Those skilled in the art can understand this, and the present disclosure will not be exhaustive.

[0177] Below is still Fig.10 The training sub-network of sub-tasks 1 to a1 is taken as an example to illustrate the training process of the multi-task neural network.

[0178] like Fig.10 As shown, each subtask corresponds to its own sample image subset, that is, sample image subset 1 of subtask 1, sample image subset 2 of subtask 2, ..., sample image subset a1 of subtask a1. The samples of each sample image subset are input into the corresponding training subnetwork, so that the image detection results of each training subnetwork can be obtained.

[0179] It can be understood that after obtaining the image detection results of each subtask, the loss value of each subtask can be obtained based on the training process of the aforementioned implementation method, and the loss value can be back-propagated to optimize and adjust the first subnetwork, the second subnetwork and the subtask network.

[0180] It is necessary to explain the characteristics that, in the embodiments of the present disclosure, for example Fig.10 As shown in the figure, when optimizing and adjusting the second sub-network 1 using the loss values ​​of sub-tasks 1 to a1, it is necessary to optimize and adjust the second sub-network 1 in all training sub-networks so that the network parameters remain consistent. Similarly, when optimizing and adjusting the first sub-network using the loss values ​​of sub-tasks 1 to m, it is necessary to optimize and adjust the first sub-network in all training sub-networks so that the network parameters remain consistent. For the private sub-task networks of each sub-task, it is only necessary to optimize and adjust their respective sub-task networks.

[0181] After the training of the sub-network is completed, you only need to restore the network to Figure 2 For the specific network training process, those skilled in the art may refer to any of the aforementioned implementations, and this disclosure will not elaborate on this.

[0182] It can be understood that in the embodiments of the present disclosure, a respective sample image subset is set for each subtask, so that the training is more targeted and the performance of the subtask is higher. In addition, in the training method of the embodiments of the present disclosure, multiple subtasks are trained in parallel at the same time, and there is no need to train each subtask in turn. On the basis of improving the performance of each subtask, the training time is shortened and the training efficiency is improved.

[0183] It can be seen from the above that in the disclosed embodiment, by dividing the image detection tasks with correlation into multiple associated task groups, and training the second sub-network shared by each associated task group separately, the mutual constraints between different tasks are reduced, while taking into account the performance of all image detection tasks, improving the multi-task detection effect, and can achieve the integration of more tasks, broadening the application scenarios of multi-task neural networks. Moreover, the bias of the network to each task can be adjusted through the weight parameters, and the learning rate of each subtask can be balanced. In addition, a respective sample image subset is set for each subtask, so that the training is more targeted, the performance of the subtask is higher, and multiple subtasks are trained in parallel at the same time. On the basis of improving the performance of each subtask, the training time is shortened and the training efficiency is improved.

[0184] In some embodiments, the present disclosure provides a multi-task detection method, which can be based on, for example, Figure 2 The multi-task neural network shown in the figure realizes various image detection tasks. Fig.11 The implementation method is described.

[0185] like Fig.11 As shown, in some embodiments, the multi-tasking processing method of the present disclosure example includes:

[0186] S1110, obtaining an image to be processed.

[0187] S1120, inputting the image to be processed into a pre-trained multi-task neural network, and obtaining the image detection result of each subtask output by the multi-task neural network.

[0188] In the embodiment of the present disclosure, the multi-task neural network may be, for example, Figure 2 The network structure shown and the network training process can be referred to the above-mentioned implementation mode, which will not be described in detail. After the multi-task neural network training is completed, the multi-task neural network can be used to process various image detection tasks.

[0189] The image to be processed refers to a scene image that is expected to perform a related image detection task. For example, in one example, the image to be processed can be a natural scene image captured by an image acquisition device. The image detection task can include, for example, face detection, human body detection, behavior detection, age detection, wearing object detection, etc., which is not limited in the present disclosure. In addition, the image to be processed can be a single image captured or a frame image in a video stream sequence, which is not limited in the present disclosure.

[0190] Combination Figure 2As shown, after obtaining the image to be processed, the image to be processed can be input into the first sub-network of the multi-task neural network, so as to obtain the feature information of the image to be processed output by the first sub-network. Then the feature information is respectively input into each second sub-network to obtain the feature map output by the second sub-network. Thereafter, the feature map output by the second sub-network is input into the sub-task network to which each second sub-network is connected, and the image detection result output by each sub-task network is obtained, that is, the multi-task result is obtained. Those skilled in the art can undoubtedly understand and fully implement it with reference to the aforementioned embodiments, and the present disclosure will not elaborate on this.

[0191] From the above, it can be seen that in the embodiments of the present disclosure, by dividing the related image detection tasks into multiple related task groups and separately training the second sub-network shared by each related task group, the mutual constraints between different image detection tasks are reduced, while taking into account the performance of all tasks, improving the multi-task image detection effect, and realizing the integration of more tasks, broadening the application scenarios of multi-task neural networks.

[0192] In some embodiments, the disclosed examples provide a training apparatus for a multi-task neural network.

[0193] like Fig.12 As shown, in some embodiments, the training device of the present disclosure example includes:

[0194] An input module 10 is configured to input a pre-acquired image data set into the multi-task neural network to be trained, obtain an image detection result of each subtask output by the multi-task neural network, and determine a loss value of the image detection result of each subtask according to the image detection result;

[0195] A first adjustment module 20 is configured to adjust network parameters of a first sub-network shared by all sub-task networks in the multi-task neural network based on the loss value of each sub-task;

[0196] The second adjustment module 30 is configured to adjust the network parameters of the second sub-network shared by each sub-task network in the associated task group for any associated task group included in the multi-task neural network, based on the loss values ​​of each sub-task included in the associated task group, until the convergence conditions are met; wherein the multi-task neural network includes at least two associated task groups, each associated task group includes at least one sub-task, and each sub-task is used to perform an image detection task.

[0197] From the above, it can be seen that in the embodiments of the present disclosure, by dividing the related image detection tasks into multiple related task groups and separately training the second sub-network shared by each related task group, the mutual constraints between different image detection tasks are reduced, while taking into account the performance of all tasks, improving the multi-task image detection effect, and realizing the integration of more tasks, broadening the application scenarios of multi-task neural networks.

[0198] like Fig.13 As shown, in some embodiments, the training device further includes:

[0199] The correlation module 40 is configured to perform network training on the image detection network to be trained using the image data set to obtain performance correlations between various subtasks of the image detection network during the network training process; the image detection network includes various subtasks of the multi-task neural network;

[0200] The grouping module 50 is configured to determine the associated task groups included in the multi-task neural network according to performance correlation.

[0201] From the above, it can be seen that in the embodiment of the present disclosure, by dividing multiple image detection subtasks into multiple related task groups, the subtasks with mutually reinforcing effects in each related task group share the same second sub-network, thereby greatly reducing the mutual constraints between tasks during training and improving the performance of the network for each task.

[0202] In some embodiments, the first adjustment module 20 is configured to:

[0203] Get the first weight parameter corresponding to each subtask;

[0204] According to the loss value of each subtask and the first weight parameter, the network parameters of the first subnetwork are adjusted.

[0205] It can be seen from the above that in the embodiments of the present disclosure, the bias of the network towards each task can be adjusted through the first weight parameter to balance the learning rate of each subtask.

[0206] In some embodiments, the second adjustment module 30 is configured to:

[0207] Obtaining a second weight parameter corresponding to each subtask included in the associated task group;

[0208] The network parameters of the second sub-network are adjusted according to the loss value of each sub-task included in the associated task group and the second weight parameter.

[0209] It can be seen from the above that in the embodiments of the present disclosure, the bias of the network towards each task can be adjusted by the second weight parameter to balance the learning rate of each subtask.

[0210] In some embodiments, the image dataset includes a sample image subset corresponding to each subtask; the input module 10 is configured to:

[0211] According to the first sub-network of the multi-task neural network and the second sub-network corresponding to each sub-task, a training sub-network corresponding to each sub-task is established;

[0212] For the training sub-network corresponding to each sub-task, the sample image subset corresponding to the sub-task is input into the training sub-network to obtain the image detection result of the sub-task output by the training sub-network.

[0213] In some embodiments, the input module 10 is configured to:

[0214] Obtain the image detection results of each subtask and the label data of the image dataset;

[0215] The loss value is obtained based on the difference between the image detection result and the label data.

[0216] It can be seen from the above that in the disclosed embodiment, by dividing the image detection tasks with correlation into multiple associated task groups, and training the second sub-network shared by each associated task group separately, the mutual constraints between different tasks are reduced, while taking into account the performance of all image detection tasks, improving the multi-task detection effect, and can achieve the integration of more tasks, broadening the application scenarios of multi-task neural networks. Moreover, the bias of the network to each task can be adjusted through the weight parameters, and the learning rate of each subtask can be balanced. In addition, a respective sample image subset is set for each subtask, so that the training is more targeted, the performance of the subtask is higher, and multiple subtasks are trained in parallel at the same time. On the basis of improving the performance of each subtask, the training time is shortened and the training efficiency is improved.

[0217] In some embodiments, the present disclosure provides a multi-tasking processing device, such as Fig.14 As shown, the multi-tasking processing device includes:

[0218] An acquisition module 60 is configured to acquire an image to be processed;

[0219] The image detection module 70 is configured to input the image to be processed into a pre-trained multi-task neural network to obtain the image detection results of each subtask output by the multi-task neural network; wherein the multi-task neural network is obtained according to the training method of any implementation method of the first aspect.

[0220] In some embodiments, the image detection module 70 is configured to:

[0221] Inputting the image to be processed into the first sub-network of the multi-task neural network to obtain feature information of the image to be processed;

[0222] The feature information is input into the second sub-network corresponding to each sub-task respectively to obtain the image detection result of each sub-task.

[0223] From the above, it can be seen that in the embodiments of the present disclosure, by dividing the related image detection tasks into multiple related task groups and separately training the second sub-network shared by each related task group, the mutual constraints between different image detection tasks are reduced, while taking into account the performance of all tasks, improving the multi-task image detection effect, and realizing the integration of more tasks, broadening the application scenarios of multi-task neural networks.

[0224] In some embodiments, the present disclosure provides an electronic device, including:

[0225] Processor; and

[0226] The memory stores computer instructions, where the computer instructions are used to enable the processor to execute the method of any of the above embodiments.

[0227] In some embodiments, the present disclosure provides a storage medium storing computer instructions, where the computer instructions are used to enable a computer to execute the method of any of the above embodiments.

[0228] Specifically, Fig.15 A schematic diagram of the structure of an electronic device 600 suitable for implementing the method disclosed in the present invention is shown. Fig.15 The electronic device shown can realize the corresponding functions of the above-mentioned processor and storage medium.

[0229] like Fig.15 As shown, the electronic device 600 includes a processor 601, which can perform various appropriate actions and processes according to the program stored in the memory 602 or the program loaded from the storage part 608 into the memory 602. In the memory 602, various programs and data required for the operation of the electronic device 600 are also stored. The processor 601 and the memory 602 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0230] The following components are connected to the I / O interface 605: an input section 606 including a keyboard, a mouse, etc.; an output section 607 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 608 including a hard disk, etc.; and a communication section 609 including a network interface card such as a LAN card, a modem, etc. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the I / O interface 605 as needed. A removable medium 611, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 610 as needed, so that a computer program read therefrom is installed into the storage section 608 as needed.

[0231] In particular, according to an embodiment of the present disclosure, the above method process can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program tangibly contained on a machine-readable medium, and the computer program includes a program code for executing the above method. In such an embodiment, the computer program can be downloaded and installed from a network through the communication part 609, and / or installed from a removable medium 611.

[0232] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and a module, a program segment or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0233] If the technical solution of this application involves personal information, the product using the technical solution of this application has clearly informed the personal information processing rules and obtained the individual's voluntary consent before processing the personal information. If the technical solution of this application involves sensitive personal information, the product using the technical solution of this application has obtained the individual's separate consent before processing the sensitive personal information, and at the same time meets the "explicit consent" requirement. For example, on personal information collection devices such as cameras, clear and prominent signs are set to inform that the personal information collection scope has been entered and personal information will be collected. If the individual voluntarily enters the collection scope, it is deemed that he or she agrees to the collection of his or her personal information; or on the device that processes personal information, the personal information processing rules are notified by obvious signs / information, and the individual's authorization is obtained through pop-up information or by asking the individual to upload his or her personal information; among them, the personal information processing rules may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the type of personal information processed.

[0234] Obviously, the above embodiments are merely examples for the purpose of clear explanation, and are not intended to limit the embodiments. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the embodiments here. The obvious changes or modifications derived therefrom are still within the scope of protection of the present invention.

Claims

1. A multi-task neural network training method, characterized in that: include: Inputting a pre-acquired image data set into a multi-task neural network to be trained, obtaining an image detection result of each subtask output by the multi-task neural network, and determining a loss value of the image detection result of each subtask according to the image detection result; Based on the loss value of each subtask, adjusting the network parameters of the first sub-network shared by all sub-task networks in the multi-task neural network, and based on the loss value of each subtask, adjusting the network parameters of the sub-task network private to the sub-task; For any associated task group included in the multi-task neural network, based on the loss values ​​of each subtask included in the associated task group, the network parameters of the second subnetwork shared by each subtask network in the associated task group are adjusted until the convergence conditions are met; wherein the multi-task neural network includes at least two associated task groups, each of the associated task groups includes at least one subtask, and each subtask is used to perform an image detection task.

2. The training method according to claim 1, characterized in that: Also includes: Using the image data set to perform network training on the image detection network to be trained, and obtaining the performance correlation between each subtask of the image detection network during the network training process; The image detection network includes each subtask of the multi-task neural network; According to the performance correlation, each of the associated task groups included in the multi-task neural network is determined.

3. The training method according to claim 1 or 2, characterized in that: The adjusting, based on the loss value of each subtask, the network parameters of the first sub-network shared by all sub-task networks in the multi-task neural network comprises: Get the first weight parameter corresponding to each subtask; According to the loss value of each subtask and the first weight parameter, the network parameters of the first subnetwork are adjusted.

4. The training method according to any one of claims 1 to 3, characterized in that: The adjusting, based on the loss values ​​of the subtasks included in the associated task group, the network parameters of the second sub-network shared by the subtask networks in the associated task group comprises: Obtaining a second weight parameter corresponding to each subtask included in the associated task group; The network parameters of the second sub-network are adjusted according to the loss value of each sub-task included in the associated task group and the second weight parameter.

5. The training method according to any one of claims 1 to 4, characterized in that: The image data set includes a sample image subset corresponding to each subtask; the pre-acquired image data set is input into the multi-task neural network to be trained to obtain the image detection result of each subtask output by the multi-task neural network, including: According to the first sub-network of the multi-task neural network and the second sub-network corresponding to each sub-task, a training sub-network corresponding to each sub-task is established; For the training sub-network corresponding to each sub-task, the sample image subset corresponding to the sub-task is input into the training sub-network to obtain the image detection result of the sub-task output by the training sub-network.

6. The training method according to any one of claims 1 to 5, characterized in that: The determining the loss value of the image detection result of each subtask according to the image detection result includes: Obtain the image detection result of each subtask and label data of the image dataset; The loss value is obtained according to the difference between the image detection result and the label data.

7. A multitasking method, characterized in that: include: Get the image to be processed; The image to be processed is input into a pre-trained multi-task neural network to obtain an image detection result of each subtask output by the multi-task neural network; wherein the multi-task neural network is obtained by the training method according to any one of claims 1 to 6.

8. The multitasking method according to claim 7, characterized in that: The step of inputting the image to be processed into a pre-trained multi-task neural network to obtain an image detection result of each sub-task output by the multi-task neural network includes: Inputting the image to be processed into the first sub-network of the multi-task neural network to obtain feature information of the image to be processed; The feature information is respectively input into the second sub-network corresponding to each sub-task to obtain the image detection result of each sub-task.

9. A multi-task neural network training device, characterized in that: include: An input module is configured to input a pre-acquired image data set into the multi-task neural network to be trained, obtain an image detection result of each subtask output by the multi-task neural network, and determine a loss value of the image detection result of each subtask according to the image detection result; A first adjustment module is configured to adjust network parameters of a first sub-network shared by all sub-task networks in the multi-task neural network based on the loss value of each sub-task, and to adjust network parameters of a sub-task network private to the sub-task based on the loss value of each sub-task; The second adjustment module is configured to adjust the network parameters of the second sub-network shared by each sub-task network in any associated task group included in the multi-task neural network based on the loss values ​​of each sub-task included in the associated task group until the convergence condition is met; wherein the multi-task neural network includes at least two associated task groups, each of the associated task groups includes at least one sub-task, and each sub-task is used to perform an image detection task.

10. A multi-tasking processing device, characterized in that: include: An acquisition module, configured to acquire an image to be processed; The image detection module is configured to input the image to be processed into a pre-trained multi-task neural network to obtain the image detection result of each subtask output by the multi-task neural network; wherein the multi-task neural network is obtained according to the training method according to any one of claims 1 to 6.

11. An electronic device, characterized in that: include: processor; and A memory storing computer instructions, wherein the computer instructions are used to enable a processor to execute the method according to any one of claims 1 to 8.

12. A storage medium, characterized in that: Computer instructions are stored, and the computer instructions are used to make a computer execute the method according to any one of claims 1 to 8.

Images