Multi-task model training method based on adaptive moment estimation

Abstract

The invention discloses a multi-task model training method based on adaptive moment estimation, and the method comprises the following steps: 1, detecting multi-task model training in a first stage: 11, obtaining detection task data, and packaging the data; step 12, carrying out adaptive scaling adjustment; a thirteenth step of data enhancement; a fourteenth step in which the backbone network is shared; step 15, loss calculation; sixteenth, gradient calculation is carried out; seventhly, a multi-task decoupling adaptive moment estimation optimization strategy is adopted; a second step of performing freezing training on the multi-task model in a second stage; and 3, carrying out unfreezing training on the multi-task model in a third stage. According to the method, a multi-task decoupling self-adaptive moment estimation optimization strategy is adopted for multi-task model training based on self-adaptive moment estimation, multi-stage training is combined for multi-task model training, and the accuracy of the multi-task model is effectively improved.

Description

Technical Field

[0001] The present invention relates to the technical field of artificial intelligence and intelligent driving, and in particular to a multi-task model training method based on adaptive moment estimation. Background Art

[0002] As an important deep learning method, multi-task learning (MTL) is widely used in natural language processing (NLP), computer vision (CV) and other fields. Compared with single-task learning, multi-task learning has significant advantages, especially in terms of memory usage and reasoning speed. Multiple tasks share parameters, which not only reduces memory consumption, but also can output the results of multiple tasks simultaneously through a forward calculation, greatly improving reasoning efficiency. At the same time, by sharing parameters between multiple tasks, multi-task learning can effectively utilize the correlation between different tasks, thereby realizing knowledge transfer in model training and improving the generalization ability and overall performance of the model.

[0003] However, multi-task learning faces many challenges in practical applications, among which negative transfer and seesaw effect are the most common problems. Negative transfer refers to the mutual interference caused by conflicts and inconsistencies between different tasks, which ultimately makes the performance of some tasks worse than when trained separately. The seesaw effect is manifested as when the performance of one task improves, the performance of another task decreases, resulting in the instability of the entire multi-task model and difficulty in convergence.

[0004] Especially in the scenario of intelligent headlights, the perception model needs to output a variety of different types of information, such as vehicle detection, pedestrian detection, traffic sign detection, and lane segmentation. Traditional multi-task learning methods have difficulty learning the effective expression of each task through training when there is low similarity or conflict between multiple tasks, which affects the overall performance of the model. How to optimize the multi-task learning strategy to improve the accuracy and robustness of the multi-task model has become a problem that needs to be solved urgently. Summary of the invention

[0005] The present invention aims to solve at least one of the technical problems existing in the prior art.

[0006] To this end, the present invention proposes a multi-task model training method based on adaptive moment estimation, adopts an adaptive moment estimation optimization strategy with multi-task decoupling, and combines multi-stage training to perform multi-task model training, thereby effectively improving the accuracy of the multi-task model.

[0007] The multi-task model training method based on adaptive moment estimation according to an embodiment of the present invention comprises the following steps:

[0008] Step 1, stage 1 detection multi-task model training, the specific steps of step 1 are as follows:

[0009] Step 11, obtaining detection task data and data packaging: obtaining detection task data and packaging the obtained detection task data;

[0010] Step 12, adaptive scaling adjustment: resizing the original image, that is, adjusting the original image to the target size through an adaptive scaling adjustment method;

[0011] Step 13: Data enhancement: Perform data enhancement processing on the resized detection data;

[0012] Step 14, shared backbone network: The detection data after data enhancement processing is input into the shared backbone network, and the shared backbone network extracts multi-scale feature maps. These feature maps are transmitted to the Head network of each detection task through the fusion module for classification and positioning;

[0013] Step 15, loss calculation: Calculate the loss value of each detection task and calculate the total loss value, which is the sum of the loss values ​​of each detection task;

[0014] Step 16, gradient calculation: obtain the shared backbone network gradient and Head network gradient of the first stage of each detection task through back propagation calculation;

[0015] Step 17, multi-task decoupled adaptive moment estimation optimization strategy: update the shared backbone network and Head network parameters according to the gradient through the multi-task decoupled adaptive moment estimation optimization strategy;

[0016] Step 2: Stage 2 multi-task model freezing training: Add the segmentation task to the detection multi-task model of stage 1, and freeze the shared backbone network of the multi-task model at the same time;

[0017] Step 3, stage 3 multi-task model thawing training: thawing the frozen shared backbone network and giving a very low learning rate when updating the parameters.

[0018] The beneficial effects of the present invention are as follows: the present invention adopts a method combining multi-stage freeze-thaw training with a multi-task decoupled adaptive moment estimation optimization strategy to achieve a more effective multi-task model training method and improve the accuracy of the multi-task model; the detection task and the segmentation task are trained in stages, and this training method helps to avoid mutual interference problems caused by conflicts and inconsistencies between different tasks, thereby improving the performance of each detection task and segmentation task; the multi-task decoupled adaptive moment estimation optimization strategy innovatively decouples the gradients generated by multiple tasks, further preventing mutual interference between different tasks, and improving the adaptability and versatility of the multi-task model; generally speaking, the present invention improves the accuracy of each task in the multi-task model while sharing a backbone network to save memory.

[0019] According to one embodiment of the present invention, the specific steps of step 2 are as follows:

[0020] Step 21, obtaining detection task data, obtaining segmentation task data, and data packaging: obtaining detection task data and segmentation task data, and packaging the obtained detection task data and segmentation task data;

[0021] Step 22, adaptive scaling adjustment: resizing the original image, that is, adjusting the original image to a target size by an adaptive scaling adjustment method;

[0022] Step 23, data enhancement: perform data enhancement processing on the resized detection data and segmentation data;

[0023] Step 24, shared backbone network: The detection data and segmentation data after data enhancement are input into the shared backbone network, and the shared backbone network extracts multi-scale feature maps. These feature maps are transmitted to the Head network corresponding to the detection and segmentation tasks through the fusion module for further classification, positioning and segmentation;

[0024] Step 25, loss calculation: Calculate the loss value of each detection task, and at the same time calculate the loss value of the segmentation task, and calculate the total loss value. The total loss value is the sum of the segmentation task loss value and the loss value of each detection task;

[0025] Step 26, gradient calculation: The Head network gradient and segmentation Head network gradient of the second stage of each detection task are obtained through back propagation calculation;

[0026] Step 27, multi-task decoupled adaptive moment estimation optimization strategy: Update the detection Head network and segmentation Head network parameters according to the gradient through the multi-task decoupled adaptive moment estimation optimization strategy.

[0027] According to one embodiment of the present invention, the specific steps of step 3 are as follows:

[0028] Step 31, obtaining detection task data, obtaining segmentation task data, and data packaging: obtaining detection task data and segmentation task data, and packaging the obtained detection task data and segmentation task data;

[0029] Step 32, adaptive scaling adjustment: resizing the original image, that is, adjusting the original image to a target size by an adaptive scaling adjustment method;

[0030] Step 33, data enhancement: perform data enhancement processing on the resized detection data and segmentation data;

[0031] Step 34, shared backbone network: the detection data and segmentation data after data enhancement are input into the shared backbone network, and the shared backbone network extracts multi-scale feature maps, which are transmitted to the head network corresponding to the detection and segmentation tasks through the fusion module for further classification, positioning and segmentation;

[0032] Step 35, loss calculation: Calculate the loss value of each detection task, and at the same time calculate a segmentation task loss value, and calculate the total loss value, which is the sum of the segmentation task loss value and the loss value of each detection task;

[0033] Step 36, gradient calculation: The Head network gradient and segmentation Head network gradient of the third stage of each detection task are obtained through back propagation calculation;

[0034] In step 37, the parameters of the shared backbone network, detection Head network and segmentation Head network are updated according to the gradient through a multi-task decoupled adaptive moment estimation optimization strategy.

[0035] According to an embodiment of the present invention, in the twelfth step, the original image is resized, that is, the original image is resized to a target size of 960×512 by an adaptive scaling adjustment method. The specific steps of the adaptive scaling adjustment are as follows:

[0036] Step 121, calculate the scaling ratio. Assuming that the original size of the image is W×H and the target size is 960×512, first calculate two scaling ratios 960 / W and 512 / H. In order to ensure that the longest side of the image does not exceed the target size, choose the smaller scaling ratio.

[0037] Step 122, according to the selected scaling ratio, multiplying the width and height of the original image by the ratio respectively to obtain a scaled image size;

[0038] Step 123, determine the difference between the scaled image size and the target size, and calculate the number of pixels that need to be filled.

[0039] According to one embodiment of the present invention, in the fifteenth step, the loss value of the detection task consists of a category loss and a positioning loss.

[0040] According to one embodiment of the present invention, for category loss, the detection task adopts binary cross entropy loss.

[0041] According to one embodiment of the present invention, for the positioning loss, the detection task adopts the comprehensive IoU loss.

[0042] According to one embodiment of the present invention, in the step 25, the segmentation task loss adopts Dice loss.

[0043] According to one embodiment of the present invention, in the fourteenth step, the multi-scale feature maps extracted by the shared backbone network are fused through a fusion module.

[0044] According to one embodiment of the present invention, in the seventeenth step, the parameter at time t+1 is equal to the difference between the parameter at time t and the product of the update term and the learning rate.

[0045] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention are realized and obtained by the structures particularly pointed out in the description and the drawings.

[0046] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0048] Figure 1 It is a structural diagram of a multi-task model for detection in stage one of the present invention;

[0049] Figure 2 It is a flow chart of the adaptive moment estimation optimization strategy of multi-task decoupling of the present invention;

[0050] Figure 3 This is a structural diagram of the multi-task freezing model in the second stage of the present invention;

[0051] Figure 4 It is a structural diagram of the multi-task unfreezing model in stage three of the present invention;

[0052] Figure 5 It is an overall flow chart of the present invention. DETAILED DESCRIPTION

[0053] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0054] The following describes in detail the multi-task model training method based on adaptive moment estimation according to an embodiment of the present invention with reference to the accompanying drawings.

[0055] See Figure 5 The multi-task model training method based on adaptive moment estimation of the present invention comprises the following steps:

[0056] Step 1, stage 1 detection multi-task model training: see Figure 1 In the first stage, a multi-task model for training detection tasks is used. The detection tasks include but are not limited to vehicle detection, pedestrian detection, and traffic sign detection. For example, the number of detection tasks is 3: the detection task data (detection task data 1, detection task data 2, and detection task data 3) is obtained and packaged. The original image in the packaged data is adaptively scaled and adjusted, and the data is enhanced using the data enhancement method. The data is then input into the shared backbone network and the Head network corresponding to each detection task. That is, the adjusted detection data is input into the shared backbone network and the Head network corresponding to each detection task after the data enhancement method. The parameters of the shared backbone network are: The parameters of the detection Head network are Through the shared backbone network and the Head network corresponding to each detection task, the loss value of each detection task is obtained. The loss value of the detection task consists of category loss and positioning loss. The total loss value is the sum of the loss values ​​of each detection task. Then, the shared backbone network gradient of each detection task in the first stage is calculated through back propagation. and Head network gradient According to the gradient, the shared backbone network and the Head network are updated through the multi-task decoupled adaptive moment estimation optimization strategy, and the parameter update learning rate is α, that is, the parameters are updated using the learning rate α.

[0057] in,

[0058] are the shared backbone network parameters of the first phase;

[0059] Head network parameters for the first stage of detection task data 1;

[0060] Head network parameters for the first stage of detection task data 2;

[0061] Head network parameters for the first stage of detection task data 3;

[0062] To detect task data 1, the gradient of the backbone network is shared in the first stage;

[0063] To detect task data 2, the gradient of the backbone network is shared in the first stage;

[0064] To detect task data 3, the gradient of the backbone network is shared in the first stage;

[0065] To detect the gradient of the Head network part in the first stage of task data 1;

[0066] To detect the gradient of the Head network part in the first stage of task data 2;

[0067] This is the gradient of the Head network part in the first stage for detection task data 3.

[0068] The specific steps of step 1 are as follows:

[0069] Step 11: Acquire detection task data and package the data: First, acquire the detection task data and package the acquired detection task data;

[0070] Step 12, Adaptive scaling adjustment: resize the original image, that is, adjust the original image to a target size of 960×512 by using the adaptive scaling adjustment (Letter Resize) method, that is, the width of the target size is 960 pixels and the height is 512 pixels; the specific steps are as follows:

[0071] Step 121, calculate the scaling ratio. Assuming that the original size of the image is W×H, that is, the width of the original size is W pixels and the height is H pixels, and the target size is 960×512, first calculate two scaling ratios, the width scaling ratio is 960 / W, and the height scaling ratio is 512 / H. In order to ensure that the longest side of the image does not exceed the target size, a smaller scaling ratio is selected. That is to say, when 960 / W<512 / H, the selected scaling ratio is 960 / W; when 960 / W>512 / H, the selected scaling ratio is 512 / H.

[0072] Step 122, according to the selected scaling ratio, multiplying the width and height of the original image by the ratio respectively to obtain a scaled image size;

[0073] Step 123, determine the difference between the scaled image size and the target size, and calculate the number of pixels that need to be filled. Based on these differences, fill the top and bottom or left and right sides of the scaled image to make the final image meet the target size. In this method, the value of the filled pixel is zero (black filling).

[0074] Step 13: Data enhancement: Perform data enhancement processing on the resized detection data;

[0075] Step 14, shared backbone network: the detection data after data enhancement processing is input into the shared backbone network, and the shared backbone network extracts multi-scale feature maps, which are transmitted to the head network of each detection task through the fusion module for classification and positioning; the multi-scale feature maps extracted by the shared backbone network are fused through the fusion module, and the role of the fusion module is to improve the positioning and classification accuracy in target detection through multi-scale feature fusion, especially to provide better detection effects for targets of different sizes;

[0076] Step 15, loss calculation: Calculate the loss value of each detection task and calculate the total loss value. The total loss value is the sum of the loss values ​​of each detection task. The calculation formula for the total loss value of the detection task is as follows:

[0077]

[0078] The meanings of the symbols in formula (1) are:

[0079] L det Represents the total loss value of the detection task;

[0080] n represents the number of detection tasks;

[0081] represents the category loss of detection task i;

[0082] represents the localization loss of detection task i.

[0083] For category loss, the detection task uses binary cross entropy loss (BCE), which is calculated as follows:

[0084]

[0085] BCE(s,y)=-(ylog(s)+(1-y)log(1-s)) (3)

[0086] The meanings of the symbols in formula (2) and formula (3) are as follows:

[0087] L cls represents the class loss;

[0088] BCE(s m ,1) represents the cross entropy loss of positive samples in the category loss;

[0089] BCE(s n ,0) represents the cross entropy loss of negative samples in the category loss;

[0090] N pos represents the total number of positive samples;

[0091] N neg Represents the total number of negative samples;

[0092] BCE(s,y) represents the cross entropy loss of the predicted value s and the true value y;

[0093] s represents the predicted value;

[0094] y represents the true value;

[0095] s m is the predicted probability of the mth positive sample;

[0096] s n is the predicted probability of the nth negative sample.

[0097] For the positioning loss, the detection task uses the comprehensive IoU loss, which is calculated as follows:

[0098]

[0099] The meanings of the symbols in formula (4) are:

[0100] L CIoU represents the positioning loss, which is the overall IOU loss;

[0101] IoU represents the ratio of the overlap and union of the predicted box and the true box;

[0102] b represents the center point of the prediction box;

[0103] b gt Indicates the center point of the real box;

[0104] ρ represents the Euclidean distance between the center point of the predicted box and the center point of the real box;

[0105] c represents the distance between the diagonal of the minimum enclosing area containing the predicted box and the true box;

[0106] v represents the similarity of the aspect ratio between the predicted box and the real box;

[0107] α represents the weight coefficient.

[0108] v and α are calculated as follows:

[0109]

[0110] The meanings of the symbols in formula (5) and formula (6) are as follows:

[0111] width gt Indicates the width of the real box;

[0112] height gt Indicates the height of the real frame;

[0113] Width indicates the width of the prediction box;

[0114] Height indicates the height of the prediction box.

[0115] Step 16: Gradient calculation: Obtain the shared backbone network gradient of the first stage of each detection task through back propagation calculation and Head network gradient The gradient of parameter w is obtained according to the partial derivative of the loss function with respect to the parameter, and its calculation formula is:

[0116]

[0117] The meanings of the symbols in formula (7) are:

[0118] g represents the gradient, which is applicable to both the shared backbone network gradient and the Head network gradient;

[0119] L represents the loss value;

[0120] w represents parameters, which are applicable to both shared backbone network parameters and Head network parameters.

[0121] Step 17, multi-task decoupled adaptive moment estimation optimization strategy: Update the parameters of the shared backbone network and the Head network according to the gradient through the multi-task decoupled adaptive moment estimation optimization strategy. The parameter update learning rate is α, and the multi-task decoupled adaptive moment estimation optimization strategy flow chart is as follows Figure 2 As shown, the parameter update calculation formula is:

[0122]

[0123] The meanings of the symbols in formula (8) are:

[0124] w (t) represents the parameter at time t;

[0125] w (t+1) Represents the parameters at time t+1;

[0126] ∈ represents a small constant used to prevent division by zero;

[0127] represents the deviation correction value of the exponential moving average of the gradient at time t;

[0128] Represents the bias correction of the exponential moving average of the squared gradient at time t.

[0129] Formula (8) shows that the parameter at time t+1 is the parameter at time t minus the update term * learning rate (α), that is, the parameter at time t+1 is equal to the difference between the parameter at time t and the product of the update term and the learning rate. Formula (8) can be converted into formula (9):

[0130]

[0131] Among them, in formula (9) It indicates the update item.

[0132] and The calculation formulas are:

[0133]

[0134] The meanings of the symbols in formula (10) and formula (11) are as follows:

[0135] represents the deviation correction value of the exponential moving average of the gradient at time t-1;

[0136] Represents the deviation correction value of the exponential moving average of the square of the gradient at time t-1;

[0137] ρ1 represents The exponential decay rate of ρ1 is 0.9 in the present invention;

[0138] ρ2 represents The exponential decay rate of ρ2 is 0.999 in the present invention.

[0139] Step 2, stage 2 multi-task model frozen training: see Figure 3 In the second stage, segmentation tasks are added on the basis of the detection multi-task model of the first stage. The segmentation tasks include but are not limited to lane segmentation, semantic segmentation tasks, etc. At the same time, the shared backbone network of the multi-task model is frozen and does not participate in the parameter update process to prevent mutual influence between different tasks. First, the detection task and segmentation task data are obtained and the data is packaged. At the same time, the original image is resized by the adaptive scaling adjustment (LetterResize) method. Assume that the number of detection tasks is 3 and the number of segmentation tasks is 1: the packaged data will be enhanced by the data enhancement method, and then input into the frozen shared backbone network and the Head network corresponding to each task. The loss value of each task is obtained through forward propagation, that is, the adjusted data is input into the shared backbone network and the Head network corresponding to each task after the data enhancement method. The parameters of the shared backbone network are The parameters of the Head network are

[0140] in,

[0141] are the shared backbone network parameters of the second phase;

[0142] Head network parameters for detection task data 1 in the second stage;

[0143] Head network parameters for detection task data 2 in the second stage;

[0144] Head network parameters for detection task data 3 in the second stage;

[0145] Head network parameters for segmentation task data in the second stage;

[0146] The segmentation task loss uses Dice loss, and the calculation formula is as follows:

[0147]

[0148] The meanings of the symbols in formula (12) are:

[0149] L segRepresents the loss value of the segmentation task;

[0150] p i represents the predicted probability of the i-th pixel by the multi-task frozen model in stage 2;

[0151] y i represents the true label of the i-th pixel of the multi-task frozen model in stage 2;

[0152] ∈ represents a small constant that prevents the denominator from being zero.

[0153] Then, the Head network gradient of the second stage of each task is calculated by back propagation The Head network parameters are updated according to the gradient through the multi-task decoupled adaptive moment estimation optimization strategy, and the parameter update learning rate is α, that is, the parameters are updated using the learning rate α.

[0154] in,

[0155] To detect the gradient of the Head network part of task data 1 in the second stage;

[0156] To detect the gradient of the Head network part in the second stage of task data 2;

[0157] To detect the gradient of the Head network part of task data 3 in the second stage;

[0158] It is the gradient of the Head network part of the segmentation task data in the second stage.

[0159] The specific steps of step 2 are as follows:

[0160] Step 21, obtaining detection task data, obtaining segmentation task data, and data packaging: obtaining detection task data and segmentation task data, and packaging the obtained detection task data and segmentation task data;

[0161] Step 22, adaptive scaling adjustment: resizing the original image, that is, adjusting the original image to a target size by an adaptive scaling adjustment method;

[0162] Step 23, data enhancement: perform data enhancement processing on the resized detection data and segmentation data;

[0163] Step 24, shared backbone network: The detection data and segmentation data after data enhancement are input into the shared backbone network, and the shared backbone network extracts multi-scale feature maps. These feature maps are transmitted to the Head network corresponding to the detection and segmentation tasks through the fusion module for further classification, positioning and segmentation;

[0164] Step 25, loss calculation: Calculate the loss value of each detection task, and at the same time calculate the loss value of the segmentation task, and calculate the total loss value. The total loss value is the sum of the segmentation task loss value and the loss value of each detection task;

[0165] Step 26, gradient calculation: The Head network gradient and segmentation Head network gradient of the second stage of each detection task are obtained through back propagation calculation;

[0166] Step 27, multi-task decoupled adaptive moment estimation optimization strategy: Update the detection Head network and segmentation Head network parameters according to the gradient through the multi-task decoupled adaptive moment estimation optimization strategy.

[0167] Step 3, stage 3 multi-task model unfreezing training: see Figure 4 In stage three, the frozen shared backbone network is unfrozen and a very low learning rate is given when the parameters are updated. First, the packaged data of the detection and segmentation tasks are obtained. The packaged data is the same as the data in stage two. Assume that the number of detection tasks is 3 and the number of segmentation tasks is 1. At the same time, the original image is resized by the adaptive scaling adjustment (Letter Resize) method. The adjusted data is input into the shared backbone network and the Head network corresponding to each task after the data enhancement method. The parameters of the shared backbone network and the Head network corresponding to each task are the same as those in stage two. The parameters of the shared backbone network are The parameters of the Head network are The loss value of each task is obtained through forward propagation. In the back propagation stage, the shared backbone network gradient of each task in the third stage is then calculated through back propagation. and Head network gradient According to the gradient, the shared backbone network and Head network parameters are updated through the multi-task decoupled adaptive moment estimation optimization strategy, and the parameter update learning rate is That is, using the learning rate Update the parameters.

[0168] in,

[0169] are the shared backbone network parameters of the third phase;

[0170] Head network parameters for detection task data 1 in the third stage;

[0171] Head network parameters for detection task data 2 in the third stage;

[0172] To detect the Head network parameters of task data 3 in the third stage;

[0173] Head network parameters for segmentation task data in the third stage;

[0174] To detect task data 1, the gradient of the backbone network is shared in the third stage;

[0175] To detect task data 2, the gradient of the backbone network is shared in the third stage;

[0176] To detect task data 3, the gradient of the backbone network is shared in the third stage;

[0177] To segment the task data, the gradient of the backbone network is shared in the third stage;

[0178] To detect the gradient of the Head network part in the first stage of task data 1;

[0179] To detect the gradient of the Head network part in the first stage of task data 2;

[0180] To detect the gradient of the Head network part in the first stage of task data 3;

[0181] It is the gradient of the segmentation task data in the third stage Head network part.

[0182] The specific steps of step 3 are as follows:

[0183] Step 31, obtaining detection task data, obtaining segmentation task data, and data packaging: obtaining detection task data and segmentation task data, and packaging the obtained detection task data and segmentation task data;

[0184] Step 32, adaptive scaling adjustment: resizing the original image, that is, adjusting the original image to a target size through an adaptive scaling adjustment (Letter Resize) method;

[0185] Step 33, data enhancement: perform data enhancement processing on the resized detection data and segmentation data;

[0186] Step 34, shared backbone network: the detection data and segmentation data after data enhancement are input into the shared backbone network, and the shared backbone network extracts multi-scale feature maps, which are transmitted to the head network corresponding to the detection and segmentation tasks through the fusion module for further classification, positioning and segmentation;

[0187] Step 35, loss calculation: Calculate the loss value of each detection task, and at the same time calculate a segmentation task loss value, and calculate the total loss value, which is the sum of the segmentation task loss value and the loss value of each detection task;

[0188] Step 36, gradient calculation: The Head network gradient and segmentation Head network gradient of the third stage of each detection task are obtained through back propagation calculation;

[0189] In step 37, the parameters of the shared backbone network, detection Head network and segmentation Head network are updated according to the gradient through a multi-task decoupled adaptive moment estimation optimization strategy.

[0190] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A multi-task model training method based on adaptive moment estimation, characterized in that: The following steps are involved: Step 1, stage 1 detection multi-task model training, the specific steps of step 1 are as follows: Step 11, obtaining detection task data and data packaging: obtaining detection task data and packaging the obtained detection task data; Step 12, adaptive scaling adjustment: resizing the original image, that is, adjusting the original image to a target size through an adaptive scaling adjustment method; Step 13: Data enhancement: Perform data enhancement processing on the resized detection data; Step 14, shared backbone network: The detection data after data enhancement processing is input into the shared backbone network, and the shared backbone network extracts multi-scale feature maps. These feature maps are transmitted to the Head network of each detection task through the fusion module for classification and positioning; Step 15, loss calculation: Calculate the loss value of each detection task and calculate the total loss value, which is the sum of the loss values ​​of each detection task; Step 16, gradient calculation: obtain the shared backbone network gradient and Head network gradient of the first stage of each detection task through back propagation calculation; Step 17, multi-task decoupled adaptive moment estimation optimization strategy: update the shared backbone network and Head network parameters according to the gradient through the multi-task decoupled adaptive moment estimation optimization strategy; Step 2: Stage 2 multi-task model freezing training: Add the segmentation task to the detection multi-task model of stage 1, and freeze the shared backbone network of the multi-task model at the same time; Step 3, stage 3 multi-task model thawing training: thawing the frozen shared backbone network and giving a very low learning rate when updating the parameters.

2. The multi-task model training method based on adaptive moment estimation according to claim 1, characterized in that: The specific steps of the second step are as follows: Step 21, obtaining detection task data, obtaining segmentation task data, and data packaging: obtaining detection task data and segmentation task data, and packaging the obtained detection task data and segmentation task data; Step 22, adaptive scaling adjustment: resizing the original image, that is, adjusting the original image to a target size by an adaptive scaling adjustment method; Step 23, data enhancement: perform data enhancement processing on the resized detection data and segmentation data; Step 24, shared backbone network: The detection data and segmentation data after data enhancement are input into the shared backbone network, and the shared backbone network extracts multi-scale feature maps. These feature maps are transmitted to the Head network corresponding to the detection and segmentation tasks through the fusion module for further classification, positioning and segmentation; Step 25, loss calculation: Calculate the loss value of each detection task, and at the same time calculate the loss value of the segmentation task, and calculate the total loss value. The total loss value is the sum of the segmentation task loss value and the loss value of each detection task; Step 26, gradient calculation: The Head network gradient and segmentation Head network gradient of the second stage of each detection task are obtained through back propagation calculation; Step 27, multi-task decoupled adaptive moment estimation optimization strategy: Update the detection Head network and segmentation Head network parameters according to the gradient through the multi-task decoupled adaptive moment estimation optimization strategy.

3. The multi-task model training method based on adaptive moment estimation according to claim 1, characterized in that: The specific steps of the third step are as follows: Step 31, obtaining detection task data, obtaining segmentation task data, and data packaging: obtaining detection task data and segmentation task data, and packaging the obtained detection task data and segmentation task data; Step 32, adaptive scaling adjustment: resizing the original image, that is, adjusting the original image to a target size by an adaptive scaling adjustment method; Step 33, data enhancement: perform data enhancement processing on the resized detection data and segmentation data; Step 34, shared backbone network: the detection data and segmentation data after data enhancement are input into the shared backbone network, and the shared backbone network extracts multi-scale feature maps, which are transmitted to the head network corresponding to the detection and segmentation tasks through the fusion module for further classification, positioning and segmentation; Step 35, loss calculation: Calculate the loss value of each detection task, and at the same time calculate a segmentation task loss value, and calculate the total loss value, which is the sum of the segmentation task loss value and the loss value of each detection task; Step 36, gradient calculation: The Head network gradient and segmentation Head network gradient of the third stage of each detection task are obtained through back propagation calculation; In step 37, the parameters of the shared backbone network, detection Head network and segmentation Head network are updated according to the gradient through a multi-task decoupled adaptive moment estimation optimization strategy.

4. The multi-task model training method based on adaptive moment estimation according to claim 1, characterized in that: In the twelfth step, the original image is resized, that is, the original image is adjusted to a target size of 960×512 by an adaptive scaling adjustment method. The specific steps of the adaptive scaling adjustment are as follows: Step 121, calculate the scaling ratio. Assuming that the original size of the image is W×H and the target size is 960×512, first calculate two scaling ratios 960 / W and 512 / H. In order to ensure that the longest side of the image does not exceed the target size, choose the smaller scaling ratio. Step 122, according to the selected scaling ratio, multiplying the width and height of the original image by the ratio respectively to obtain a scaled image size; Step 123, determine the difference between the scaled image size and the target size, and calculate the number of pixels that need to be filled.

5. The multi-task model training method based on adaptive moment estimation according to claim 1, characterized in that: In the 15th step, the loss value of the detection task is composed of category loss and positioning loss.

6. The multi-task model training method based on adaptive moment estimation according to claim 5, characterized in that: For the category loss, the detection task uses binary cross entropy loss.

7. The multi-task model training method based on adaptive moment estimation according to claim 5, characterized in that: For the positioning loss, the detection task uses the comprehensive IoU loss.

8. The multi-task model training method based on adaptive moment estimation according to claim 2, characterized in that: In the step 25, the segmentation task loss adopts Dice loss.

9. The multi-task model training method based on adaptive moment estimation according to claim 1, characterized in that: In the fourteenth step, the multi-scale feature maps extracted by the shared backbone network are fused through a fusion module.

10. The multi-task model training method based on adaptive moment estimation according to claim 1, characterized in that: In the seventeenth step, the parameter at time t+1 is equal to the difference between the parameter at time t and the product of the update term and the learning rate.

Images