Multi-task image processing model training method and device, equipment, medium and product

Abstract

The embodiment of the invention provides a multi-task image processing model training method and device, equipment, a medium and a product. In the method, training images selected in a training data set are respectively input into an initial multi-task image processing model and a single-task image processing model corresponding to each image processing single task to obtain a feature map and a processing result corresponding to each image processing single task, and after a multi-task feature map is obtained, the multi-task image processing model and the single-task image processing model are processed. And calculating a loss value according to the feature map, the processing result and the multi-task feature map. And carrying out model training on the initial multi-task image processing model according to the loss value and the training data set to obtain a target multi-task image processing model. In the scheme, each image in the training data set has a label of an image processing single task, the images are existing images, the images do not need to be processed, the training cost and the image manufacturing cost are reduced, and the training efficiency is also improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a multi-task image processing model training method, device, equipment, medium and product. Background Art

[0002] With the development of science and technology, different single-task image processing models can be used for processing different single image processing tasks. For example, an obstacle detection model is used to detect road images to obtain the location and type of obstacles in the road images, a speed limit sign detection model is used to detect road images to obtain the location and type of speed limit signs in the road images, a lane line detection model is used to process images to obtain the location and type of lane lines in the road images, an image segmentation model is used to process images to obtain segmented images, and a depth estimation model is used to process images to obtain depth maps, etc. When there are multiple single image processing tasks to be processed, in order to improve processing efficiency, a multi-task image processing model can be used to process one image to obtain the processing results of multiple single image processing tasks.

[0003] In the prior art, for the training of a multi-task image processing model, a new label is usually firstly annotated on images with a label of a single image processing task to obtain images with labels corresponding to multiple image processing tasks, and then these images are used to train the initial deep learning model to obtain a multi-task image processing model.

[0004] In summary, existing multi-task image processing model training methods require processing the image first, resulting in high training costs and image production costs. Summary of the invention

[0005] The multi-task image processing model training method, device, equipment, medium and product provided in the embodiments of the present application are used to solve the problem that the existing multi-task image processing model training method requires the image to be processed first, resulting in high training costs and image production costs.

[0006] In a first aspect, an embodiment of the present application provides a multi-task image processing model training method, comprising:

[0007] Selecting a training image from a training data set, wherein each image in the training data set has a label of a single image processing task;

[0008] Input the training images into the single-task image processing models corresponding to each single image processing task, respectively, to obtain the feature maps corresponding to each single image processing task, and input the training images into the initial multi-task image processing model, to obtain the multi-task feature maps and the processing results corresponding to each single image processing task;

[0009] Calculating a loss value according to the feature map corresponding to each of the single image processing tasks, the multi-task feature map, and the processing result corresponding to each of the single image processing tasks;

[0010] The initial multi-task image processing model is trained according to the loss value and the training data set to obtain a target multi-task image processing model.

[0011] In a possible implementation, calculating the loss value according to the feature map corresponding to each of the single image processing tasks, the multi-task feature map, and the processing result corresponding to each of the single image processing tasks includes:

[0012] Determine a first sub-loss value corresponding to each of the image processing single tasks according to the processing result corresponding to each of the image processing single tasks and the label of the training image;

[0013] For each of the single image processing tasks, calculating a second sub-loss value corresponding to the single image processing task according to the feature map corresponding to the single image processing task and the multi-task feature map;

[0014] The loss value is calculated according to the labels of the images in the training data set, and the first sub-loss value and the second sub-loss value corresponding to each of the image processing tasks.

[0015] In a possible implementation, determining the first sub-loss value corresponding to each of the image processing tasks according to the processing result corresponding to each of the image processing tasks and the label of the training image includes:

[0016] Using the image processing single task corresponding to the label of the training image as the first target image processing single task;

[0017] Calculating a first sub-loss value corresponding to the first target image processing task according to a processing result corresponding to the first target image processing task and a label of the training image;

[0018] For each of the image processing single tasks except the first target image processing single task, 0 is used as the first sub-loss value corresponding to the image processing single task.

[0019] In a possible implementation, calculating the loss value according to the labels of the images in the training data set and the first sub-loss value and the second sub-loss value corresponding to each of the image processing tasks includes:

[0020] If the label of the image in the training data set is the label of the second target image processing single task, the loss value is calculated according to the first sub-loss value corresponding to the second target image processing single task and the second sub-loss value corresponding to each of the image processing single tasks; or, the loss value is calculated according to the second sub-loss value corresponding to each of the image processing single tasks.

[0021] In a possible implementation, the method further includes:

[0022] If the labels of the images in the training data set include the labels of each of the single image processing tasks, the loss value is calculated according to the first sub-loss value and the second sub-loss value corresponding to each of the single image processing tasks, and the preset weight.

[0023] In a possible implementation, the performing model training on the initial multi-task image processing model according to the loss value and the training data set to obtain a target multi-task image processing model includes:

[0024] Updating the initial multi-task image processing model according to the loss value to obtain a trained model;

[0025] Using the validation data set to validate the trained model, and obtaining the average precision mean;

[0026] If the average precision mean is less than the preset threshold, the trained model is used as the new initial multi-task image processing model, and a new training image is selected from the training data set. A new loss value is obtained based on the new training image, each single-task image processing model and the new initial multi-task image processing model. The above steps are repeated until the new average precision mean is greater than or equal to the preset threshold, and the new trained model is used as the target multi-task image processing model.

[0027] In a possible implementation, the target multi-task image processing model includes a backbone network, a neck network, and a head network corresponding to each of the image processing single tasks. If the label of the image in the training data set is the label of the second target image processing single task, the method further includes:

[0028] The backbone network and the neck network are fixed, and according to the data set corresponding to each single image processing task, the head network corresponding to each single image processing task is trained to obtain an improved multi-task image processing model.

[0029] In a possible implementation, the initial multi-task image processing model includes a backbone network, a neck network, and a feature converter and a head network corresponding to each of the image processing single tasks;

[0030] Before inputting the training image into the initial multi-task image processing model to obtain a multi-task feature map and a processing result corresponding to each of the image processing single tasks, the method further includes:

[0031] Generating a mask image corresponding to each of the image processing tasks according to the training image and the image segmentation model;

[0032] The step of inputting the training image into the initial multi-task image processing model to obtain a multi-task feature map and a processing result corresponding to each of the image processing tasks includes:

[0033] Processing the training image according to the backbone network and the neck network to obtain the multi-task feature map;

[0034] For each of the single image processing tasks, performing feature magnitude conversion on the multi-task feature map according to a feature converter corresponding to the single image processing task to obtain a first feature map corresponding to the single image processing task;

[0035] For each of the single image processing tasks, performing mask processing on the first feature map corresponding to the single image processing task according to the mask map corresponding to the single image processing task, so as to obtain a second feature map corresponding to the single image processing task;

[0036] For each of the single image processing tasks, the second feature map corresponding to the single image processing task is processed according to the head network corresponding to the single image processing task to obtain a processing result corresponding to the single image processing task.

[0037] In a possible implementation, calculating the second sub-loss value corresponding to the single image processing task according to the feature map corresponding to the single image processing task and the multi-task feature map includes:

[0038] According to the mask map corresponding to the single image processing task, mask processing is performed on the feature map corresponding to the single image processing task to obtain a third feature map corresponding to the single image processing task;

[0039] A second sub-loss value corresponding to the single image processing task is calculated according to the third feature map and the second feature map corresponding to the single image processing task.

[0040] In a second aspect, an embodiment of the present application provides a multi-task image processing model training device, comprising:

[0041] A selection module, used to select a training image from a training data set, where each image in the training data set has a label of a single image processing task;

[0042] Processing modules for:

[0043] Input the training images into the single-task image processing models corresponding to each single image processing task, respectively, to obtain the feature maps corresponding to each single image processing task, and input the training images into the initial multi-task image processing model, to obtain the multi-task feature maps and the processing results corresponding to each single image processing task;

[0044] Calculating a loss value according to the feature map corresponding to each of the single image processing tasks, the multi-task feature map, and the processing result corresponding to each of the single image processing tasks;

[0045] A training module is used to perform model training on the initial multi-task image processing model according to the loss value and the training data set to obtain a target multi-task image processing model.

[0046] In a third aspect, an embodiment of the present application provides an electronic device, including:

[0047] Processor, memory, communication interface;

[0048] The memory is used to store executable instructions of the processor;

[0049] Wherein, the processor is configured to execute the multi-task image processing model training method described in any one of the first aspects by executing the executable instructions.

[0050] In a fourth aspect, an embodiment of the present application provides a readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the multi-task image processing model training method described in any one of the first aspects is implemented.

[0051] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, is used to implement the multi-task image processing model training method described in any one of the first aspects.

[0052] The multi-task image processing model training method, device, equipment, medium and product provided in the embodiment of the present application, by inputting the training images selected from the training data set into the single-task image processing model corresponding to each image processing single task, respectively, to obtain the feature map corresponding to each image processing single task; and also inputting it into the initial multi-task image processing model to obtain the multi-task feature map and the processing result corresponding to each image processing single task. Then, according to the feature map and processing result corresponding to each image processing single task, and after calculating the loss value of the multi-task feature map, the initial multi-task image processing model is trained according to the loss value and the training data set to obtain the target multi-task image processing model. In this scheme, each image in the training data set has a label of a single image processing task. These images are existing images and do not need to be processed, which reduces the training cost and the cost of making images, and also improves the training efficiency. In addition, the loss value is calculated using the feature map, which can improve the accuracy of the multi-task image processing model. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0054] Figure 1a A flowchart of a first embodiment of a multi-task image processing model training method provided in this application;

[0055] Figure 1b A schematic diagram of the architecture of the single-task image processing model provided for this application;

[0056] Figure 1c Schematic diagram 1 of the architecture of the multi-task image processing model provided for this application;

[0057] Figure 2 A flowchart of the second embodiment of the multi-task image processing model training method provided in this application;

[0058] Figure 3a Schematic diagram of the architecture of the multi-task image processing model provided for this application Figure 2 ;

[0059] Figure 3b A schematic diagram of a training image processed by an image segmentation model provided in this application;

[0060] Figure 4 A schematic diagram of the structure of an embodiment of a multi-task image processing model training device provided in this application;

[0061] Figure 5 A schematic diagram of the structure of an electronic device provided in this application.

[0062] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0063] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0064] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0065] With the development of science and technology, there is a corresponding single-task image processing model to achieve each single image processing task. For example, the obstacle detection model can detect the position and type of obstacles in the road image, the speed limit sign detection model can detect the position and type of speed limit signs in the road image, the lane line detection model can detect the position and type of lane lines in the road image, the image segmentation model processes the image to obtain the segmented image, the depth estimation model processes the image to obtain the depth map, etc.

[0066] When there are multiple image processing tasks to be processed, in order to improve processing efficiency, a multi-task image processing model can be used to process one image to obtain the processing results of multiple image processing tasks.

[0067] In the prior art, for the training of a multi-task image processing model, a new label is usually firstly annotated on an image with a label of a single image processing task, so as to obtain an image with labels corresponding to multiple image processing tasks, and then these images are used to train the initial deep learning model to obtain a multi-task image processing model. Since the image needs to be processed first, the training cost and the cost of producing the image are high.

[0068] In view of the problems existing in the prior art, the inventors found in the process of studying the multi-task image processing model training method that when training a single-task image processing model, images with labels of a single image processing task will be used. These images already exist, so directly using these images for multi-task image processing model training can improve the training efficiency. First, a training image is selected from the training data set. Each image in the training data set has a label of a single image processing task, and then combined with the single-task image processing model, the feature map corresponding to each image processing task is obtained; combined with the initial multi-task image processing model, the multi-task feature map and the processing results corresponding to each image processing task are obtained. After calculating the loss value, the initial multi-task image processing model is trained in combination with the training data set to obtain the target multi-task image processing model, which can not only improve the training efficiency, but also improve the accuracy of the target multi-task image processing model. Based on the above invention concept, the multi-task image processing model training scheme in this application is designed.

[0069] The executor of the multi-task image processing model training method in this application can be a computer, or a server, terminal device, etc. This application does not limit it. The following description will take a computer as an example.

[0070] Exemplarily, the application scenario of the multi-task image processing model training method provided in this application is illustrated below.

[0071] In this application scenario, the computer has trained the deep learning model according to the data set corresponding to each single image processing task, and obtained the single task image processing model corresponding to each single image processing task. The images in the data set corresponding to a single image processing task all have the label of the single image processing task.

[0072] In order to complete multiple image processing tasks using one task to improve processing efficiency, it is necessary to train a multi-task image processing model.

[0073] First determine the training data set. The training data set can be a data set corresponding to a single image processing task or a collection of data sets corresponding to all single image processing tasks. Each image in the training data set has only one label for a single image processing task.

[0074] After the computer selects a training image from the training data set, the training image is input into the single-task image processing model corresponding to each single image processing task to obtain the feature map corresponding to each single image processing task. The training image is input into the initial multi-task image processing model to obtain the multi-task feature map and the processing result corresponding to each single image processing task. Among them, the single-task image processing model corresponding to the single image processing task is a trained model obtained by model training using images with labels of the single image processing task.

[0075] Then, the loss value is calculated based on the feature map corresponding to each single image processing task, the multi-task feature map, and the processing result corresponding to each single image processing task.

[0076] Then, the initial multi-task image processing model is trained according to the loss value and the training data set to obtain the target multi-task image processing model.

[0077] The computer can then input an image into the target multi-task image processing model to obtain the processing results corresponding to each single image processing task.

[0078] It should be noted that the above scenario is only an example of an application scenario provided by an embodiment of the present application. The embodiment of the present application does not limit the actual form of the various devices included in the scenario to the specific application of the solution and can be set according to actual needs.

[0079] The technical solution of the present application is described in detail below through specific embodiments. It should be noted that the following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0080] Figure 1a The flowchart of the first embodiment of the multi-task image processing model training method provided by the present application is as follows. The present application embodiment describes the situation in which a computer performs model training on an initial multi-task image processing model to obtain a target multi-task image processing model based on a training data set consisting of images with only one label of a single image processing task and a single-task image processing model corresponding to each single image processing task. The method in this embodiment can be implemented by software, hardware, or a combination of software and hardware. Figure 1a As shown, the multi-task image processing model training method specifically includes the following steps:

[0081] S101: Select a training image from a training data set.

[0082] In this step, in order to improve the training efficiency, the images in the training data set can be used for training. Each image in the training data set has only one label for a single image processing task, and the image processing tasks corresponding to the labels of different images can be the same or different.

[0083] First, select a training image from the training dataset.

[0084] It should be noted that the training data set may be a data set corresponding to a single image processing task, or may be a collection of data sets corresponding to each single image processing task. The images in the data set corresponding to each single image processing task have the label of the single image processing task.

[0085] S102: Input the training images into the single-task image processing models corresponding to each single image processing task, obtain the feature map corresponding to each single image processing task, input the training images into the initial multi-task image processing model, obtain the multi-task feature map and the processing results corresponding to each single image processing task.

[0086] In this step, after the computer acquires the training image, the training image is input into the single-task image processing model corresponding to each single image processing task to obtain the feature map corresponding to each single image processing task, and the training image is input into the initial multi-task image processing model to obtain the multi-task feature map and the processing result corresponding to each single image processing task.

[0087] For example, Figure 1b The schematic diagram of the architecture of the single-task image processing model provided for this application is as follows: Figure 1b As shown in the figure, the single-task image processing model includes a backbone network, a neck network, and a head network. The backbone network extracts features from the training image to obtain image features, and then the neck network enhances the image features to obtain feature maps. The head network processes the feature maps to obtain the processing results of the single-task image processing model.

[0088] For example, Figure 1c The first schematic diagram of the architecture of the multi-task image processing model provided in this application can be the architecture of the initial multi-task image processing model and the target multi-task image processing model. Figure 1c As shown in the figure, the multi-task image processing model includes a backbone network, a neck network, and a head network corresponding to each image processing task. The backbone network extracts features from the training image to obtain image features, and then the neck network enhances the image features to obtain a multi-task feature map. The head network corresponding to each image processing task processes the multi-task feature map to obtain the processing result corresponding to each image processing task.

[0089] It should be noted that the method of extracting features from training images may be: performing at least one of convolution, pooling, normalization, nonlinear transformation, etc. on the training images. The embodiment of the present application does not limit the method of extracting features from training images, which can be determined according to actual conditions.

[0090] It should be noted that the method of enhancing the image features may be to enhance the image features using at least one of feature pyramid, spatial pyramid pooling, attention mechanism, etc. The embodiment of the present application does not limit the method of enhancing the image features, which can be determined according to actual conditions.

[0091] It should be noted that the feature map and the multi-task feature map can be represented by a feature matrix.

[0092] S103: Calculate the loss value according to the feature map corresponding to each single image processing task, the multi-task feature map and the processing result corresponding to each single image processing task.

[0093] In this step, after the computer obtains the feature map corresponding to each single image processing task, the multi-task feature map and the processing result corresponding to each single image processing task, it calculates the loss value based on these data.

[0094] Since there is only one backbone network and neck network in the multi-task image processing model, the obtained multi-task feature map must have the features corresponding to each image processing task at the same time to meet the needs of multiple head networks. Therefore, the second sub-loss value is calculated based on the feature map and the multi-task feature map.

[0095] Since the multi-task processing model needs to meet the requirements of multiple image processing tasks, the first sub-loss value is determined based on the processing results and the labels of the training images.

[0096] Then, the loss value is calculated based on the first sub-loss value and the second sub-loss value.

[0097] S104: Perform model training on the initial multi-task image processing model according to the loss value and the training data set to obtain a target multi-task image processing model.

[0098] In this step, after the computer obtains the loss value, it can perform model training on the initial multi-task image processing model according to the loss value and the training data set to obtain the target multi-task image processing model.

[0099] Specifically, the initial multi-task image processing model is updated according to the loss value to obtain a trained model.

[0100] The trained model is validated using the validation data set to obtain the mean average precision (mAP).

[0101] If the average precision mean is less than the preset threshold, the trained model is used as the new initial multi-task image processing model, and a new training image is selected from the training data set. A new loss value is obtained based on the new training image, each single-task image processing model and the new initial multi-task image processing model. The new initial multi-task image processing model is updated according to the new loss value to obtain a new trained model. The validation data set is used again to verify the new trained model to obtain a new average precision mean. If the new average precision mean is less than the preset threshold, this process is repeated until the new average precision mean is greater than or equal to the preset threshold, and the new trained model is used as the target multi-task image processing model.

[0102] It should be noted that the preset threshold may be 0.4, 0.5, 0.6, etc. The embodiment of the present application does not limit the preset threshold, which may be determined according to actual conditions.

[0103] The multi-task image processing model training method provided in this embodiment is to obtain the feature map corresponding to each image processing single task by inputting the training images selected from the training data set into the single-task image processing model corresponding to each image processing single task respectively; and inputting them into the initial multi-task image processing model to obtain the multi-task feature map and the processing result corresponding to each image processing single task. Then, according to the feature map and processing result corresponding to each image processing single task, and after calculating the loss value of the multi-task feature map, the initial multi-task image processing model is trained according to the loss value and the training data set to obtain the target multi-task image processing model. Compared with the prior art that requires the image to be labeled with multiple processing single task labels, each image in the training data set in this solution has a label of a single image processing task. These images are existing images and do not need to be processed, which reduces the training cost and the cost of making images, and also improves the training efficiency. In addition, the use of feature maps to calculate the loss value can improve the accuracy of the multi-task image processing model.

[0104] In addition, the image formats used in the prior art for training are the same, but the image formats used in this solution can be different. The image format includes the image shooting type, the parameters of the image shooting, etc. The image shooting type includes the single-camera shooting type and the multi-camera shooting type. The multi-camera shooting type image is an image formed by combining images shot by multiple cameras, and the single-camera shooting type image is an image shot by a single camera.

[0105] Figure 2This is a flow chart of the second embodiment of the multi-task image processing model training method provided by this application. Based on the above embodiment, this embodiment of the application describes the situation where the computer calculates the loss value according to the feature map corresponding to each image processing task, the multi-task feature map and the processing result corresponding to each image processing task. Figure 2 As shown, the multi-task image processing model training method specifically includes the following steps:

[0106] S201: Determine a first sub-loss value corresponding to each image processing task according to a processing result corresponding to each image processing task and a label of a training image.

[0107] In this step, after the computer obtains the feature map corresponding to each single image processing task, the multi-task feature map and the processing results corresponding to each single image processing task, it determines the first sub-loss value corresponding to each single image processing task according to the processing results corresponding to each single image processing task and the label of the training image.

[0108] Specifically, the image processing single task corresponding to the label of the training image is used as the first target image processing single task.

[0109] According to the processing result corresponding to the first target image processing single task and the label of the training image, the first sub-loss value corresponding to the first target image processing single task is calculated. Since the label of the training image is the label corresponding to the first target image processing single task, the first sub-loss value can be calculated according to the label.

[0110] In one implementation, the processing result and the label both include three parts, namely, the location of the target, the category, and the detection mark of whether the target is real. Therefore, the location loss can be calculated based on the location of the target in the processing result and the label; the category loss can be calculated based on the category of the target in the processing result and the label; the detection loss can be calculated based on the detection mark in the processing result and the label; and then according to the formula , calculate the first sub-loss value, where, represents the first sub-loss value, represents the preset position weight, represents the preset category weight, Indicates the preset detection weight, represents the position loss, represents the category loss, Represents the detection loss.

[0111] It should be noted that the calculation method of position loss, category loss and detection loss can be calculated using loss functions such as cross entropy loss function, mean square error loss function, L1 norm loss function, IoU loss function, etc. The preset position weight, preset category weight and preset detection weight can be 1, 2, 3, etc. The embodiment of the present application does not limit the calculation method of position loss, category loss and detection loss, as well as the preset position weight, preset category weight and preset detection weight, which can be determined according to actual conditions.

[0112] For each image processing task other than the first target image processing task, 0 is used as the first sub-loss value corresponding to the image processing task. Since there is no label corresponding to each image processing task other than the first target image processing task in the training image, the loss value cannot be calculated based on the label, and 0 is used as the first sub-loss value corresponding to the image processing task.

[0113] S202: For each single image processing task, calculate a second sub-loss value corresponding to the single image processing task according to the feature map corresponding to the single image processing task and the multi-task feature map.

[0114] In this step, after the computer obtains the feature map corresponding to each single image processing task, the multi-task feature map and the processing result corresponding to each single image processing task, for each single image processing task, the second sub-loss value corresponding to the single image processing task is calculated based on the feature map corresponding to the single image processing task and the multi-task feature map.

[0115] The method for calculating the second sub-loss value can be to use a loss function such as a cross entropy loss function, a mean square error loss function, or an L1 norm loss function. The embodiment of the present application does not limit the method for calculating the second sub-loss value, and it can be determined according to actual conditions.

[0116] It should be noted that the execution order of step S201 and step S202 can be: first execute step S201, then execute step S202; or: first execute step S202, then execute step S201; or: execute step S201 and step S202 simultaneously. The embodiment of the present application does not limit the execution order of step S201 and step S202, which can be determined according to actual conditions.

[0117] S203: Calculate the loss value according to the labels of the images in the training data set, and the first sub-loss value and the second sub-loss value corresponding to each image processing task.

[0118] In this step, after the computer obtains the first sub-loss value and the second sub-loss value corresponding to each image processing task, it calculates the loss value based on the labels of the images in the training data set.

[0119] Specifically, if the label of the image in the training data set is the label of the second target image processing single task, that is, the training data set is a data set corresponding to a single image processing task, the single image processing task serves as the second target image processing single task.

[0120] In one implementation, according to the first sub-loss value corresponding to the second target image processing single task and the second sub-loss value corresponding to each image processing single task, combined with the formula , calculate the loss value. Among them, represents the loss value, N represents the number of single image processing tasks, Represents the preset weight of the second sub-loss value corresponding to the i-th image processing single task, represents the second sub-loss value corresponding to the i-th image processing single task, represents the preset weight of the first sub-loss value corresponding to the second target image processing single task, Represents the first sub-loss value corresponding to the second target image processing task.

[0121] In another implementation, according to the second sub-loss value corresponding to each image processing task, combined with the formula , calculate the loss value. Among them, represents the loss value, N represents the number of single image processing tasks, Represents the preset weight of the second sub-loss value corresponding to the i-th image processing single task, Represents the second sub-loss value corresponding to the i-th image processing task.

[0122] It should be noted that the preset weight of the second sub-loss value corresponding to the single image processing task and the preset weight of the first sub-loss value corresponding to the second target image processing task can be 1, 2, 3, etc. The embodiment of the present application does not limit the preset weights and can be determined according to actual conditions.

[0123] If the labels of the images in the training data set include the labels of each single image processing task, that is, the training data set is the collection of data sets corresponding to each single image processing task, then according to the first sub-loss value and the second sub-loss value corresponding to each single image processing task, and the preset weight, according to the formula , calculate the loss value.

[0124] in, represents the loss value, N represents the number of single image processing tasks, Represents the preset weight of the second sub-loss value corresponding to the i-th image processing single task, represents the second sub-loss value corresponding to the i-th image processing single task, Represents the preset weight of the first sub-loss value corresponding to the i-th image processing single task, Represents the first sub-loss value corresponding to the i-th image processing task.

[0125] It should be noted that the preset weights include the preset weight of the second sub-loss value corresponding to the single image processing task and the preset weight of the first sub-loss value corresponding to the single image processing task. The two weights can be 1, 2, 3, etc. The embodiment of the present application does not limit the preset weights and can be determined according to actual conditions.

[0126] The multi-task image processing model training method provided in this embodiment can improve the accuracy of the loss value and thus improve the accuracy of the target multi-task image processing model by calculating the loss value based on the feature map, the multi-task feature map, the processing results, and the labels of the training images.

[0127] Based on the above embodiments, the following is a third embodiment of the multi-task image processing model training method provided by the present application. After the computer obtains the target multi-task image processing model, the target multi-task image processing model is improved according to the data set corresponding to each single image processing task.

[0128] When the label of the image in the training data set is the label of the second target image processing single task, that is, the training data set is a data set corresponding to a single image processing task. During the training process, except that the first sub-loss value corresponding to the second target image processing single task is not 0, the first sub-loss values ​​corresponding to other image processing single tasks are all 0. Therefore, after the training of the target multi-task image processing model is completed, the backbone network and the neck network are both trained. For the head network, at most only the head network corresponding to the second target image processing single task is trained, and the head networks corresponding to other image processing single tasks are not trained.

[0129] Therefore, in order to further improve the accuracy of the multi-task image processing model, the backbone network and the neck network can be fixed, and the head network corresponding to each single image processing task can be trained according to the data set corresponding to each single image processing task to obtain an improved multi-task image processing model.

[0130] Fixing the backbone network and the neck network means that during the model training of the head network, the backbone network and the neck network are not updated.

[0131] For each single image processing task, according to the data set corresponding to the single image processing task, the process of model training of the head network corresponding to the single image processing task is as follows:

[0132] An image is selected from the data set corresponding to the single image processing task, and the image is sequentially input into the backbone network and the neck network of the target multi-task image processing model to obtain a feature map.

[0133] The feature map is then input into the head network corresponding to the single image processing task to obtain the single task processing result.

[0134] After calculating the single-task loss value based on the single-task processing result and the label of the image, the head network corresponding to the image processing single task is updated according to the single-task loss value to obtain a new target multi-task image processing model. The backbone network and neck network are not updated at this time.

[0135] The validation data set corresponding to the single image processing task is used to validate the new target multi-task image processing model and obtain the average precision mean.

[0136] If the average precision mean is less than the preset threshold, the above process is repeated until the new average precision mean is greater than or equal to the preset threshold, and the new target multi-task image processing model is used as the improved multi-task image processing model.

[0137] It should be noted that if during the training process of the target multi-task image processing model, the loss value is calculated based on the first sub-loss value corresponding to the second target image processing single task and the second sub-loss value corresponding to each image processing single task, it means that the head network corresponding to the second target image processing single task has been trained, and the head network no longer needs to be trained. Only the head networks corresponding to other image processing single tasks need to be trained based on the data sets corresponding to other image processing single tasks.

[0138] The multi-task image processing model training method provided in this embodiment can improve the accuracy of the multi-task image processing model by performing model training on the head network corresponding to each single image processing task according to the data set corresponding to each single image processing task.

[0139] Based on the above embodiments, the following describes the situation in which the model training is performed in combination with the image segmentation model when the multi-task image processing model also includes a feature converter corresponding to each single image processing task through the fourth embodiment of the multi-task image processing model training method provided by the present application.

[0140] For example, Figure 3a Schematic diagram of the architecture of the multi-task image processing model provided for this application Figure 2 , the architecture can be the architecture of the initial multi-task image processing model and the target multi-task image processing model. Figure 3a As shown, the multi-task image processing model includes a backbone network, a neck network, and a feature converter and a head network corresponding to each single image processing task.

[0141] After the computer selects a training image from the training data set, the training image is input into the single-task image processing model corresponding to each image processing task to obtain the feature map corresponding to each image processing task.

[0142] According to the training image and the image segmentation model, a mask image corresponding to each single image processing task is generated. That is, the training image is input into the image segmentation model to obtain multiple sub-images. Each sub-image has a corresponding detection type and image position. Different single image processing tasks correspond to different detection types. Therefore, the sub-image corresponding to each single image processing task can be determined according to the correspondence between the single image processing task and the detection type. Then, for each single image processing task, a mask image corresponding to the single image processing task is generated according to the sub-image corresponding to the single image processing task and the position of the sub-image. The value corresponding to the image position in the mask image is 1, and the values ​​corresponding to the remaining positions are 0.

[0143] It should be noted that the image segmentation model is a trained model obtained by performing model training based on a data set.

[0144] For example, Figure 3b A schematic diagram of a training image processed by an image segmentation model provided in this application, such as Figure 3b As shown, the part within the dotted box in the figure is the sub-image.

[0145] For example, there are five sub-images, namely A, B, C, D, and E. The detection type of sub-image A is car, the detection type of sub-image B is bus, the detection type of sub-image C is motorcycle, the detection type of sub-image D is traffic sign, and the detection type of sub-image E is traffic sign. The detection types corresponding to image processing task A include car, bus, and motorcycle, and the detection types corresponding to image processing task B include traffic sign. Therefore, sub-images corresponding to image processing task A are A, B, and C, and sub-images corresponding to image processing task B are D and E.

[0146] It should be noted that the size of the mask image corresponding to the single image processing task needs to be adjusted to the size of the feature map corresponding to the single image processing task. The size of the feature map corresponding to each single image processing task is the same as the size of the multi-task feature map.

[0147] Then, the training images are processed according to the backbone network and the neck network to obtain a multi-task feature map.

[0148] For each single image processing task, the feature magnitude conversion is performed on the multi-task feature map according to the feature converter corresponding to the single image processing task to obtain the first feature map corresponding to the single image processing task. Since the magnitudes of the feature maps corresponding to different single image processing tasks may vary greatly, and there is only one multi-task feature map in a model training process, directly using the multi-task feature map and the feature maps corresponding to each single image processing task to calculate the loss value may cause the loss value to be inaccurate, so it is necessary to use the feature converter to convert the feature magnitude of the multi-task feature map.

[0149] It should be noted that the feature converter can be a 1*1 convolutional network, which performs a linear transformation on the multi-task feature map to adjust the magnitude of the multi-task feature map so that the magnitude of the first feature map corresponding to each single image processing task is close.

[0150] For each single image processing task, the first feature map corresponding to the single image processing task is masked according to the mask map corresponding to the single image processing task to obtain the second feature map corresponding to the single image processing task. Using the mask map to mask the first feature map can highlight the processing target corresponding to the single image processing task, so as to improve the accuracy of the loss value calculated subsequently and alleviate the problem of unbalanced detection effect.

[0151] The process of mask processing is: generate a blank second feature map of the same size as the first feature map, and the first feature map is the same size as the mask map; then for each position of the blank second feature map, if the position in the mask map is 1, the value of the position in the first feature map is used as the value of the position in the second feature map; if the position in the mask map is 0, the value of the position in the second feature map is 0.

[0152] It should be noted that the multi-task feature map can be masked and then processed using a feature converter to obtain a second feature map.

[0153] For each single image processing task, the second feature map corresponding to the single image processing task is processed according to the head network corresponding to the single image processing task, that is, the second feature map corresponding to the single image processing task is input into the head network corresponding to the single image processing task to obtain the processing result corresponding to the single image processing task.

[0154] Then, according to the processing results corresponding to each single image processing task and the labels of the training images, the first sub-loss value corresponding to each single image processing task is determined; for each single image processing task, according to the feature map and the third feature map corresponding to the single image processing task, the second sub-loss value corresponding to the single image processing task is calculated; according to the labels of the images in the training data set, and the first sub-loss value and the second sub-loss value corresponding to each single image processing task, the loss value is calculated.

[0155] The initial multi-task image processing model is updated according to the loss value to obtain a trained model; the trained model is verified using a verification data set to obtain an average precision mean.

[0156] It should be noted that during the updating of the initial multi-task image processing model, the feature converter corresponding to each single image processing task will also be updated.

[0157] If the average precision mean is less than the preset threshold, repeat the above steps until the new average precision mean is greater than or equal to the preset threshold, and use the new trained model as the target multi-task image processing model.

[0158] It should be noted that the method for generating the second sub-loss value corresponding to a single image processing task can be: according to the mask map corresponding to the single image processing task, mask processing is performed on the feature map corresponding to the single image processing task to obtain the third feature map corresponding to the single image processing task; and then, according to the third feature map and the second feature map corresponding to the single image processing task, the second sub-loss value corresponding to the single image processing task is calculated.

[0159] It should be noted that in the subsequent process of improving the target multi-task image processing model, a single-task mask map can be generated based on the image and image segmentation model. After obtaining the feature map, the feature map can be processed using the feature converter and the single-task mask map, and then the processed feature map is input into the head network corresponding to the single image processing task to obtain the single-task processing result.

[0160] Exemplarily, Table 1 is a comparison table of detection effect indicators of the single-task image processing model and the target multi-task image processing model provided by this application.

[0161] Table 1

[0162]

[0163] As shown in Table 1, the single-task image processing model is an obstacle detection model. The mAP and NDS (nuScenes Detection Score) values ​​of the single-task image processing model and the target multi-task image processing model are large and close, indicating that the target multi-task image processing model has higher accuracy, and the detection accuracy of the target multi-task image processing model in detecting obstacles is close to that of the single-task image processing model.

[0164] Exemplarily, Table 2 is a comparison table 2 of processing effect indicators of the single-task image processing model and the target multi-task image processing model provided by this application.

[0165] Table 2

[0166]

[0167] As shown in Table 2, the single-task image processing model is a speed limit sign detection model. The mAP values ​​of the single-task image processing model and the target multi-task image processing model are large and close, indicating that the accuracy of the target multi-task image processing model is higher, and the detection accuracy of the target multi-task image processing model in detecting speed limit signs is close to that of the single-task image processing model.

[0168] The multi-task image processing model training method provided in this embodiment can further improve the accuracy of the target multi-task image processing model by performing model training through an image segmentation model, a feature converter, and a mask image.

[0169] The following is an embodiment of the device of the present application, which can be used to execute the embodiment of the method of the present application. For details not disclosed in the embodiment of the device of the present application, please refer to the embodiment of the method of the present application.

[0170] Figure 4 This is a schematic diagram of the structure of an embodiment of a multi-task image processing model training device provided in this application. Figure 4 As shown, the multi-task image processing model training device 40 includes:

[0171] A selection module 41 is used to select a training image from a training data set, where each image in the training data set has a label of a single image processing task;

[0172] The processing module 42 is used to:

[0173] Input the training images into the single-task image processing models corresponding to each single image processing task, respectively, to obtain the feature maps corresponding to each single image processing task, and input the training images into the initial multi-task image processing model, to obtain the multi-task feature maps and the processing results corresponding to each single image processing task;

[0174] Calculating a loss value according to the feature map corresponding to each of the single image processing tasks, the multi-task feature map, and the processing result corresponding to each of the single image processing tasks;

[0175] The training module 43 is used to perform model training on the initial multi-task image processing model according to the loss value and the training data set to obtain a target multi-task image processing model.

[0176] Furthermore, the processing module 42 is specifically configured to:

[0177] Determine a first sub-loss value corresponding to each of the image processing single tasks according to the processing result corresponding to each of the image processing single tasks and the label of the training image;

[0178] For each of the single image processing tasks, calculating a second sub-loss value corresponding to the single image processing task according to the feature map corresponding to the single image processing task and the multi-task feature map;

[0179] The loss value is calculated according to the labels of the images in the training data set, and the first sub-loss value and the second sub-loss value corresponding to each of the image processing tasks.

[0180] Furthermore, the processing module 42 is further configured to:

[0181] Using the image processing single task corresponding to the label of the training image as the first target image processing single task;

[0182] Calculating a first sub-loss value corresponding to the first target image processing task according to a processing result corresponding to the first target image processing task and a label of the training image;

[0183] For each of the image processing single tasks except the first target image processing single task, 0 is used as the first sub-loss value corresponding to the image processing single task.

[0184] Furthermore, the processing module 42 is further configured to:

[0185] If the label of the image in the training data set is the label of the second target image processing single task, the loss value is calculated according to the first sub-loss value corresponding to the second target image processing single task and the second sub-loss value corresponding to each of the image processing single tasks; or, the loss value is calculated according to the second sub-loss value corresponding to each of the image processing single tasks.

[0186] Furthermore, the processing module 42 is further configured to:

[0187] If the labels of the images in the training data set include the labels of each of the single image processing tasks, the loss value is calculated according to the first sub-loss value and the second sub-loss value corresponding to each of the single image processing tasks, and the preset weight.

[0188] Furthermore, the training module 43 is specifically used for:

[0189] Updating the initial multi-task image processing model according to the loss value to obtain a trained model;

[0190] Using the validation data set to validate the trained model, and obtaining the average precision mean;

[0191] If the average precision mean is less than the preset threshold, the trained model is used as the new initial multi-task image processing model, and a new training image is selected from the training data set. A new loss value is obtained based on the new training image, each single-task image processing model and the new initial multi-task image processing model. The above steps are repeated until the new average precision mean is greater than or equal to the preset threshold, and the new trained model is used as the target multi-task image processing model.

[0192] Furthermore, the target multi-task image processing model includes a backbone network, a neck network, and a head network corresponding to each of the image processing single tasks. If the label of the image in the training data set is the label of the second target image processing single task, the training module 43 is further used to:

[0193] The backbone network and the neck network are fixed, and according to the data set corresponding to each single image processing task, the head network corresponding to each single image processing task is trained to obtain an improved multi-task image processing model.

[0194] Furthermore, the initial multi-task image processing model includes a backbone network, a neck network, and a feature converter and a head network corresponding to each of the image processing single tasks; before inputting the training image into the initial multi-task image processing model to obtain a multi-task feature map and a processing result corresponding to each of the image processing single tasks, the processing module 42 is further used to:

[0195] Generating a mask image corresponding to each of the image processing tasks according to the training image and the image segmentation model;

[0196] The processing module 42 is further specifically used for:

[0197] Processing the training image according to the backbone network and the neck network to obtain the multi-task feature map;

[0198] For each of the single image processing tasks, performing feature magnitude conversion on the multi-task feature map according to a feature converter corresponding to the single image processing task to obtain a first feature map corresponding to the single image processing task;

[0199] For each of the single image processing tasks, performing mask processing on the first feature map corresponding to the single image processing task according to the mask map corresponding to the single image processing task, so as to obtain a second feature map corresponding to the single image processing task;

[0200] For each of the single image processing tasks, the second feature map corresponding to the single image processing task is processed according to the head network corresponding to the single image processing task to obtain a processing result corresponding to the single image processing task.

[0201] Furthermore, the processing module 42 is further configured to:

[0202] According to the mask map corresponding to the single image processing task, mask processing is performed on the feature map corresponding to the single image processing task to obtain a third feature map corresponding to the single image processing task;

[0203] A second sub-loss value corresponding to the single image processing task is calculated according to the third feature map and the second feature map corresponding to the single image processing task.

[0204] The multi-task image processing model training device provided in this embodiment is used to execute the technical solution in any of the aforementioned method embodiments. Its implementation principle and technical effects are similar and will not be repeated here.

[0205] Figure 5 This is a schematic diagram of the structure of an electronic device provided in this application. Figure 5 As shown, the electronic device 50 includes:

[0206] Processor 51, memory 52, and communication interface 53;

[0207] The memory 52 is used to store executable instructions of the processor 51;

[0208] The processor 51 is configured to execute the technical solution in any of the aforementioned method embodiments by executing the executable instructions.

[0209] Optionally, the memory 52 may be independent or integrated with the processor 51 .

[0210] Optionally, when the memory 52 is a device independent of the processor 51, the electronic device 50 may further include:

[0211] The bus 54 , the memory 52 and the communication interface 53 are connected to the processor 51 via the bus 54 and communicate with each other. The communication interface 53 is used to communicate with other devices.

[0212] Optionally, the communication interface 53 may be implemented by a transceiver. The communication interface is used to implement communication between the database access device and other devices (such as a client, a read-write library, and a read-only library). The memory may include a random access memory (RAM) and may also include a non-volatile memory (non-volatile memory), such as at least one disk storage.

[0213] The bus 54 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0214] The above-mentioned processor can be a general-purpose processor, including a central processing unit CPU, a network processor (NP), etc.; it can also be a digital signal processor DSP, an application-specific integrated circuit ASIC, a field programmable gate array FPGA or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0215] The electronic device is used to execute the technical solution in any of the aforementioned method embodiments, and its implementation principle and technical effect are similar and will not be repeated here.

[0216] An embodiment of the present application also provides a computer program product, including a computer program, which is used to implement the technical solution provided by any of the aforementioned method embodiments when executed by a processor.

[0217] An embodiment of the present application also provides a readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the technical solution provided by any of the aforementioned method embodiments is implemented.

[0218] The above-mentioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The readable storage medium can be any available medium that can be accessed by a general or special-purpose computer.

[0219] An exemplary readable storage medium is coupled to a processor so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (Application Specific Integrated Circuits, referred to as: ASIC). Of course, the processor and the readable storage medium can also exist in the device as discrete components.

[0220] The division of units is only a logical function division, and there may be other divisions in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interface, device or unit, which can be electrical, mechanical or other forms.

[0221] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0222] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0223] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0224] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk and other media that can store program codes.

[0225] Finally, it should be noted that those skilled in the art will readily conceive of other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. The present invention is intended to cover any variations, uses or adaptations of the present invention, which follow the general principles of the present invention and include common knowledge or customary technical means in the art not disclosed by the present invention, are not limited to the precise structure described above and shown in the drawings, and may be modified and changed in various ways without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.

Claims

1. A multi-task image processing model training method, characterized in that: include: Selecting a training image from a training data set, wherein each image in the training data set has a label of a single image processing task; Input the training images into the single-task image processing models corresponding to each single image processing task, respectively, to obtain the feature maps corresponding to each single image processing task, and input the training images into the initial multi-task image processing model, to obtain the multi-task feature maps and the processing results corresponding to each single image processing task; Calculating a loss value according to the feature map corresponding to each of the single image processing tasks, the multi-task feature map, and the processing result corresponding to each of the single image processing tasks; The initial multi-task image processing model is trained according to the loss value and the training data set to obtain a target multi-task image processing model.

2. The method according to claim 1, characterized in that The calculating the loss value according to the feature map corresponding to each of the single image processing tasks, the multi-task feature map and the processing result corresponding to each of the single image processing tasks includes: Determine a first sub-loss value corresponding to each of the image processing single tasks according to the processing result corresponding to each of the image processing single tasks and the label of the training image; For each of the single image processing tasks, calculating a second sub-loss value corresponding to the single image processing task according to the feature map corresponding to the single image processing task and the multi-task feature map; The loss value is calculated according to the labels of the images in the training data set, and the first sub-loss value and the second sub-loss value corresponding to each of the image processing tasks.

3. The method according to claim 2, characterized in that Determining the first sub-loss value corresponding to each of the image processing single tasks according to the processing result corresponding to each of the image processing single tasks and the label of the training image includes: Using the image processing single task corresponding to the label of the training image as the first target image processing single task; Calculating a first sub-loss value corresponding to the first target image processing task according to a processing result corresponding to the first target image processing task and a label of the training image; For each of the image processing single tasks except the first target image processing single task, 0 is used as the first sub-loss value corresponding to the image processing single task.

4. The method according to claim 2, characterized in that: The calculating the loss value according to the labels of the images in the training data set and the first sub-loss value and the second sub-loss value corresponding to each of the image processing tasks includes: If the label of the image in the training data set is the label of the second target image processing single task, the loss value is calculated according to the first sub-loss value corresponding to the second target image processing single task and the second sub-loss value corresponding to each of the image processing single tasks; or, the loss value is calculated according to the second sub-loss value corresponding to each of the image processing single tasks.

5. The method according to claim 4, characterized in that The method further comprises: If the labels of the images in the training data set include the labels of each of the single image processing tasks, the loss value is calculated according to the first sub-loss value and the second sub-loss value corresponding to each of the single image processing tasks, and the preset weight.

6. The method according to claim 1, characterized in that The performing model training on the initial multi-task image processing model according to the loss value and the training data set to obtain a target multi-task image processing model includes: Updating the initial multi-task image processing model according to the loss value to obtain a trained model; Using the validation data set to validate the trained model, and obtaining the average precision mean; If the average precision mean is less than the preset threshold, the trained model is used as the new initial multi-task image processing model, and a new training image is selected from the training data set. A new loss value is obtained based on the new training image, each single-task image processing model and the new initial multi-task image processing model. The above steps are repeated until the new average precision mean is greater than or equal to the preset threshold, and the new trained model is used as the target multi-task image processing model.

7. The method according to any one of claims 1 to 6, characterized in that: The target multi-task image processing model includes a backbone network, a neck network, and a head network corresponding to each of the image processing single tasks. If the label of the image in the training data set is the label of the second target image processing single task, the method further includes: The backbone network and the neck network are fixed, and according to the data set corresponding to each single image processing task, the head network corresponding to each single image processing task is trained to obtain an improved multi-task image processing model.

8. The method according to claim 2, characterized in that: The initial multi-task image processing model includes a backbone network, a neck network, and a feature converter and a head network corresponding to each single image processing task; Before inputting the training image into the initial multi-task image processing model to obtain a multi-task feature map and a processing result corresponding to each of the image processing single tasks, the method further includes: Generating a mask image corresponding to each of the image processing tasks according to the training image and the image segmentation model; The step of inputting the training image into the initial multi-task image processing model to obtain a multi-task feature map and a processing result corresponding to each of the image processing tasks includes: Processing the training image according to the backbone network and the neck network to obtain the multi-task feature map; For each of the single image processing tasks, performing feature magnitude conversion on the multi-task feature map according to a feature converter corresponding to the single image processing task to obtain a first feature map corresponding to the single image processing task; For each of the single image processing tasks, performing mask processing on the first feature map corresponding to the single image processing task according to the mask map corresponding to the single image processing task, so as to obtain a second feature map corresponding to the single image processing task; For each of the single image processing tasks, the second feature map corresponding to the single image processing task is processed according to the head network corresponding to the single image processing task to obtain a processing result corresponding to the single image processing task.

9. The method according to claim 8, characterized in that The calculating, according to the feature map corresponding to the single image processing task and the multi-task feature map, a second sub-loss value corresponding to the single image processing task comprises: According to the mask map corresponding to the single image processing task, mask processing is performed on the feature map corresponding to the single image processing task to obtain a third feature map corresponding to the single image processing task; A second sub-loss value corresponding to the single image processing task is calculated according to the third feature map and the second feature map corresponding to the single image processing task.

10. A multi-task image processing model training device, characterized in that: include: A selection module, used to select a training image from a training data set, where each image in the training data set has a label of a single image processing task; Processing modules for: Input the training images into the single-task image processing models corresponding to each single image processing task, respectively, to obtain the feature maps corresponding to each single image processing task, and input the training images into the initial multi-task image processing model, to obtain the multi-task feature maps and the processing results corresponding to each single image processing task; Calculating a loss value according to the feature map corresponding to each of the single image processing tasks, the multi-task feature map, and the processing result corresponding to each of the single image processing tasks; A training module is used to perform model training on the initial multi-task image processing model according to the loss value and the training data set to obtain a target multi-task image processing model.

11. An electronic device, characterized in that: include: Processor, memory, communication interface; The memory is used to store executable instructions of the processor; Wherein, the processor is configured to execute the multi-task image processing model training method described in any one of claims 1 to 9 by executing the executable instructions.

12. A readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the multi-task image processing model training method described in any one of claims 1 to 9 is implemented.

13. A computer program product, characterized in that It includes a computer program, which, when executed by a processor, is used to implement the multi-task image processing model training method described in any one of claims 1 to 9.

Images