A method and device for obtaining a multi-loss model

Abstract

The embodiment of the present application relates to the field of artificial intelligence, and provides a method and device for acquiring a multi-loss model, which is used to dynamically adjust the loss weight value during the multi-loss model training process, so that the multi-loss model can obtain better training results. During the multi-loss model training process, in the first iteration cycle, the first loss value output by the sub-network is obtained; then the candidate weight is generated according to the first loss value; the first loss value and the candidate weight are input into the weight prediction model, and the first weight parameter of the sub-network is output, wherein the first weight parameter is used for the training of the second iteration cycle in the multi-loss model training process, and the second iteration cycle is the next iteration cycle of the first iteration cycle; finally, the parameters of the multi-loss model are updated according to the first weight parameter.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence, and in particular to a method and device for acquiring a multi-loss model. Background Art

[0002] Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a branch of computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making. Research in the field of artificial intelligence includes robotics, natural language processing, computer vision, decision-making and reasoning, human-computer interaction, recommendation and search, basic AI theory, etc.

[0003] At present, various AI functions are usually realized through neural networks, such as natural language processing, image processing, and speech processing. With the development of AI technology, the functions of AI have gradually diversified. Taking the panoramic segmentation task as an example, the panoramic segmentation task includes multiple subtasks such as instance segmentation, classification, regression, instance mask, and semantic segmentation. Each subtask has an optimization goal. In the model training process corresponding to the panoramic segmentation task, each subtask corresponds to a subnetwork when it is optimized, and each subnetwork corresponds to multiple loss values. Then, the neural network parameters are optimized by the back propagation algorithm according to the loss value. Therefore, the final output model is also called a multi-loss model. In the training process of the multi-loss model, the competitive balance between each subnetwork determines the final training effect of the multi-loss model. For example, in the training of the neural network model corresponding to the panoramic segmentation task, the training of the detection task and the segmentation task competes with each other, which will result in poor segmentation effect of a subtask. Therefore, it is necessary to automatically optimize the training problems between multiple tasks. At the same time, in the multi-task application scenario, each subnetwork is very sensitive to the loss weight value.

[0004] However, existing methods usually use sampling and other methods to search and optimize hyperparameters, and cannot optimize the loss weighted values ​​in a targeted manner. Therefore, how to better optimize the loss weighted values ​​during the training of multi-loss models is an urgent problem to be solved. Summary of the invention

[0005] The embodiments of the present application provide a method and device for acquiring a multi-loss model, which are used to dynamically adjust the loss weight value during the multi-loss model training process, so that the multi-loss model can obtain better training results.

[0006] In the first aspect, the embodiment of the present application provides a multi-loss model acquisition method, which is applied to the acquisition scenario of a multi-loss model including at least one sub-network (i.e., sub-task). During the multi-loss model training process, in the first iteration cycle, the multi-loss model acquisition device acquires the first loss value output by the sub-network; then the multi-loss model acquisition device generates an alternative weight according to the first loss value; next, the multi-loss model acquisition device inputs the first loss value and the alternative weight into a weight prediction model, and outputs the first weight parameter of the sub-network, wherein the first weight parameter is used for the training of the second iteration cycle in the multi-loss model training process, wherein the second iteration cycle is the next iteration cycle of the first iteration cycle; finally, the multi-loss model training updates the parameters of the multi-loss model according to the first weight parameter.

[0007] It is understandable that the above only describes one iteration process in the training process, and the first iteration cycle and the second iteration cycle are only used to distinguish the iteration cycles, and are not used to limit a specific iteration cycle in the training process. In specific applications, the multi-loss model training process has multiple iteration processes mentioned above. After the training iteration cycle of the multi-loss model is completed (that is, equivalent to the completion of the model search process), the multi-loss model acquisition device outputs the trained multi-loss model.

[0008] In this embodiment, during the training of the multi-loss model, each iteration cycle updates the optimized weight value for multiple loss values, and at the same time, the advantages of parallel computing are used to explore the effectiveness of the weights and inherit excellent network parameters and weights during the training process, so as to achieve the optimal training model within one training time. Optionally, the specific operation of the multi-loss model acquisition device in generating candidate weights according to the first loss value is as follows: after obtaining the first loss value, the multi-loss model acquisition device determines the sampling space of the first loss value; and then samples in the sampling space to obtain the candidate weight.

[0009] In this embodiment, the sampling space includes but is not limited to the log-normal distribution of the first loss value or the log-uniform distribution of the first loss value.

[0010] When the sampling space is the lognormal distribution of the first loss value, the multi-loss model acquisition device determines that the lognormal distribution is the sampling space; then the sampling variance of the sampling space is obtained, and the sampling mean of the sampling space is calculated, wherein the sampling variance is a preset parameter; finally, the alternative weight is obtained by sampling in the sampling space according to the sampling mean and the sampling variance. In this embodiment, the sampling space generated by the first loss value and the sampling mean are used to generate the alternative weights, so that the loss value and the weighting can be associated together to obtain a targeted high-quality weighting scheme. At the same time, the use of the lognormal distribution can be based on the good generalization of the lognormal distribution to ensure that the alternative weights obtained by sampling are greater than zero.

[0011] Optionally, the number of candidate weights can be determined according to the number of the first loss values ​​(which is equivalent to being determined according to the number of the subnetworks), that is, the number of candidate weights is 2 to the power of N, where N is the number of the first loss values. In the case where the first loss value is small, the value of N can be set in this embodiment to ensure that enough candidate weights can be selected. In this embodiment, enough candidate weights are selected to achieve the purpose of selecting high-quality weight parameters.

[0012] Optionally, the multi-loss model acquisition device may also update the sampling mean in real time during the training process, and the specific operation is as follows: the multi-loss model acquisition device calculates the sampling mean using a first formula and the first loss value, wherein the first formula is:

[0013] Wherein, μ is used to represent the sampled mean, c is used to represent the first loss value, i is used to represent the identifier of the first loss value, and n is used to represent the number of the first loss values. In this embodiment, by updating the sampled mean according to the loss value, the loss value can be more closely associated with the weighting in real time, thereby achieving a higher quality weighting scheme.

[0014] Optionally, after generating the candidate weights, the multi-loss model acquisition device can also screen the candidate weights to obtain the weight parameters of the sub-network trained in the second iteration cycle, and the specific operation is as follows: the multi-loss model acquisition device inputs the first loss value and the candidate weights into the weight prediction model to calculate the expected scores corresponding to each candidate weight, wherein the expected scores are used to evaluate the training effect of the candidate weights; then the expected scores are arranged in descending order, and the first K candidate weights are selected as the first weight parameters corresponding to the sub-network, wherein K is the number of the sub-networks. In this embodiment, by selecting better weight parameters based on the prediction of the training effect, the sampling efficiency and the quality of the weight parameters can be improved.

[0015] Optionally, the weight prediction model can also be updated according to historical training information, wherein the historical training information includes the loss value, weight parameter and training effect corresponding to the first iteration cycle and the loss value, weight parameter and training effect corresponding to the iteration cycle before the first iteration cycle. In this embodiment, the multi-loss model acquisition device is constructed based on the historical information in the training process as training data, which can effectively improve the sampling efficiency and weight generation quality.

[0016] Optionally, the multi-loss model training parameters can also be initially trained using a small amount of training samples in the initial stage of training to obtain initialization training parameters, wherein the initialization training parameters include initialization weights, sampling variances, and initial parameters of the weight prediction model, and then the training of the multi-loss model is started according to the initialization training parameters. It can be understood that in this embodiment, the multi-loss model acquisition device can use a small amount of samples for initialization training or the user can directly set the corresponding initialization data in the initialization stage. The specific method is not limited here.

[0017] In the second aspect, an embodiment of the present application provides a multi-loss model acquisition device, which includes at least one functional module, and the at least one functional module is used to implement the method described in the above-mentioned first aspect or any possible implementation method of the first aspect. The functional modules can communicate with each other in order to implement the method steps.

[0018] In a third aspect, an embodiment of the present application provides a multi-loss model acquisition device, which includes a memory and a processor; the memory stores programmable instructions; the processor is used to call the programmable instructions to execute the method described in the above-mentioned first aspect or any possible implementation of the first aspect.

[0019] In a fourth aspect, an embodiment of the present application provides a multi-loss model acquisition device, the device comprising a processor; the processor is used to couple with a memory, read instructions in the memory and execute the method described in the first aspect or any possible implementation of the first aspect according to the instructions. Optionally, the memory is an internal memory of the device, or it can be an external memory of the device. Optionally, the device can be a dedicated chip for multi-loss model training, then the memory can be integrated in the dedicated chip, or it can be independent of the dedicated chip.

[0020] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium having instructions stored thereon. When the instructions are executed on a computer, the computer executes the method described in the first aspect or any possible implementation of the first aspect.

[0021] In a sixth aspect, an embodiment of the present application provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute the methods described in the above aspects.

[0022] In the seventh aspect, an embodiment of the present application provides a multi-loss model acquisition device, which includes: an acquisition module, used to obtain a first loss value corresponding to the sub-network, the first loss value is the loss value output in the first iteration cycle under the multi-task training scenario; a generation module, used to generate alternative weights according to the first loss value; a screening module, used to input the first loss value and the alternative weight into a weight prediction model, and output a first weight parameter of the sub-network, the first weight parameter is used for training in a second iteration cycle, and the second iteration cycle is the next iteration cycle of the first iteration cycle; a first update module, used to update the parameters of the multi-loss model according to the first weight parameter.

[0023] Optionally, the generating module is specifically used to determine a sampling space of the first loss value; and sample in the sampling space to obtain the candidate weights.

[0024] Optionally, the sampling space includes a log-normal distribution of the first loss value or a log-uniform distribution of the first loss value.

[0025] Optionally, when the sampling space is a log-normal distribution of the first loss value, the generating module is specifically used to obtain the sampling variance of the sampling space and calculate the sampling mean of the sampling space, where the sampling variance is a preset parameter; and sample in the sampling space according to the sampling mean and the sampling variance to obtain the alternative weights.

[0026] Optionally, the generating module is specifically configured to calculate the sampling mean using a first formula and the first loss value; wherein the first formula is:

[0027]

[0028] Among them, μ is used to represent the sampling mean, c is used to represent the first loss value, i is used to represent the identifier of the first loss value, and n is used to represent the number of the first loss values.

[0029] Optionally, the screening module is specifically used to input the first loss value and the alternative weight into the weight prediction model to calculate the expected score corresponding to the alternative weight, and the expected score is used to evaluate the training effect of the alternative weight; arrange the expected scores in descending order, and select the first K alternative weight outputs as the first weight parameter, where K is the number of sub-networks.

[0030] Optionally, the device also includes: an initialization module, used to obtain initialization training parameters, the initialization training parameters including initialization weights, sampling variance, and initial parameters of the weight prediction model; and starting the multi-loss model training according to the initialization training parameters.

[0031] Optionally, the device also includes: a second updating module, used to update the weight prediction model according to historical training information, the historical training information including the loss value of the first iteration cycle and before the first iteration cycle, the weight parameters corresponding to the loss value, and the training effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 A schematic diagram of an artificial intelligence main framework provided for an embodiment of the present application;

[0033] Figure 2 A schematic diagram of an application environment provided for an embodiment of the present application;

[0034] Figure 3 A system architecture diagram provided for an embodiment of the present application;

[0035] Figure 4 A training system architecture diagram provided for an embodiment of the present application;

[0036] Figure 5 This is an embodiment of the method for obtaining a multi-loss model in the embodiments of the present application;

[0037] Figure 6 A schematic diagram of a workflow of a method for obtaining a multi-loss model in an embodiment of the present application;

[0038] Figure 7 This is a comparison diagram of experimental results of a multi-loss model in an embodiment of the present application;

[0039] Figure 8 This is another comparison diagram of experimental results of the multi-loss model in the embodiment of the present application;

[0040] Fig. 9 This is another comparison diagram of experimental results of the multi-loss model in the embodiment of the present application;

[0041] Fig.10 This is another comparison diagram of experimental results of the multi-loss model in the embodiment of the present application;

[0042] Fig.11 A visualization comparison diagram of the multi-loss model in the embodiment of the present application;

[0043] Fig.12 This is a schematic diagram of an embodiment of a device for acquiring a multi-loss model in an embodiment of the present application;

[0044] Fig.13 It is a schematic diagram of another embodiment of the multi-loss model acquisition device in the embodiment of the present application;

[0045] Fig.14 A system framework diagram of a multi-loss model acquisition device in an embodiment of the present application;

[0046] Fig.15 It is a schematic diagram of another embodiment of the multi-loss model acquisition device in the embodiment of the present application. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solutions and advantages of the present application more clear, the embodiments of the present application are described below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. It is known to those skilled in the art that with the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0048] The terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments described here can be implemented in an order other than that illustrated or described here. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or modules is not necessarily limited to those steps or modules clearly listed, but may include other steps or modules that are not clearly listed or inherent to these processes, methods, products or devices. The naming or numbering of steps in this application does not mean that the steps in the method flow must be executed in the time / logical sequence indicated by the naming or numbering. The process steps that have been named or numbered can change the execution order according to the technical purpose to be achieved, as long as the same or similar technical effects can be achieved. The division of units in this application is a logical division. There may be other division methods when it is implemented in actual applications. For example, multiple units can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection between units can be electrical or other similar forms, which are not limited in this application. In addition, the units or sub-units described as separate components may or may not be physically separated, may or may not be physical units, or may be distributed in multiple circuit units, and some or all of the units may be selected according to actual needs to achieve the purpose of the present application.

[0049] In order to better understand the embodiments of the present invention, the following concepts that may be involved in the embodiments are first explained:

[0050] Hyperparameter optimization (HPO): refers to the use of algorithms to tune the hyperparameters in existing models or algorithms. Hyperparameters generally refer to the model or algorithm settings that are set. When using a hyperparameter optimization algorithm, appropriate hyperparameters can be given for a specific model without manual intervention, thereby maximizing the performance of the model. Hyperparameter optimization is an important part of automated machine learning.

[0051] Automatic machine learning (AutoML): refers to designing a series of advanced control systems to operate machine learning models so that the models can automatically learn appropriate parameters and configurations without human intervention. In learning models based on deep neural networks, automatic computational learning mainly includes network architecture search and global parameter setting. Among them, network architecture search is a recent research hotspot. It is used to allow computers to generate the most suitable neural network architecture for the problem based on data. It has the characteristics of high training complexity and great performance improvement.

[0052] Panoptic Segmentation: Panoptic segmentation is the latest image parsing task. This task combines two core computer vision tasks, instance segmentation and semantic segmentation, and requires algorithms to solve the pain points of both tasks at the same time.

[0053] Loss: For neural network training, the loss between the network output and the annotation is usually calculated, and then the neural network parameters are optimized through the back propagation algorithm. For models of tasks such as panoptic segmentation, it generally includes the losses of multiple subtasks, and finally the final loss value of the network is obtained by weighted summation, and network training is usually sensitive to weighted values.

[0054] Multi-Layer Perceptron (MLP): It is a simple artificial neural network, generally including input layer, hidden layer and output layer. The input layer is usually represented by a vector, and the network parameters are optimized by the back propagation algorithm.

[0055] Figure 1 A schematic diagram of an artificial intelligence main framework is shown, which describes the overall workflow of the artificial intelligence system and is applicable to general artificial intelligence field requirements.

[0056] The following is an explanation of the above artificial intelligence theme framework from two dimensions: "intelligent information chain" (horizontal axis) and "IT value chain" (vertical axis).

[0057] The "intelligent information chain" reflects a series of processes from data acquisition to processing. For example, it can be a general process of intelligent information perception, intelligent information representation and formation, intelligent reasoning, intelligent decision-making, intelligent execution and output. In this process, data undergoes a condensed process of "data-information-knowledge-wisdom".

[0058] The "IT value chain" reflects the value that artificial intelligence brings to the information technology industry, from the underlying infrastructure of human intelligence, information (providing and processing technology implementation) to the system's industrial ecological process.

[0059] (1) Infrastructure:

[0060] The infrastructure provides computing power support for the AI ​​system, enables communication with the outside world, and supports it through the basic platform. It communicates with the outside world through sensors; computing power is provided by smart chips (CPU, NPU, GPU, ASIC, FPGA and other hardware acceleration chips); the basic platform includes distributed computing frameworks and networks and other related platform guarantees and support, which can include cloud storage and computing, interconnected networks, etc. For example, sensors communicate with the outside world to obtain data, and these data are provided to the smart chips in the distributed computing system provided by the basic platform for calculation.

[0061] (2) Data

[0062] The data on the upper layer of the infrastructure is used to represent the data sources in the field of artificial intelligence. The data involves graphics, images, voice, text, and IoT data of traditional devices, including business data of existing systems and perception data such as force, displacement, liquid level, temperature, and humidity.

[0063] (3) Data processing

[0064] Data processing usually includes data training, machine learning, deep learning, search, reasoning, decision-making and other methods.

[0065] Among them, machine learning and deep learning can symbolize and formalize data for intelligent information modeling, extraction, preprocessing, and training.

[0066] Reasoning refers to the process of simulating human intelligent reasoning in computers or intelligent systems, using formalized information to perform machine thinking and solve problems based on reasoning control strategies. Typical functions are search and matching.

[0067] Decision-making refers to the process of making decisions after intelligent information is reasoned, usually providing functions such as classification, sorting, and prediction.

[0068] (4) General ability

[0069] After the data has undergone the data processing mentioned above, some general capabilities can be further formed based on the results of the data processing, such as an algorithm or a general system, for example, translation, text analysis, computer vision processing, speech recognition, image recognition, etc.

[0070] (5) Smart products and industry applications

[0071] Smart products and industry applications refer to the products and applications of artificial intelligence systems in various fields. They are the encapsulation of the overall artificial intelligence solution, which productizes intelligent information decision-making and realizes practical application. Its application areas mainly include: smart manufacturing, smart transportation, smart home, smart medical care, smart security, autonomous driving, safe cities, smart terminals, etc.

[0072] The multi-loss model provided in the embodiment of the present application can be applied to various multi-task scenarios. In an exemplary solution, the multi-task scenario can be a computer vision scenario, including Figure 2 The panoramic segmentation scene in the unmanned vehicle perception system shown in the figure. In this panoramic segmentation task, the network structure can include three stages of classification, regression, instance mask loss and one semantic segmentation loss, a total of 10 losses need to be optimized and tuned at the same time. That is, in the panoramic segmentation scene, each target object in the collected image is segmented, and an instance mask of each object (such as each car, person, etc.) is obtained. At the same time, the neural network needs to output the semantic segmentation results of the background (such as the sky, the ground). Figure 2 The figure a shows a scene picture collected by the unmanned vehicle perception system; Figure 2 The b in FIG. 1 shows the output image after panoramic segmentation.

[0073] In this embodiment, a computer vision scene is used as an example to illustrate. Figure 3As shown, the embodiment of the present application provides a system architecture 300. The data acquisition device 360 ​​is used to obtain sample data and loss values ​​generated by training and store them in the database 330. The training device 320 generates a target model / rule 301 based on the sample data maintained in the database 330 and the loss values ​​generated by training. The following will describe in more detail how the training device 320 obtains the target model / rule 301 based on the sample data and the loss values ​​generated by training. The target model / rule 301 can adaptively adjust the weight parameters corresponding to the loss value, and at the same time, use the advantages of parallel computing to explore the effectiveness of the weights and inherit excellent network parameters and weights during the training process, so as to achieve the optimal training model within one training time.

[0074] like Figure 4 As shown, the embodiment of the present application provides a training system 400 for a multi-loss model, including a weight generation module 401, a search module 402, and a weight prediction and update module 403. Among them, the weight generation module 401 includes a weight sampling 4011 and a weight prediction 4012. The search module 402 outputs the loss value of each iteration cycle to the weight generation module 401, and then the weight sampling 4011 of the weight generation module 401 determines the sampling space and the sampling mean according to the loss value, and samples and generates alternative weights in the sampling space according to the sampling mean and the preset sampling variance; then the expected score corresponding to each alternative weight is predicted by the weight prediction 4012, and then the alternative weight with the highest expected score is selected as the weight parameter of the next stage. The search module 402 adds the weight parameter output by the weight prediction 4012 to the training process, and the search module 402 will train at least one sub-network. It can be understood that the training process of the search module 402 also includes other network parameters, which are not specifically limited here. The above process is repeated to achieve iterative training. At the same time, the search module 402 uses the loss value output by each iterative training, the weight parameter corresponding to the loss value, and the training effect as the input of the weight prediction update module 403, and generates the weight prediction 4012 corresponding to the model capability through MLP.

[0075] This application embodiment provides a method for obtaining a multi-loss model. Figure 5 As shown, the specific steps of the multi-loss model acquisition method are as follows:

[0076] 501. Obtain a first loss value corresponding to the sub-network, wherein the first loss value is a loss value outputted in a first iteration cycle.

[0077] During the model training process, the multi-loss model acquisition device acquires the first loss value output by the at least one sub-network in the first iteration cycle training. Assuming that the training process of the multi-loss model includes two sub-networks, each sub-network corresponds to 10 loss values, the number of iterations is 2000, and the first iteration cycle is the 100th, the multi-loss model acquisition device will acquire 20 loss values ​​output during the 100th training process (that is, these 20 loss values ​​are used as the first loss value).

[0078] 502. Generate an alternative weight according to the first loss value.

[0079] In this embodiment, the specific operation of the multi-loss model acquisition device to generate the candidate weights according to the first loss value is as follows: the multi-loss model acquisition device determines the sampling space corresponding to the first loss value, and then samples in the sampling space to obtain the candidate weights.

[0080] It is understandable that, in this embodiment, the sampling space corresponding to the first loss value includes but is not limited to the log-normal distribution of the first loss value or the log-uniform distribution of the first loss value. At the same time, for different sampling spaces, the parameters for sampling by the multi-loss model acquisition device are different.

[0081] When the sampling space is a log-normal distribution of the first loss value, the specific operation of the multi-loss model acquisition device to generate candidate weights according to the first loss value is as follows:

[0082] The multi-loss model acquisition device determines the log-normal distribution of the first loss value; obtains the sampling variance, and calculates the sampling mean, wherein the sampling variance is a parameter preset by the user according to the empirical value in the initialization stage; then takes the set of values ​​in the log-normal distribution of the first loss value as the sampling space; finally, selects the alternative weights in the sampling space according to the sampling variance and the sampling mean. It can be understood that in order to ensure a sufficient number of alternative weights, the selection can be made according to the number of the first loss values. In this embodiment, the number of the alternative weights can be set to 2 to the power of N groups, wherein N is the number of the first loss values. For example, if the training process of the multi-loss model includes 2 loss values, the number of the alternative weights can be 4 groups, namely (A1, B1), (A2, B2), (A3, B3) and (A4, B4).

[0083] In this embodiment, the multi-loss model acquisition device can calculate the sampling mean using the first formula and the first loss value;

[0084] Among them, the first formula is:

[0085]

[0086] Among them, μ is used to represent the sampling mean, c is used to represent the first loss value, i is used to represent the identifier of the first loss value, and n is used to represent the number of the first loss values.

[0087] When the sampling space is a logarithmic uniform distribution of the first loss value, the multi-loss model acquisition device samples the sampling space according to the sampling probability to obtain the candidate weight.

[0088] 503. Input the first loss value and the candidate weight into a weight prediction model, and output a first weight parameter corresponding to the sub-network, wherein the first weight parameter is used for training in a second iteration cycle, and the second iteration cycle is the next iteration cycle of the first iteration cycle.

[0089] In this embodiment, the first loss value corresponds to multiple groups of candidate weights; the multi-loss model acquisition device inputs the first loss value and each group of candidate weights into the weight prediction model in sequence, and then calculates the expected score of each group of candidate weights, that is, predicts the training effect of each group of candidate weights after training; the expected scores are sorted in descending order, and finally the top K candidate weights are selected as the first weight parameter according to the number of sub-networks trained in parallel in the multi-task training scenario. Assuming that the training process of the multi-loss model includes two loss values, and the two loss values ​​correspond to the same sub-network, the number of candidate weights can be 4 groups, namely (A1, B1), (A2, B2), (A3, B3) and (A4, B4). The expected scores corresponding to each group of candidate weights are obtained through the weight prediction model, assuming that they are C1, C2, C3 and C4, and decrease in the order of C1, C2, C3, C4; since the number of sub-networks is 1, the candidate weight group with the expected value of C1 is selected as the first weight parameter of the sub-network. In this embodiment, when the number of the loss values ​​is small, the number of the candidate weights can be set by the user as long as the number of the candidate weights is sufficient.

[0090] In this embodiment, the weight prediction model can also be updated according to historical training information, which is equivalent to training according to historical training information to obtain a better weight prediction model. The historical training information includes the loss value of the previous iteration cycle, the weight parameter corresponding to the loss value, and the training result. In this embodiment, the weight prediction model can be a simple neural network model, such as MLP.

[0091] 504. Update the parameters of the multi-loss model according to the first weight parameter.

[0092] The multi-loss model acquisition device uses the first weight parameter to update and replace the weight parameter of the first iteration cycle, and then adds the first weight parameter to the updated network parameter as the training parameter of the multi-loss model to perform training in the second iteration cycle.

[0093] In this embodiment, after the training iteration cycle of the multi-loss model is completed, the training result of the multi-loss model is evaluated. After the training result achieves the expected effect, the multi-loss model is output.

[0094] The following is a specific training process to illustrate the method for obtaining a multi-loss model in the embodiment of the present application. Figure 6 As shown:

[0095] At the beginning of multi-loss model training, first initialize the weight generator and initialize the network parameters, and use the weight generator to generate the initialization weight parameters. Use the initialization weight parameters and the initialization network parameters to perform parallel model training. Among them, n sub-networks are trained in parallel, and each sub-network is trained for a single training phase (such as 2000 iterations). After the single training phase of each sub-network is completed, the model performance of the single model is evaluated, and the model performance (i.e., training effect), loss value (loss) and weight parameters corresponding to the loss value are output. Determine whether the search phase of the multi-loss model is completed. If not, enter the update phase. In this update phase, use the above Figure 5 Generate new weight parameters in the manner shown. Use the new weight parameters to update the weight parameters of the previous iteration cycle, and update the network parameters. Use the updated weight parameters and network parameters to perform parallel model training. After the single training phase of each sub-network is completed, the model performance of the single model is evaluated, and the model performance (i.e., training effect), loss value (loss) and weight parameters corresponding to the loss value are output. Determine whether the search phase of the multi-loss model is completed. If not, return to the update phase. If completed, output the optimal model.

[0096] above Figure 5 to Figure 6 The universal scenarios of the multi-loss model are described. In the specific panoramic segmentation task experiment, it is assumed that the panoramic segmentation dataset contains 80 target object categories and 53 background category categories, corresponding to approximately 118,000 training images and 5,000 test images (also known as the COCO dataset). The panoramic segmentation quality (PQ) is used as the overall measurement indicator, and the panoramic segmentation quality of the foreground and background (PQth, PQst) is considered. The model output by the model acquisition method in the embodiment of the present application can achieve the best effect. The performance of this model is greatly improved compared to the model performance obtained without tuning or manual tuning. The experimental results are as follows. Figures 7 to 9 As shown. Figures 7 to 9It can be seen from the results shown that, on the data set of the embodiment of the present application, the model obtained by the acquisition method of the present application achieved the best total PQ effect on the test set compared to the manually adjusted parameter model; and the best trade-off was achieved in the foreground-background, without the imbalance of effects seen in other methods. The model obtained by the acquisition method of the present application can reach 47.5% PQ. On the validation set, by dynamically adjusting the weights during the training process of the present application, it can be seen that the model obtained by the acquisition method of the present application has a very large improvement (an increase of 2.4%).

[0097] Assume that the panoramic segmentation dataset contains 100 target object categories and 50 background category categories, corresponding to 20,000 training images and 2,000 test images (also known as the ADE20K dataset). Panoptic segmentation quality (PanopticQuality, PQ) is also used as the overall measurement indicator, and the panoramic segmentation quality (PQth, PQst) of the foreground and background is also considered. The model output by the model acquisition method in the embodiment of the present application can achieve the best effect. The performance of this model is greatly improved compared with the model performance obtained without tuning or manual tuning. The experimental results are as follows. Fig.10 shown. Fig.10 Among them, (Our Baselinee) is used to indicate the baseline without adjusting the weight parameters, (Our Baselinee*) is used to indicate the baseline with manually adjusted weight parameters, and (Our Baselinee+Ada-Segment) is used to indicate the baseline obtained by the method of this application; it can be seen that the model obtained by the method of this application has a significant improvement.

[0098] And in Fig.11 It can also be seen from the visualization effect that the effect of the model output by the acquisition method of the embodiment of the present application can achieve better results. Among them, image is used to indicate the original image, Ground truth is used to indicate the image segmentation result output by the model that does not use the acquisition method of the embodiment of the present application, and ours is used to indicate the image segmentation result output by the model that uses the acquisition method of the embodiment of the present application.

[0099] In an embodiment of the present application, during the training process of a multi-loss model, the loss values ​​of multiple tasks are updated and optimized for each iteration cycle. At the same time, the advantages of parallel computing are used during the training process to explore the effectiveness of the weights and inherit excellent network parameters and weights, thereby achieving the optimal training model within one training time.

[0100] The above describes the method for obtaining a multi-loss model in the embodiment of the present application. The following describes the device for obtaining a multi-loss model in the embodiment of the present application. For details, please refer to Fig.12As shown, an embodiment of the multi-loss model acquisition device 1200 in the embodiment of the present application includes: an acquisition module 1201, used to obtain a first loss value corresponding to the sub-network, the first loss value is the loss value output in the first iteration cycle in the multi-task training scenario;

[0101] A generating module 1202, configured to generate a candidate weight according to the first loss value;

[0102] A screening module 1203 is used to input the first loss value and the candidate weight into a weight prediction model, and output a first weight parameter of the sub-network, where the first weight parameter is used for training in a second iteration cycle, and the second iteration cycle is an iteration cycle next to the first iteration cycle;

[0103] The first updating module 1204 is used to update the parameters of the multi-loss model according to the first weight parameter.

[0104] The acquisition module 1201 is used to execute step 501 of the above embodiment, that is, to acquire the first loss value corresponding to the sub-network.

[0105] The generating module 1202 is used to execute step 502 of the above embodiment, that is, to generate candidate weights according to the first loss value.

[0106] The screening module 1203 is used to execute step 503 of the above embodiment, that is, input the first loss value and the candidate weight into the weight prediction model, and output the first weight parameter of the sub-network.

[0107] The first updating module 1204 is used to execute step 504 of the above embodiment, that is, to update the training parameters of the multi-loss model according to the first weight parameter.

[0108] The present application embodiment provides a multi-loss model acquisition device, such as Fig.13 As shown, the device 1300 includes: at least one processor 1301, a memory 1302, at least one communication bus 1303, and at least one network interface 1304 or other user interface 1305. The communication bus 1303 is used to realize the connection and communication between these components.

[0109] The memory 1302 may include a read-only memory and a random access memory, and provides instructions and data to the processor 1301. A portion of the memory 1302 may also include a non-volatile random access memory (NVRAM).

[0110] In some embodiments, the memory 1302 stores the following elements, executable modules or data structures, or a subset thereof, or an extended set thereof:

[0111] Operating system 13021, including various system programs, such as Fig.14 The framework layer, core library layer, driver layer, etc. shown are used to implement various basic services and handle hardware-based tasks;

[0112] The application module 13022 includes various applications, such as Fig.14 The desktop (launcher), media player (Media Player), browser (Browser), etc. are used to implement various application services.

[0113] In the embodiment of the present application, by calling the program or instruction stored in the memory 1302, the processor 1301 is used to implement Figure 5 and Figure 6 The method described in the corresponding embodiment of this application.

[0114] The embodiment of the present application provides a multi-loss model acquisition device 1500, such as Fig.15 As shown, the device 1500 includes: a first processor 1501, a memory 1502, a transceiver 1503, a second processor 1504, and a communication bus 1505. The communication bus 1505 is used to realize connection and communication between these components.

[0115] The transceiver 1503 is used for data transmission with the outside.

[0116] The memory 1502 may include a read-only memory and a random access memory, and provide instructions and data to the first processor 1501 and the second processor 1504. A portion of the memory 1502 may also include a non-volatile random access memory (NVRAM), such as RAM, ROM, EEPROM, CD-ROM, optical disk, hard disk, magnetic storage device, etc.; the memory 1502 may be used to store one or more of computer program instructions, preset parameters, data obtained by computer intermediate operations, etc.

[0117] The first processor 1501 and the second processor 1504 may be a central processing unit (CPU), or a digital processing unit, etc.

[0118] In the embodiment of the present application, the first processor 1501 may optionally include an on-chip memory, such as TCM, Cache, SRAM, in which instructions are stored, and the first processor 1501 is coupled to the on-chip memory to implement Figure 5 and Figure 6 Corresponding to the method described in the embodiment of the present application, or, the first processor 1501 is coupled with the on-chip memory to call the instruction in the on-chip memory, and is coupled with the memory 1502 to obtain data, so as to implement Figure 5 and Figure 6 The corresponding method described in the embodiment of the present application. In practice, the first processor 1501 may be a chip sold independently, or may be integrated on a chip that includes the first processor 1501.

[0119] Optionally, by calling the program or instruction stored in the memory 1502, the second processor 1504 is used to implement Figure 5 and Figure 6 The method described in the corresponding embodiment of this application.

[0120] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk Solid State Disk (SSD)), etc.

[0121] Each module of the embodiment of the present application can also realize Figure 5 The other method steps described in the corresponding embodiments of the present application will not be repeated here.

[0122] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0123] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0124] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0125] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0126] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk.

[0127] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for obtaining a multi-loss model, comprising at least one sub-network, characterized in that: include: Obtaining a first loss value corresponding to the subnetwork, where the first loss value is a loss value output by a first iteration cycle in a multi-loss model acquisition scenario, where the multi-loss model is used to process an input image to obtain a panoramic segmentation result of the input image; Determining a sampling space of the first loss value; Sampling in the sampling space to obtain candidate weights; Inputting the first loss value and the candidate weight into a weight prediction model to calculate an expected score corresponding to the candidate weight, wherein the expected score is used to evaluate the training effect of the candidate weight; Arrange the expected scores in descending order, select the first K candidate weight outputs as the first weight parameter, where K is the number of the sub-networks, and the first weight parameter is used for training in a second iteration cycle, which is the next iteration cycle of the first iteration cycle; Update the parameters of the multi-loss model according to the first weight parameter.

2. The method according to claim 1, characterized in that The sampling space includes a log-normal distribution of the first loss value or a log-uniform distribution of the first loss value.

3. The method according to claim 2, characterized in that When the sampling space is a log-normal distribution of the first loss value, sampling the sampling space to obtain the candidate weights includes: Obtaining a sampling variance of the sampling space, and calculating a sampling mean of the sampling space, wherein the sampling variance is a preset parameter; The candidate weights are obtained by sampling in the sampling space according to the sampling mean and the sampling variance.

4. The method according to claim 3, characterized in that The calculating the sampling mean of the sampling space comprises: Calculating the sampling mean using a first formula and the first loss value; Among them, the first formula is: Among them, μ is used to represent the sampling mean, c is used to represent the first loss value, i is used to represent the identifier of the first loss value, and n is used to represent the number of the first loss values.

5. The method according to any one of claims 1 to 4, characterized in that The number of the alternative weights is 2 to the power of N, where N is the number of the first loss values.

6. The method according to any one of claims 1 to 4, characterized in that The method further comprises: Obtaining initialization training parameters, wherein the initialization training parameters include initialization weights, sampling variances, and initial parameters of the weight prediction model; Initiate training of the multi-loss model according to the initialization training parameters.

7. The method according to any one of claims 1 to 4, characterized in that The method further comprises: The weight prediction model is updated according to historical training information, wherein the historical training information includes the loss value during the first iteration cycle and before the first iteration cycle, the weight parameter corresponding to the loss value, and the training effect.

8. A multi-loss model acquisition device, applied to a multi-loss model acquisition scenario including at least one sub-network, characterized in that: include: An acquisition module, used to acquire a first loss value corresponding to the sub-network, wherein the first loss value is a loss value outputted by a first iteration cycle in a multi-loss model acquisition scenario, wherein the multi-loss model is used to process an input image to obtain a panoptic segmentation result of the input image; A generating module, used for determining a sampling space of the first loss value; Sampling in the sampling space to obtain candidate weights; A screening module, used for inputting the first loss value and the candidate weight into a weight prediction model to calculate an expected score corresponding to the candidate weight, wherein the expected score is used to evaluate the training effect of the candidate weight; Arrange the expected scores in descending order, select the first K candidate weight outputs as the first weight parameter, where K is the number of sub-networks, and the first weight parameter is used for training in a second iteration cycle, which is the next iteration cycle of the first iteration cycle; A first updating module is used to update the parameters of the multi-loss model according to the first weight parameter.

9. The device according to claim 8, characterized in that The sampling space includes a log-normal distribution of the first loss value or a log-uniform distribution of the first loss value.

10. The device according to claim 9, characterized in that When the sampling space is a log-normal distribution of the first loss value, the generation module is specifically used to obtain the sampling variance of the sampling space and calculate the sampling mean of the sampling space, where the sampling variance is a preset parameter; and sample in the sampling space according to the sampling mean and the sampling variance to obtain the alternative weights.

11. The device according to claim 10, characterized in that The generating module is specifically used to calculate the sampling mean using the first formula and the first loss value; Among them, the first formula is: Among them, μ is used to represent the sampling mean, c is used to represent the first loss value, i is used to represent the identifier of the first loss value, and n is used to represent the number of the first loss values.

12. The device according to any one of claims 8 to 11, characterized in that The number of the alternative weights is 2 to the power of N, where N is the number of the first loss values.

13. The device according to any one of claims 8 to 11, characterized in that The device also includes: An initialization module, used to obtain initialization training parameters, wherein the initialization training parameters include initialization weights, sampling variances, and initial parameters of the weight prediction model; Initiate training of the multi-loss model according to the initialization training parameters.

14. The device according to any one of claims 8 to 11, characterized in that The device also includes: The second updating module is used to update the weight prediction model according to historical training information, wherein the historical training information includes the loss value of the first iteration cycle and before the first iteration cycle, the weight parameter corresponding to the loss value, and the training effect.

15. A training device for a multi-loss model, comprising at least one sub-network, characterized in that: include: A memory and at least one processor, wherein the memory is used to store program instructions, and when the at least one processor calls the program instructions, the multi-loss model acquisition method according to any one of claims 1 to 7 is implemented.

16. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, which, when executed on a processor, implement the multi-loss model acquisition method described in any one of claims 1-7.

17. A training system for a multi-loss model, characterized in that: include: Weight generation module, search module and weight prediction update module; The weight generation module includes a weight sampling module and a weight prediction module; The weight sampling module obtains a first loss value corresponding to the sub-network, the first loss value is a loss value outputted in a first iteration cycle in a multi-loss model acquisition scenario, the multi-loss model is used to process an input image to obtain a panoramic segmentation result of the input image; and determines a sampling space for the first loss value; Sampling in the sampling space to obtain candidate weights; The weight prediction module inputs the first loss value and the candidate weight into a weight prediction model to calculate an expected score corresponding to the candidate weight, where the expected score is used to evaluate the training effect of the candidate weight; Arrange the expected scores in descending order, select the first K candidate weight outputs as the first weight parameter, where K is the number of sub-networks, and the first weight parameter is used for training in a second iteration cycle, which is the next iteration cycle of the first iteration cycle; The search module performs training for a second iteration cycle according to the first weight parameter and the network parameter corresponding to the sub-network and outputs a training effect and a second loss value corresponding to the sub-network; The weight prediction update module updates the parameters of the weight prediction module according to the second loss value and the training effect.

Images