Image processing method, and image processing model training method and apparatus

Abstract

Embodiments of the present application relate to the field of artificial intelligence, and in particular relate to an image processing method, and an image processing model training method and apparatus, which are used for preventing a model analysis result from falling into local optimum. In the image processing method of the present application, a mapping relationship set having a minimum transmission cost is used as a target mapping set, and the transmission cost of the mapping relationship set is the sum of transmission costs respectively corresponding to mapping relationships in the mapping relationship set. The mapping relationship set comprises a mapping relationship between each label in a target domain label and a source domain label, wherein each label in the target domain label corresponds to only one source domain label, and each source domain label also corresponds to only one target domain label. During implementation of a visual prompt learning method, a target mapping set having a minimum transmission cost is used as output label mappings, and the target mapping set is selected from the perspective of the total transmission cost of all mappings, thereby effectively reducing the risk of a mapping relationship falling into local optimum, and improving the accuracy of model inference.

Description

Method for image processing, method and device for training image processing model

[0001] The present application claims priority from the Chinese patent application No. CN202411310927.5 filed on September 19, 2024, and entitled "Method for image processing, method and device for training image processing model", the whole content of which is incorporated herein by reference. TECHNICAL FIELD

[0002] The present application relates to the field of artificial intelligence, in particular to a method for image processing, a method and device for training an image processing model. BACKGROUND

[0003] As a kind of pre-training model, when large visual model (LVM) is used for image analysis in downstream visual tasks, the LVM predicts target domain images as source domain labels, and then obtains the labels corresponding to samples by using the mapping relationship between source domain labels and target domain labels.

[0004] The creation of the mapping between source domain labels and target domain labels is directly based on the analysis results of the LVM, and the mapping relationship between the source domain label with the highest frequency corresponding to the target domain label and the target domain label is established to realize the transfer learning of the LVM. This mapping method does not need to adjust the parameters of the LVM, but the creation of the mapping relationship by using the greedy algorithm leads to the mapping relationship falling into local optimum. SUMMARY

[0005] The embodiments of the present application provide a method for image processing, a method and device for training an image processing model, which are used to avoid the mapping relationship of the model falling into local optimum.

[0006] In a first aspect, the present application provides a method for image processing, comprising:

[0007] obtaining a target sample;

[0008] processing the target sample based on the updated input transformation parameter to obtain a processed target sample;

[0009] analyzing the processed target sample to obtain a pre-training label, the pre-training label being a source domain label;

[0010] According to a mapping relationship between the pre-training label and the target label, the label corresponding to the target sample is determined as the target label, the mapping relationship between the pre-training label and the target label is included in the target mapping set, the target mapping set is a mapping relationship set with a minimum transmission cost, the transmission cost of the mapping relationship set is a sum of transmission costs corresponding to each mapping relationship in the mapping relationship set, the mapping relationship set is composed of mapping relationships between each label in the target domain label and the source domain label, and the target label is the target domain label.

[0011] In the embodiments of the application, the mapping relationship set with the minimum transmission cost is taken as the target mapping set, the transmission cost of the mapping relationship set is a sum of transmission costs corresponding to each mapping relationship in the mapping relationship set, the mapping relationship set includes mapping relationships between each label in the target domain label and the source domain label, each label in the target domain label corresponds to only one source domain label, and each source domain label corresponds to only one target domain label. In the implementation process of the visual prompt learning method, the target mapping set with the minimum transmission cost is taken as the output label mapping, the target mapping set is selected from the perspective of the total transmission cost of all mappings, the risk of the mapping relationship falling into a local optimum is effectively reduced, and the accuracy of model reasoning is improved.

[0012] In a possible implementation manner of the first aspect, before the label corresponding to the target sample is determined as the target label according to the mapping relationship between the pre-training label and the target label, the method further includes:

[0013] obtaining training samples and training labels, the training labels being included in S target domain labels, S being a positive integer;

[0014] processing the training samples based on the input transformation parameter to obtain processed training samples;

[0015] analyzing the processed training samples to obtain preset labels, the preset labels being included in N source domain labels, N being a positive integer, and N being greater than S;

[0016] determining the label corresponding to the training sample as a reference label based on a mapping relationship between the preset label and the reference label, the mapping relationship between the reference label and the preset label being included in the target mapping set;

[0017] updating the input transformation parameter based on a loss value of the training label and the reference label to obtain an updated input transformation parameter.

[0018] In the embodiment of the present application, the collected training samples are input transformed to obtain processed training samples, the processed training samples are analyzed by the visual pre-training model to obtain analysis results of the training samples in the source domain labels, and the label corresponding to the training sample is determined as the reference label based on the mapping relationship between the preset label and the reference label. Then, the data transformation parameter in the input transformation is updated based on the loss value of the training label and the reference label to obtain the updated input transformation parameter, the training of the image processing model is completed, the label mapping problem is converted into a partial optimal transmission problem, the target mapping set is located to minimize the transmission load, that is, the minimum carrying burden of the image processing model, and the one-to-one correspondence relationship between each label in the target domain label and the label in the source domain label is fully utilized. Linear programming is used to find the optimal solution of the mapping relationship set as the target mapping set.

[0019] In a possible implementation of the first aspect, before determining the label corresponding to the training sample as the reference label based on the mapping relationship between the preset label and the reference label, the method further includes:

[0020] Obtaining N source domain labels;

[0021] Obtaining the transmission cost of the mapping relationship, the transmission cost of the mapping relationship including the transmission cost of the mapping relationship between the first label and the second label, the first label being any one of the S target domain labels, the second label being any one of the N source domain labels, the second label corresponding to at most one of the S target domain labels, and the first label corresponding to one of the N source domain labels;

[0022] Based on the S target domain labels, the N source domain labels, and the transmission cost of the mapping relationship, analyzing the mapping relationship set with the minimum transmission cost to obtain the target mapping set.

[0023] In the embodiment of the present application, after obtaining the transmission cost (the transmission cost of the mapping relationship) between each label in the S target domain labels and the labels in the N source domain labels, based on the S target domain labels, the N source domain labels, and the transmission cost of the mapping relationship, the mapping relationship set with the minimum transmission cost is analyzed to obtain the target mapping set. Based on the transmission cost of the mapping relationship, a linear programming is constructed to find the optimal solution of the mapping relationship set, and the finding efficiency of the target mapping set is improved.

[0024] In a possible implementation of the first aspect, based on the target domain label, the N source domain labels, and the transmission cost of the mapping relationship, analyzing the mapping relationship set with the minimum transmission cost to obtain the target mapping set includes:

[0025] A mapping matrix of N*S dimensions is constructed, each row in the N rows includes at most one element 1 and at least S-1 elements 0, each column in the S columns includes one element 1 and N-1 elements 0, and the mapping matrix is used to represent the mapping relationship between the first label and the second label.

[0026] According to the transmission cost of the mapping relationship, a transmission cost matrix of N*S dimensions is constructed, and an element of a first coordinate in the transmission cost matrix is the transmission cost of the mapping relationship corresponding to the first coordinate in the mapping matrix;

[0027] The mapping relationship set with the minimum transmission cost is taken as the target mapping set based on the Frobenius inner product of the mapping matrix and the transmission cost matrix.

[0028] In the embodiments of the present application, the actual problem is converted into a mathematical problem, and the mapping relationship with the minimum transmission cost is found by using the Frobenius inner product of the mapping matrix and the transmission cost matrix, thereby effectively improving the efficiency of finding the target mapping set.

[0029] In a possible implementation of the first aspect, the transmission cost of the mapping relationship is obtained, including:

[0030] The preset label is analyzed to obtain a distribution of an analysis result of a reference sample in N source domain labels, the reference sample being at least two samples corresponding to a third label, and the third label being included in S target domain labels;

[0031] Based on the distribution of the analysis result of the reference sample in the S source domain labels, a distribution matrix of N*S dimensions is constructed;

[0032] According to the distribution matrix, the transmission cost of the second coordinate is calculated, the second coordinate representing a corresponding relationship between the first label and the second label, and the second coordinate being a coordinate in the distribution matrix.

[0033] In the embodiments of the present application, the distribution matrix of N*S dimensions is constructed by analyzing the preset label, so as to utilize the corresponding relationship between the preset label (source domain label) and the training label (target domain label). Moreover, the transmission cost of the second coordinate is calculated based on the distribution matrix, and the transmission cost of the second coordinate is converted into a mathematical problem, so that the analysis of the transmission cost of the second coordinate can be more efficiently and directly realized by mathematical operation.

[0034] In a possible implementation of the first aspect, the training sample after the analysis processing is obtained, and the preset label includes:

[0035] The training sample after the analysis processing is analyzed to obtain a first distribution probability, the first distribution probability being a probability that the label corresponding to the training sample after the processing is each label in the S source domain labels;

[0036] Based on the first distribution probability, a label with the highest probability in the S source domain labels is taken as the preset label;

[0037] A transmission cost matrix of N*S dimensions is constructed, including:

[0038] analyze the first distribution probability and the training label to obtain a distribution of an analysis result of a reference sample in N source domain labels, the reference sample being at least two samples corresponding to a third label, the third label being included in the S target domain labels;

[0039] construct an N*S-dimensional distribution matrix based on the distribution of the analysis result of the reference sample in the source domain labels;

[0040] calculate a transfer cost of a second coordinate according to the distribution matrix, the second coordinate representing a correspondence between the first label and the second label.

[0041] In the embodiments of the present application, when calculating the cost matrix, the output features of the model softmax layer are used as the basis for calculating the transfer cost matrix, rather than the classification results of the voting system. The prior experience of a finer granularity is used as the basis for constructing the transfer cost matrix, which can effectively improve the accuracy of the model.

[0042] In a possible implementation of the first aspect, the transfer cost of the second coordinate is the sum of the element values corresponding to all coordinates in the column of the second coordinate in the distribution matrix except the second coordinate, or the transfer cost of the second coordinate is the sum of the element values corresponding to all coordinates in the row of the second coordinate in the distribution matrix except the second coordinate.

[0043] In the embodiments of the present application, based on the distribution matrix, the sum of the elements corresponding to all coordinates in the row of the second coordinate in the distribution matrix except the second coordinate is used as the transfer cost of the second coordinate, and the earth-moving distance is calculated from the perspective of the target domain label, so as to evaluate the fitting difficulty between the source domain label corresponding to the second coordinate and the target domain label. Or, the sum of the elements corresponding to all coordinates in the column of the second coordinate in the distribution matrix except the second coordinate is used as the transfer cost of the second coordinate. The earth-moving distance is calculated from the perspective of the source domain label, which not only realizes the evaluation of the fitting difficulty between the source domain label corresponding to the second coordinate and the target domain label, but also ensures that each moving is related to the source domain label, effectively reduces unnecessary noise, and improves the reliability of the transfer cost of the second coordinate. At the same time, since in real use scenarios, the number of source domain labels often increases exponentially compared with the number of target domain labels, calculating the earth-moving distance from the perspective of the source domain label can also greatly reduce the calculation amount and improve the analysis efficiency.

[0044] The second aspect of the present application provides a training method of an image processing model, comprising:

[0045] obtain training samples and training labels, the training labels being included in S target domain labels, S being a positive integer;

[0046] process the training samples based on the input transformation parameters to obtain processed training samples;

[0047] analyzing the processed training samples to obtain preset labels, the preset labels being included in N source domain labels, N being a positive integer, and N being greater than S;

[0048] determining, based on a mapping relationship between the preset labels and reference labels, that a label corresponding to the training sample is the reference label, the mapping relationship between the preset labels and the reference labels being included in a target mapping set, the target mapping set being a mapping relationship set with minimum transmission cost, the transmission cost of the target mapping set being a sum of transmission costs corresponding to each mapping relationship in the target mapping set, and the mapping relationship set being composed of mapping relationships between each label in the target domain labels and the source domain labels;

[0049] updating the input transformation parameter based on a loss value of the training label and the reference label to obtain an updated input transformation parameter.

[0050] In a possible implementation manner of the second aspect, before determining, based on a mapping relationship between the preset labels and reference labels, that a label corresponding to the training sample is the reference label, the method further includes:

[0051] obtaining N source domain labels;

[0052] obtaining transmission costs of the mapping relationships, the transmission cost of the mapping relationship including a transmission cost corresponding to a mapping relationship between a first label and a second label, the first label being any one of the S target domain labels, the second label being any one of the N source domain labels, the second label corresponding to at most one of the S target domain labels, and the first label corresponding to one of the N source domain labels;

[0053] analyzing, based on the S target domain labels, the N source domain labels, and the transmission costs of the mapping relationships, a mapping relationship set with minimum transmission cost to obtain the target mapping set.

[0054] In a possible implementation manner of the second aspect, analyzing, based on the target domain labels, the N source domain labels, and the transmission costs of the mapping relationships, a mapping relationship set with minimum transmission cost to obtain the target mapping set includes:

[0055] constructing an N*S-dimensional mapping matrix, each row in the N rows including at most one element 1 and at least S-1 elements 0, and each column in the S columns including one element 1 and N-1 elements 0, the mapping matrix being used to represent a mapping relationship between a first label and a second label;

[0056] constructing an N*S-dimensional transmission cost matrix according to the transmission costs of the mapping relationships, an element at a first coordinate in the transmission cost matrix being a transmission cost of a mapping relationship corresponding to the first coordinate in the mapping matrix;

[0057] taking, as the target mapping set, the mapping relationship set with minimum transmission cost based on a Frobenius inner product of the mapping matrix and the transmission cost matrix.

[0058] In a possible implementation of the second aspect, the transmission cost of the mapping relationship is obtained by:

[0059] The preset label is analyzed to obtain a distribution of an analysis result of the reference sample in the N source domain labels, the reference sample being at least two samples corresponding to a third label, the third label being included in the S target domain labels;

[0060] A distribution matrix of N*S dimensions is constructed based on the distribution of the analysis result of the reference sample in the N source domain labels;

[0061] The transmission cost of the second coordinate is calculated according to the distribution matrix, the second coordinate representing a corresponding relationship between the first label and the second label, the second coordinate being a coordinate in the distribution matrix.

[0062] In a possible implementation of the second aspect, the analysis of the processed training sample to obtain the preset label comprises:

[0063] The processed training sample is analyzed to obtain a first distribution probability, the first distribution probability being a probability that a label corresponding to the processed training sample is each label in the S source domain labels;

[0064] The transmission cost of the mapping relationship is obtained by:

[0065] The first distribution probability and the training label are analyzed to obtain a distribution of an analysis result of the reference sample in the N source domain labels, the reference sample being at least two samples corresponding to a third label, the third label being included in the S target domain labels;

[0066] A distribution matrix of N*S dimensions is constructed based on the distribution of the analysis result of the reference sample in the source domain labels;

[0067] The transmission cost of the second coordinate is calculated according to the distribution matrix, the second coordinate representing a corresponding relationship between the first label and the second label.

[0068] In a possible implementation of the second aspect, the transmission cost of the second coordinate is a sum of element values of all coordinates except the second coordinate in a column in which the second coordinate is located in the distribution matrix, or the transmission cost of the second coordinate is a sum of element values of all coordinates except the second coordinate in a row in which the second coordinate is located in the distribution matrix.

[0069] The third aspect of the present application provides an image processing device, comprising:

[0070] An acquisition unit is configured to acquire a target sample;

[0071] A processing unit is configured to process the target sample based on the updated input transformation parameter to obtain a processed target sample;

[0072] an analysis unit, configured to analyze the processed target sample to obtain a pre-training label, the pre-training label being a source domain label;

[0073] The processing unit is further configured to determine, according to a mapping relationship between the pre-training label and the target label, that the label corresponding to the target sample is the target label, the mapping relationship between the pre-training label and the target label being included in a target mapping set, the target mapping set being a mapping relationship set with minimum transmission cost, the transmission cost of the mapping relationship set being a sum of transmission costs corresponding to each mapping relationship in the mapping relationship set, the mapping relationship set being composed of mapping relationships between each label in the target domain label and the source domain label, and the target label being a target domain label.

[0074] The fourth aspect of the present application provides a training device of an image processing model, comprising:

[0075] The acquisition unit is configured to acquire a training sample and a training label, the training label being included in S target domain labels, S being a positive integer;

[0076] The processing unit is configured to process the training sample based on an input transformation parameter to obtain a processed training sample;

[0077] The analysis unit is configured to analyze the processed training sample to obtain a preset label, the preset label being included in N source domain labels, N being a positive integer, and N being greater than S;

[0078] The processing unit is further configured to determine, based on a mapping relationship between the preset label and a reference label, that the label corresponding to the training sample is the reference label, the mapping relationship between the preset label and the reference label being included in the target mapping set, the target mapping set being a mapping relationship set with minimum transmission cost, the transmission cost of the target mapping set being a sum of transmission costs corresponding to each mapping relationship in the target mapping set, and the mapping relationship set being composed of mapping relationships between each label in the target domain label and the source domain label.

[0079] The processing unit is further configured to update the input transformation parameter based on a loss value of the training label and the reference label to obtain an updated input transformation parameter.

[0080] The fifth aspect of the present application provides a computer device, comprising a processor and a memory, the processor storing instructions, when the instructions stored on the memory are run on the processor, the method shown in the first aspect, any possible implementation manner of the first aspect, the second aspect or any possible implementation manner of the second aspect is implemented.

[0081] The sixth aspect of the present application provides a computer readable storage medium, and the computer readable storage medium stores instructions. When the instructions are executed on a processor, the method shown in the first aspect, any possible implementation of the first aspect, the second aspect, or any possible implementation of the second aspect is implemented.

[0082] The seventh aspect of the present application provides a computer program product. When the computer program product is executed on a processor, the method shown in the first aspect, any possible implementation of the first aspect, the second aspect, or any possible implementation of the second aspect is implemented.

[0083] The beneficial effects of any one of the fifth aspect to the seventh aspect are similar to those of the first aspect, any possible implementation of the first aspect, the second aspect, or any possible implementation of the second aspect, which will not be repeated here. BRIEF DESCRIPTION OF DRAWINGS

[0084] FIG. 1 is a schematic diagram of a training method of an image processing model provided by the present application;

[0085] FIG. 2 is a schematic diagram of an analysis of a distribution matrix provided by the present application;

[0086] FIG. 3 is another schematic diagram of a training method of an image processing model provided by the present application;

[0087] FIG. 4 is another schematic diagram of an analysis of a distribution matrix provided by the present application;

[0088] FIG. 5 is a schematic diagram of experimental results provided by the present application;

[0089] FIG. 6 is another schematic diagram of experimental results provided by the present application;

[0090] FIG. 7 is a schematic diagram of a flow of an image processing method provided by the present application;

[0091] FIG. 8 is a schematic diagram of an image processing model provided by an embodiment of the present application;

[0092] FIG. 9 is a schematic diagram of a structure of an image processing device provided by the present application;

[0093] FIG. 10 is a schematic diagram of a structure of a training device of an image processing model provided by the present application;

[0094] FIG. 11 is a schematic diagram of a structure of a computing device provided by the present application. DETAILED DESCRIPTION

[0095] The embodiments of the present application provide a method of image processing, a training method and device of an image processing model, which are used to avoid the mapping relationship of the model from falling into a local optimum.

[0096] The embodiments of the present application are described below with reference to the accompanying drawings. Those skilled in the art can know that with the development of technology and the emergence of new scenarios, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems.

[0097] The terms "first", "second", and the like in the specification and claims of the present application and the above-described drawings are used to distinguish similar objects, and do not necessarily have to be used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchanged under appropriate circumstances, and this is only a way of distinguishing the objects with the same attributes in the description of the embodiments of the present application. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, so that the processes, methods, systems, products or equipment containing a series of units do not have to be limited to those units, but can include other units not clearly listed or inherent to these processes, methods, products or equipment. In addition, "at least one" means one or more, and "multiple" means two or more. The term "and / or" describes the relationship between the associated objects, which means that there can be three relationships, for example, A and / or B, which can represent the following cases: A exists alone, A and B exist together, and B exists alone, where A and B can be singular or plural. The character " / " generally represents an "or" relationship between the associated objects. "At least one of the following" or similar expressions means any combination of these items, including any combination of single or multiple items. For example, at least one of a, b, or c can represent a, b, c, a-b, a-c, b-c, or a-b-c, where a, b, and c can be single or multiple.

[0098] The visual pre-training model, such as the large visual model, can be applied to downstream visual tasks by fine tuning the visual pre-training model or creating a mapping relationship from the source domain label to the target domain label to realize the application of the visual pre-training model in different data sets.

[0099] In specific application scenarios, the output label mapping of the visual pre-training model can be realized by visual prompt learning, that is, the output label of the visual pre-training model is migrated from the source domain label to the target domain label. The source domain label is all possible output labels of the visual pre-training model, and the target domain label is all possible labels of the data set. The target domain label can be included in the source domain label, or the target domain label can not be included in the target domain label.

[0100] The visual prompt learning method is a method for applying a visual pre-training model to a downstream visual task, which includes input transformation, pre-training classifier analysis, and output label mapping.

[0101] In the creation process of the output label mapping, the frequency of predicting the corresponding source domain label based on the visual pre-training model is calculated, and the target domain label corresponding to the highest frequency is assigned to the target domain label. This mapping method does not need to adjust the parameters of the visual pre-training model, but directly based on the analysis result of the visual pre-training model, the mapping relationship between the target domain label and the source domain label can be created, and the transfer learning of the visual pre-training model can be realized. However, when creating the output label mapping, the greedy algorithm is used to create the mapping relationship, which causes the mapping relationship created by the model to fall into a local optimum.

[0102] To solve this problem, the present application proposes that the mapping relationship set with the minimum transmission cost can be used as the target mapping set, and the transmission cost of the mapping relationship set is the sum of the transmission costs of each mapping relationship in the mapping relationship set. Each label in the target domain label corresponds to only one source domain label, and each source domain label corresponds to only one target domain label. In the implementation process of the visual prompt learning method, the target mapping set with the minimum transmission cost is used as the output label mapping, and the target mapping set is selected from the total transmission cost of all mappings, which effectively reduces the risk of the mapping relationship falling into a local optimum and improves the accuracy of model reasoning.

[0103] For ease of understanding, first introduce the training method of the image processing model provided by the present application from the perspective of training.

[0104] Please refer to FIG. 1, which is a schematic diagram of the training method of the image processing model provided by the present application.

[0105] 101, obtain training samples and training labels;

[0106] Wherein, the training label is contained in S target domain labels, and S is a positive integer.

[0107] Specifically, the training sample is a training sample determined based on the target sample. For example, the trained image processing model will be applied to the animal classification scene, and the S target domain labels can include swallow, sparrow, crow, parrot, etc. The training label can include swallow, sparrow, crow or parrot, etc.

[0108] In this case, the training sample can include pictures or video frames in which swallows appear, etc., which is not limited here.

[0109] 102, process the training sample based on the input transformation parameter to obtain the processed training sample;

[0110] The input transformation module of the image processing model is configured to add a pixel-level prompt around the training sample, and the input transformation module adjusts the input transformation parameter to add different pixel-level prompts around the training sample. The characteristics of the input sample are modified to match the feature distribution of the training sample (target domain) with the source domain feature distribution.

[0111] 103. Analyzing the processed training sample to obtain a preset label;

[0112] The preset label is included in N source domain labels, N is a positive integer, and N is greater than S.

[0113] Specifically, the visual pre-training model is used to analyze the processed training sample to obtain a preset label. The N source domain labels are N labels that the visual pre-training model can obtain by analyzing the sample. It should be noted that the N source domain labels can be related to the S target domain labels, or can be unrelated to the S target domain labels. This is not limited here.

[0114] For example, when training the pre-training model, a relatively wide data set is used. Therefore, the N source domain labels may partially overlap with the S target domain labels. Alternatively, due to the relatively small application scenarios of the image processing model, the S target domain labels and the N source domain labels can have no overlapping part. The present application takes the case where the S target domain labels and the N source domain labels do not overlap as an example for illustration.

[0115] For example, the N source domain labels can include chrysanthemum, rose, carnation, sunflower, and lily.

[0116] 104. Obtain N source domain labels;

[0117] Specifically, since the degree of overlap between the N source domain labels and the S target domain labels is limited, the N source domain labels are obtained to obtain all possible output labels of the visual pre-training model.

[0118] It should be noted that step 104 can be implemented before step 105. There is no specific order between steps 104 and steps 101 to 103. The description here is only an example. In actual applications, it should be combined with the specific application scenario to set it up. This is not limited here.

[0119] 105. Analyzing the preset label to obtain the distribution of the analysis result of the reference sample in the N source domain labels;

[0120] The reference sample is at least two samples corresponding to a third label, the third label is included in the S target domain labels, the reference sample is included in the training sample, and the third label is included in the training label.

[0121] Specifically, since the preset label includes a result obtained by analyzing each training sample in the plurality of training samples via the visual pre-training model, the plurality of training samples can be classified into S different categories according to the analysis result of each training sample in the plurality of training samples via the visual pre-training model. Any label in the S target domain labels is taken as a third label, and at least two samples corresponding to the third label are taken as reference samples.

[0122] For example, in order to facilitate the subsequent use of mathematical language to describe the scheme provided in the present application, p is used to represent the distribution of the source domain label, p contains N categories, q is used to represent the distribution of the target domain label, q contains S categories, wherein N and S are positive integers, and N is greater than S according to the performance of the target domain label and the source domain label in the actual scene.

[0123] In order to facilitate understanding, S target domain labels and N source domain labels are taken as an example, and the scheme provided in the present application is introduced in combination with FIG. 2.

[0124] The reference samples are input into the transformation and the visual pre-processing model for analysis to obtain labels corresponding to the reference samples. Since the labels corresponding to the reference samples are source domain labels, the N source domain labels can be voted based on the labels corresponding to the reference samples. Each occurrence of a label corresponding to a reference sample is taken as label 1, and the value corresponding to label 1 in the N source domain labels is calculated by one. The distribution of the analysis result of the reference samples in the N source domain labels can be a data table, or a 1*N matrix, or an array, and the like, which is not limited here.

[0125] It can be understood that the description of the distribution of the analysis result of the reference samples in the N source domain labels is only an example. In actual application, each label in the target domain label can be taken as a third label in turn to complete the analysis of the samples corresponding to each label in the source domain label, and the distribution of the analysis result of the S groups of reference samples in the N source domain labels can be obtained, which is not limited here.

[0126] 106、Based on the distribution of the analysis result of the reference samples in the N source domain labels, an N*S-dimensional distribution matrix is constructed;

[0127] Specifically, the present application converts the label mapping problem into a partially optimal transmission problem. Here, the performance form of the distribution of the analysis result of the reference samples in the N source domain labels is a 1*N matrix.

[0128] In step 105, the distribution of the analysis result of the S groups of reference samples in the N source domain labels can be obtained. The combination of the distribution of the analysis result of the S groups of reference samples in the N source domain labels can obtain an N*S-dimensional distribution matrix.

[0129] Exemplarily, each row in the distribution matrix corresponds to a target domain label, and each column corresponds to a source domain label. Assuming that the labels corresponding to at least two reference images are the third label, in the row corresponding to the third label, the element corresponding to each coordinate can represent the number of times each source domain label appears in the label obtained by processing the at least two reference images by the visual preprocessing model.

[0130] Assuming that the third label is sparrow, in the label obtained by processing the at least two reference images by the visual preprocessing model, there are 1 chrysanthemum, 1 rose, 0 carnation, 5 sunflower and 2 lily. When the third label is swallow, crow or parrot, the distribution of the analysis result of the reference sample in the S source domain labels is similar to the analysis of the at least two reference images corresponding to the third label, and the distribution matrix shown in FIG. 2 is obtained by integration.

[0131] 107. According to the distribution matrix, the transmission cost of the second coordinate is calculated and obtained;

[0132] The second coordinate represents the corresponding relationship between the first label and the second label, the second coordinate is a coordinate in the distribution matrix, the first label is any one of the S target domain labels, and the second label is any one of the N source domain labels.

[0133] Specifically, in order to find the optimal solution of the mapping relationship set, the present application introduces the earth mover's distance (EMD) based on the distribution matrix to help evaluate the difficulty of fitting the source domain label distribution and the target domain label distribution. The transmission cost of the second coordinate is also the distribution difference between the first label and the second label, that is, the difficulty of fitting the first label and the second label distribution. The transmission cost of the mapping relationship set is the sum of the transmission cost corresponding to each mapping relationship in the mapping relationship set, and a lower transmission cost represents a lower fitting difficulty.

[0134] Therefore, in the scheme provided by the present application, the earth mover's distance of the first label and the second label is used to define the transmission cost of the second coordinate, and the specific calculation method is as follows:

[0135] Wherein, EMD is the earth mover's distance calculation, f i is the feature of the input image i, T is the result of label mapping, h t is the N-dimensional one-hot vector representation of the source domain label t.

[0136] In combination with the specific scenario in the present application, in order to ensure the one-to-one mapping relationship between the source domain label and the target domain label, the number of source domain labels should be at least equal to the number of target domain labels. In a specific application scenario, since the number of samples in the training set of the visual pre-training model covers all aspects, and the image processing model is usually a model applied to a specific field, the present application assumes that the number of source domain labels is greater than the number of target domain labels, that is, N is greater than S.

[0137] Based on the above idea, the present application defines a transmission cost calculation method suitable for the label mapping strategy based on the minimum transmission algorithm. The present application provides two ideas for calculating the transmission cost, which will be introduced as follows:

[0138] Idea one: transporting the elements corresponding to all coordinates except the second coordinate in the row where the second coordinate is located in the distribution matrix;

[0139] Since the mapping relationship between the target domain label represented by the first row and the source domain label represented by the third column needs to be created, the mapping relationship between the target domain label represented by the first row and the source domain label represented by other columns except the third column needs to be migrated to the mapping relationship between the target domain label represented by the first row and the source domain label represented by the third column.

[0140] In this case, the calculation method of the earth moving distance is as follows:

[0141] Using C i,j represents the unit transmission cost from the source domain label i to the target domain label j, where i is included in the category p and j is included in the category q, the second coordinate is (i, j), and the transmission cost of the second coordinate is calculated as follows:

[0142] Where f l',i is an element corresponding to any coordinate in the column where the second coordinate is located in the distribution matrix, except the second coordinate, l' is the target domain label corresponding to the element, C i is the coordinate of the target domain label i.

[0143] Still taking the distribution matrix shown in FIG. 2 as an example, assume that the second coordinate is (1, 3).

[0144] Based on the above idea, the transmission cost of the second coordinate is the sum of the element values of the other columns except the third column in the first row of the distribution matrix. That is, the transmission cost of the second coordinate is the sum of the element values corresponding to all coordinates except the second coordinate in the row where the second coordinate is located in the distribution matrix.

[0145] In the embodiments of the present application, based on the distribution matrix, the sum of the elements corresponding to all coordinates except the second coordinate in the row where the second coordinate is located in the distribution matrix is taken as the transmission cost of the second coordinate. The calculation of the earth moving distance is performed from the perspective of the target domain label, so as to realize the evaluation of the fitting difficulty between the source domain label corresponding to the second coordinate and the target domain label. A reliable basis is provided for the subsequent calculation of the partial optimal transmission.

[0146] Thought two: transport the elements corresponding to all coordinates except the second coordinate in the column where the second coordinate is located in the distribution matrix.

[0147] The scheme provided by the above thought one fully considers the possible influence of all source domain categories when analyzing the transmission cost of the second coordinate. However, not all source domain labels are related to the target domain label. From the perspective of the target domain label, the possible mapping relationship between the target domain label and the source domain label is introduced, which introduces part of irrelevant noise. In order to eliminate this noise as much as possible, the present application proposes that the possible mapping relationship between the source domain label and the target domain label can be transported from the perspective of the source domain label.

[0148] Based on the above thought, the mapping relationship between the target domain label represented by the first row and the source domain label represented by the third column needs to migrate the mapping relationship between the source domain label represented by the third column and the source domain label represented by the row other than the first row to the mapping relationship between the target domain label represented by the first row and the source domain label represented by the third column.

[0149] In this case, the calculation method of the earth moving distance is as follows:

[0150] Use C i,j to represent the unit transmission cost from the source domain label i to the target domain label j, wherein i is included in the category p and j is included in the category q, the second coordinate is (i, j), and the transmission cost of the second coordinate is calculated as follows:

[0151] Wherein, f k',i is an element corresponding to any coordinate in the column where the second coordinate is located in the distribution matrix, k' is the target domain label corresponding to the element, C j is a coordinate with j as the target domain label.

[0152] Still taking the distribution matrix shown in the foregoing FIG. 2 as an example, it is assumed that the second coordinate is (1, 3).

[0153] Based on the above idea, the transmission cost of the second coordinate is the sum of the element values of the third column of the distribution matrix except the first row. That is, the transmission cost of the second coordinate is the sum of the element values corresponding to all coordinates except the second coordinate in the column where the second coordinate is located in the distribution matrix.

[0154] In the embodiments of the present application, based on the distribution matrix, the sum of the elements corresponding to all coordinates except the second coordinate in the column where the second coordinate is located in the distribution matrix is taken as the transmission cost of the second coordinate. The calculation of the earth-moving distance from the perspective of the source domain label ensures that each move is related to the source domain label, effectively reduces unnecessary noise, and improves the reliability of the transmission cost of the second coordinate. At the same time, since in real use scenarios, the number of source domain labels often increases exponentially compared to target domain labels, calculating the earth-moving distance from the perspective of the source domain label can also greatly reduce the computational complexity and improve the analysis efficiency.

[0155] In the embodiments of the present application, by analyzing the preset label, the corresponding relationship between the preset label (source domain label) and the training label (target domain label) is used to construct an N*S-dimensional distribution matrix. And based on the distribution matrix, the transmission cost of the second coordinate is calculated. The transmission cost of the second coordinate is converted into a mathematical problem, which can be more efficient and direct to realize the analysis of the transmission cost of the second coordinate through mathematical operations.

[0156] It can be understood that the explanation of the second coordinate here is only an example. In actual application, the transmission cost corresponding to any coordinate in the distribution matrix can be obtained by a method similar to the method of calculating the transmission cost of the second coordinate, which is not limited here.

[0157] 108、According to the transmission cost of the mapping relationship, an N*S-dimensional transmission cost matrix is constructed;

[0158] Among them, the transmission cost of the mapping relationship includes the transmission cost corresponding to the mapping relationship between the first label and the second label, and the transmission cost of the second coordinate is the transmission cost corresponding to the mapping relationship between the first label and the second label.

[0159] Based on the description in the foregoing step 107, in the present application, the plurality of transmission costs included in the transmission cost of the mapping relationship are all obtained by a method similar to the method of calculating the transmission cost of the second coordinate. Therefore, the transmission cost of the mapping relationship is also the transmission cost of the second coordinate.

[0160] Specifically, based on the transmission cost of the second coordinate calculated in the foregoing step 107, the transmission cost of the second coordinate is filled into the coordinate position corresponding to the second coordinate in the N*S-dimensional matrix according to the position of the second coordinate, and an N*S-dimensional transmission cost matrix is obtained.

[0161] 109. constructing a mapping matrix of N*S dimensions;

[0162] wherein the mapping matrix is used to represent that each row in the N rows of the first label and the second label includes one element 1 and at least S-1 elements 0, and each column in the S columns includes one element 1 and N-1 elements 0, the mapping matrix is a mathematical representation of the mapping relationship set, in the present application, T is used to represent the mapping matrix.

[0163] Since the mapping matrix and the transmission cost matrix are both N*S dimensional matrices, the element of the first coordinate in the transmission cost matrix is the transmission cost of the mapping relationship corresponding to the first coordinate in the mapping matrix. The row and column of the mapping matrix are the same as the mapping relationship corresponding to the row and column in the transmission cost matrix.

[0164] Specifically, sampling is performed from the training samples to define the distribution between the target domain label domain and the source domain label in the mapping relationship set, (p, q)∈∑ N ×∑ S ,∑ N and are histograms composed of N and S components, respectively.

[0165] wherein p and q are represented as follows, respectively:

[0166] wherein δ f,j is the Dirac function at position . M is the total number of sample sets of the input data set of the downstream task, that is, the total number of training samples. f ij is the jth component of the feature f i , and f i is the feature of the input image i. p i and p j are the probabilities of the jth element of the feature f i . In the VP-LM framework, each target domain label is associated with a source domain label, and for the task of class-balanced data Σ j p j =1, and

[0167] For example, the mapping relationship set between two discrete probability measures p and q can be defined as:

[0168] wherein 1 d is a d-dimensional vector with all elements being 1, and T = (T i,j ) i,j represents the mapping matrix.

[0169] 110、based on the Frobenius inner product of the mapping matrix and the transmission cost matrix, taking the mapping relationship with the minimum transmission cost as the target mapping set;

[0170] The mapping matrix represents the mapping relationship between the S target domain labels and the N source domain labels, and the transmission cost matrix is a mathematical representation of the transmission cost of the mapping relationship. Therefore, based on the Frobenius inner product of the mapping matrix and the transmission cost matrix, the transmission cost of the mapping relationship set is calculated, and the mapping relationship with the minimum transmission cost is taken as the target mapping set, which is a specific implementation means for analyzing the mapping relationship set with the minimum transmission cost based on the S target domain labels, the N source domain labels, and the transmission cost of the mapping relationship.

[0171] Specifically, the present application evaluates the transmission cost of the mapping relationship set by calculating the Frobenius inner product of the mapping matrix and the transmission cost matrix, ensuring that the mapping relationship set with the minimum transmission cost can be found from multiple mapping relationship sets that meet the mapping matrix condition, and the mapping relationship set with the minimum transmission cost is taken as the target mapping set.

[0172] For example, based on the above introduction, the optimal transmission in p and q can be realized by the following way:

[0173] wherein, T i,j represents the transmission cost from the source domain label to the target domain label, C i,j represents the unit transmission cost from class i∈p to class j∈q, is the cost matrix. <·,·> F is the Frobenius inner product.

[0174] In this case, the target mapping set can be solved by the explicit programming algorithm, which is shown as follows:

[0175] wherein, T0 is the target mapping set.

[0176] It can be understood that the above description of the solution of the target mapping set is only an example, and in actual application, it should be set according to the specific application scenario, which is not limited here.

[0177] In the embodiment of the present application, the actual problem is converted into a mathematical problem, and the mapping relationship with the minimum transmission cost is found by using the Frobenius inner product of the mapping matrix and the transmission cost matrix, which effectively improves the efficiency of finding the target mapping set.

[0178] 111、based on the mapping relationship between the preset label and the reference label, determining the label corresponding to the training sample as the reference label;

[0179] The mapping relationship between the preset label and the reference label is included in the target mapping set, the preset label is included in the source domain label, and the reference label is included in the target domain label.

[0180] Specifically, after the visual pre-training model analyzes the input sample and obtains the preset label, the target mapping set is searched to obtain the reference label having the mapping relationship with the preset label, and the reference label is taken as the label corresponding to the training sample.

[0181] 112. Update the input transformation parameter based on the loss value of the training label and the reference label to obtain an updated input transformation parameter.

[0182] After the target mapping set is determined, the input transformation parameter can be updated based on the difference between the training label and the reference label to obtain an updated input transformation parameter.

[0183] Specifically, the input transformation parameter can be updated by using the following loss function Loss:

[0184] X is the input sample, that is, the training sample, y is the training label, θ is the trainable parameter, h(X, θ) is the processed training sample obtained by input transformation on the training sample, t t p t p is the preset label output by the visual and training model, is the mapping relationship between the preset label and the reference label, is the addition loss of the image processing model.

[0185] It can be understood that the description of the calculation method of the loss value of the input transformation parameter herein is only an example, and in actual application, the specific application scenario should be combined for setting, which is not limited herein.

[0186] ​​​​​In the embodiment of the present application, in the training stage, the collected training samples are input transformed to obtain processed training samples, and then the processed training samples are analyzed by the visual pre-training model to obtain the analysis result of the training samples in the source domain label, and the label corresponding to the training sample is determined as the reference label based on the mapping relationship between the preset label and the reference label, and then the data transformation parameter in the input transformation is updated based on the loss value of the training label and the reference label to obtain the updated input transformation parameter, complete the training of the image processing model, convert the label mapping problem into a partial optimal transmission problem, and locate the target mapping set to minimize the transmission load, that is, the minimum carrying burden of the image processing model, and fully utilize the one-to-one correspondence relationship between each label in the target domain label and the label in the source domain label, and use linear programming to find the optimal solution of the mapping relationship set as the target mapping set.

[0187] Figure 1 introduces one embodiment of the training method of the image processing model provided by the present application, and another embodiment of the training method of the image processing model provided by the present application will be introduced below in combination with Figure 3.

[0188] Please refer to Figure 3, which is another flowchart of the training method of the image processing model provided by the present application.

[0189] 301, obtain training samples and training labels;

[0190] 302, process the training samples based on the input transformation parameter to obtain processed training samples;

[0191] Steps 301 and 302 are similar to steps 101 and 102 in the aforementioned Figure 1, and the specific operations are described in the aforementioned steps 101 and 102, which will not be repeated here.

[0192] 303, obtain N source domain labels;

[0193] Step 303 is similar to step 104 in the aforementioned Figure 1, and the specific operation is described in the aforementioned step 104, which will not be repeated here.

[0194] It should be noted that there is no clear sequence between step 303 and step 301, step 302, and the description here is only an example. In actual application, it should be set according to the specific application scene, which is not limited here.

[0195] 304, analyze the processed training samples to obtain a first distribution probability;

[0196] The first distribution probability is the probability that the label corresponding to the processed training sample is each label in the N source domain labels.

[0197] Specifically, the visual and processing model does not need to complete the normalization processing of the processed training sample, and directly outputs the distribution probability of each label in the S source domain labels.

[0198] 305、analyzing the first distribution probability and the training label to obtain the distribution of the analysis result of the reference sample in the N source domain labels;

[0199] The reference sample is at least two samples corresponding to the third label, the third label is included in the S target domain labels, the reference sample is included in the training sample, and the third label is included in the training label.

[0200] Specifically, since the first distribution probability includes the distribution probability of each training sample in the source domain label obtained by analyzing the training sample via the visual pre-training model, according to the plurality of training labels, the plurality of training samples can be divided into S different categories. Any label in the S target domain label is taken as the third label, and at least two samples corresponding to the third label are taken as the reference sample.

[0201] For example, in order to facilitate the subsequent use of mathematical language to describe the scheme provided by the present application, p is used to represent the distribution of the source domain label, p contains N categories, q is used to represent the distribution of the target domain label, q contains S categories, wherein N and S are positive integers, and according to the performance of the target domain label and the source domain label in the actual scene, it can be assumed that N is greater than S.

[0202] In order to facilitate understanding, taking S target domain labels and N source domain labels as an example, the scheme provided by the present application is introduced in combination with FIG. 4.

[0203] The reference sample is input into the transformation and visual preprocessing model to obtain the first distribution probability corresponding to the reference sample. The first distribution probability corresponding to the reference sample can be in the form of a 1*N matrix as shown in FIG. 4. The 1*N matrix corresponding to the first distribution probability of the at least two reference samples is subjected to matrix addition operation to obtain the distribution of the analysis result of the reference sample in the N source domain labels.

[0204] It can be understood that the description of the distribution of the analysis result of the reference sample in the N source domain labels is only an example, and in actual application, it should be set in combination with the specific use scene, which is not limited here.

[0205] 306、based on the distribution of the analysis result of the reference sample in the N source domain labels, an N*S dimensional distribution matrix is constructed;

[0206] Specifically, the label mapping problem is converted into a partially optimal transmission problem, and the distribution of the analysis result of the reference sample in the N source domain labels is represented as a 1*N matrix.

[0207] In step 305, the distribution of the analysis result of the S set of reference samples in the N source domain labels is obtained, and the distribution of the analysis result of the S set of reference samples in the N source domain labels is combined, that is, a distribution matrix of N*S dimensions is obtained.

[0208] For example, each row in the distribution matrix corresponds to a target domain label, and each column corresponds to a source domain label. Assuming that the labels corresponding to at least two reference images are the third label, in the row corresponding to the third label, the elements corresponding to each coordinate can represent the number of times each source domain label appears in the label obtained by processing the at least two reference images by the visual preprocessing model.

[0209] For example, assuming that the third label is sparrow, after the analysis of the at least two reference images by the visual preprocessing model, the label obtained includes 2.5 chrysanthemums, 0.3 roses, 4.2 carnations, 1.3 sunflowers, and 0.6 lilies. When the third label is a swallow, a raven, or a parrot, the distribution of the analysis result of the reference sample in the S source domain labels is calculated in the same way as the analysis of the at least two reference images corresponding to the third label, and the distribution matrix shown in FIG. 4 is obtained by integration.

[0210] 307、According to the distribution matrix, the transmission cost of the second coordinate is calculated and obtained;

[0211] The second coordinate represents the corresponding relationship between the first label and the second label, the second coordinate is a coordinate in the distribution matrix, the first label is any one of the S target domain labels, and the second label is any one of the N source domain labels.

[0212] Specifically, in order to find the optimal solution of the mapping relationship set, the present application helps to evaluate the difficulty of fitting the source domain label distribution and the target domain label distribution based on the earth moving distance. The transmission cost of the second coordinate is the distribution difference between the first label and the second label, that is, the difficulty of fitting the first label and the second label distribution. The transmission cost of the mapping relationship set is the sum of the transmission cost corresponding to each mapping relationship in the mapping relationship set. A lower transmission cost represents a lower fitting difficulty, that is, the training can be completed using fewer training rounds, effectively improving the training speed of the model.

[0213] Therefore, in the scheme provided by the present application, the earth moving distance of the first label and the second label is used to define the transmission cost of the second coordinate, and the specific calculation method is as follows:

[0214] wherein EMD is the earth moving distance calculation, f i is the feature of the input image i, T is the result of the label mapping, h t is the S-dimensional one-hot vector representation of the target domain label t.

[0215] In combination with the specific scenario in the present application, in order to ensure the one-to-one mapping relationship between the source domain label and the target domain label, the number of source domain labels should be at least equal to the number of target domain labels. In a specific application scenario, since the number of samples in the training set of the visual pre-training model covers all aspects, and the image processing model is usually a model applied to a specific field, the present application assumes that the number of source domain labels is greater than the number of target domain labels, that is, N is greater than S.

[0216] Based on the above idea, the present application defines a transmission cost calculation method suitable for the label mapping strategy based on the minimum transmission algorithm. The present application provides two ideas for calculating the transmission cost, which will be introduced as follows:

[0217] Idea one: transport the elements corresponding to all coordinates except the second coordinate in the row where the second coordinate is located in the distribution matrix;

[0218] Since the mapping relationship between the target domain label represented by the first row and the source domain label represented by the third column needs to be created, the mapping relationship between the target domain label represented by the first row and the source domain label represented by the column other than the third column needs to be migrated to the mapping relationship between the target domain label represented by the first row and the source domain label represented by the third column.

[0219] In this case, the calculation method of the earth moving distance of the second coordinate is as follows:

[0220] wherein j represents the target domain label, t is the first label (the target domain label corresponding to the second coordinate).

[0221] C i,j represents the unit transmission cost from the source domain label i to the target domain label j, wherein i is included in the category p and j is included in the category q, the second coordinate is (i, j), and the transmission cost of the second coordinate is calculated as follows:

[0222] wherein EMD is the earth moving distance calculation, f l',i is the element corresponding to any coordinate other than the second coordinate in the column where the second coordinate is located in the distribution matrix, l' is the target domain label corresponding to the element, C i is the coordinate whose target domain label is i.

[0223] Still taking the distribution matrix shown in FIG. 2 as an example, it is assumed that the second coordinate is (1, 3).

[0224] Based on the above idea, the transmission cost of the second coordinate is the sum of the element values of the columns other than the third column in the first row of the distribution matrix. That is, the transmission cost of the second coordinate is the sum of the element values corresponding to all coordinates other than the second coordinate in the row where the second coordinate is located in the distribution matrix.

[0225] In the embodiments of the present application, based on the distribution matrix, the sum of the elements corresponding to all coordinates other than the second coordinate in the row where the second coordinate is located in the distribution matrix is taken as the transmission cost of the second coordinate. The calculation of the earth moving distance is carried out from the perspective of the target domain label, so as to realize the evaluation of the fitting difficulty between the source domain label corresponding to the second coordinate and the target domain label. A reliable basis is provided for the subsequent calculation of partial optimal transmission.

[0226] Idea two: transport the elements corresponding to all coordinates other than the second coordinate in the column where the second coordinate is located in the distribution matrix.

[0227] The scheme provided by the above idea one fully considers the possible influence of all source domain categories when analyzing the transmission cost of the second coordinate. However, not all source domain labels are related to the target domain label. From the perspective of the target domain label, the possible mapping relationship between the target domain label and the source domain label is introduced, which introduces some irrelevant noise. In order to eliminate this noise as much as possible, the present application proposes that the possible mapping relationship between the source domain label and the target domain label can be transported from the perspective of the source domain label.

[0228] Based on the above idea, to create the mapping relationship between the target domain label represented by the first row and the source domain label represented by the third column, the mapping relationship between the source domain label represented by the third column and the source domain label represented by the row other than the first row needs to be migrated to the mapping relationship between the target domain label represented by the first row and the source domain label represented by the third column.

[0229] In this case, the calculation method of the earth moving distance of the second coordinate is as follows:

[0230] Where j represents the target domain label, and t is the first label (the target domain label corresponding to the second coordinate).

[0231] C i,j represents the unit transmission cost from the source domain label i to the target domain label j, where i is included in the category p, j is included in the category q, and the second coordinate is (i, j). The calculation method of the transmission cost of the second coordinate is as follows:

[0232] Where f k',iFor the elements corresponding to any coordinates in the column where the second coordinate is located in the distribution matrix, k' is the target domain label corresponding to the element, C j is the coordinate with the target domain label j.

[0233] Still taking the distribution matrix shown in FIG. 2 as an example, it is assumed that the second coordinate is (1, 3).

[0234] Based on the above idea, the transmission cost of the second coordinate is the sum of the element values of the other columns in the third column of the distribution matrix except the first row. That is, the transmission cost of the second coordinate is the sum of the element values corresponding to all coordinates in the column where the second coordinate is located in the distribution matrix except the second coordinate.

[0235] In the embodiments of the present application, based on the distribution matrix, the sum of the elements corresponding to all coordinates in the column where the second coordinate is located in the distribution matrix except the second coordinate is taken as the transmission cost of the second coordinate. From the perspective of the source domain label, the calculation of the earth-moving distance ensures that each move is related to the source domain label, effectively reduces unnecessary noise, and improves the reliability of the transmission cost of the second coordinate. At the same time, since in real use scenarios, the number of source domain labels often increases exponentially compared to the target domain labels, calculating the earth-moving distance from the perspective of the source domain label can also greatly reduce the computational complexity and improve the analysis efficiency.

[0236] 308. Construct a transmission cost matrix of N*S dimensions according to the transmission cost of the mapping relationship;

[0237] 309. Construct a mapping matrix of N*S dimensions;

[0238] 310. Based on the Frobenius inner product of the mapping matrix and the transmission cost matrix, take the mapping relationship with the minimum transmission cost as the target mapping set;

[0239] 311. Based on the mapping relationship between the preset label and the reference label, determine that the label corresponding to the training sample is the reference label;

[0240] 312. Update the input transformation parameter based on the loss value of the training label and the reference label to obtain an updated input transformation parameter.

[0241] Steps 308 to 312 are similar to steps 108 to 112 shown in FIG. 1, and the specific operations are described in the foregoing description of steps 108 to 112 in FIG. 1, which will not be repeated here.

[0242] In the embodiments of the present application, when the cost matrix is calculated, the output features of the model softmax layer are used as the basis for calculating the transmission cost matrix, rather than the classification results of the voting system. The prior experience of a finer granularity is used as the basis for constructing the transmission cost matrix, which can effectively improve the precision of the model.

[0243] To demonstrate this conclusion, the scheme provided in the present application was used in a comparative experiment on different visual pre-training models. First, the scheme provided in the present application was used on the basis of the visual pre-training model ViT-b-1K, and the method provided in the present application was applied to different visual prompt methods, and these models were applied to different data sets, and the experimental results shown in FIG. 5 were obtained. Among them, the data sets include describable textures dataset (DTD), caltech-UCSD birds (CUB200), North America birds (NAbirds), Stanford dogs dataset (StanfordDogs), flowers-102, food-101, Canadian institute for advanced research 10 (CIFAR10), Canadian institute for advanced research 100 (CIFAR100), German traffic sign recognition benchmark (GTSRB), and street view house numbers (SVNH), and the visual prompt methods include virtual prompt (VP), frequency-based mapping (FLM-VP), and iterative label mapping (ILM-VP). The experimental results show that the image processing method provided in the present application has better analysis results in different data sets, indicating that the scheme provided in the present application has good generalization on downstream visual tasks.

[0244] In addition, the scheme provided in the present application also applies the method provided in the present application to different visual prompt modes on the basis of the visual pre-training model ResNet50-1K, and applies these models to different data sets. The specific data set types are the same as those in the foregoing FIG. 5. As can be seen from the experimental results shown in FIG. 6, the image processing method provided in the present application can not only obtain better analysis results in different data sets, but also obtain better effects in different models, which shows that the scheme provided in the present application has good adaptability to different models.

[0245] In addition, the image processing method provided in the present application performs the method shown in FIG. 7 in the inference stage. It should be noted that the image processing method shown in FIG. 7 can be implemented after the training method of the image processing model shown in the foregoing FIG. 1 or FIG. 3, and will not be described here.

[0246] 701, obtaining a target sample;

[0247] The target sample is an image associated with a target domain. For example, when the target domain label includes sparrow, swallow and the like, the target sample can be an image including birds, which is not limited here.

[0248] 702, processing the target sample based on the updated input transformation parameter to obtain a processed target sample;

[0249] The updated input transformation parameter can be the parameter of the input change processor.

[0250] Specifically, the target sample is input into the input change processor with the updated input transformation parameter, and the processed target sample can be obtained.

[0251] 703, analyzing the processed target sample to obtain a pre-training label;

[0252] The pre-training label is included in the N source domain labels.

[0253] Specifically, the target sample is processed by the visual pre-training model to obtain the pre-training label.

[0254] 704, determining that the label corresponding to the target sample is the target label according to the mapping relationship between the pre-training label and the target label.

[0255] The target label is a target domain label, and the mapping relationship between the pre-training label and the target label is included in a target mapping set. The target mapping set is designed to be a mapping relationship set with the minimum transmission cost. The transmission cost of the mapping relationship set is the transmission cost of each mapping relationship in the mapping relationship set. The mapping relationship set is composed of the mapping relationship between each label in the target domain label and the source domain label.

[0256] Specifically, the target mapping set can be calculated based on the training method of the image processing model shown in FIG. 1 or FIG. 3, which will not be repeated here.

[0257] In the embodiments of the present application, the mapping relationship set with the minimum transmission cost is taken as the target mapping set, and the transmission cost of the mapping relationship set is the sum of the transmission costs corresponding to each mapping relationship in the mapping relationship set, the mapping relationship set including the mapping relationship between each label in the target domain label and the source domain label, wherein each label in the target domain label corresponds to only one source domain label, and each source domain label also corresponds to only one target domain label. In the implementation process of the visual hint learning method, the target mapping set with the minimum transmission cost is taken as the output label mapping, the selection of the target mapping set is performed from the perspective of the total transmission cost of all mappings, the risk of the mapping relationship falling into local optimum is effectively reduced, and the accuracy of model reasoning is improved.

[0258] For ease of understanding, the data processing flow of the image processing model provided by the present application will be introduced below in conjunction with FIG. 8:

[0259] After obtaining the training sample and the training label, the distribution of the training sample in the target domain is determined based on the training label, and then the training sample is input into the visual pre-training model to obtain the processed training sample. The pre-set label can be obtained by inputting the processed training sample into the visual pre-training model, and the pre-set label is the source domain label, so the distribution of the pre-set label in the source domain can be analyzed. Finally, the reference label is obtained based on the mapping relationship between the reference label and the pre-set label, wherein the mapping relationship between the reference label and the pre-set label is included in the target mapping set, the target mapping set is obtained based on the Frobenius inner product analysis of the mapping matrix and the transmission cost matrix, and the mapping relationship set with the minimum transmission cost.

[0260] The image processing method and the training method of the image processing model provided by the present application are introduced above, and the image processing device and the training device of the image processing model provided by the present application will be introduced below in conjunction with the drawings.

[0261] Please refer to FIG. 9, which is a structural schematic diagram of an image processing device provided by the present application.

[0262] An image processing device 90 comprises:

[0263] An acquisition unit 910 is configured to acquire a target sample.

[0264] A processing unit 920 is configured to process the target sample based on the updated input transformation parameter to obtain a processed target sample.

[0265] An analysis unit 930 is configured to analyze the processed target sample to obtain a pre-training label, and the pre-training label is a source domain label.

[0266] The processing unit 920 is further configured to determine, according to a mapping relationship between the pre-training label and the target label, that the label corresponding to the target sample is the target label, the mapping relationship between the pre-training label and the target label is included in a target mapping set, the target mapping set is a mapping relationship set with a minimum transmission cost, the transmission cost of the mapping relationship set is a sum of transmission costs corresponding to each mapping relationship in the mapping relationship set, the mapping relationship set is composed of mapping relationships between each label in the target domain label and the source domain label, and the target label is the target domain label.

[0267] Optionally, the obtaining unit 910 is further configured to obtain training samples and training labels, the training labels are included in S target domain labels, S is a positive integer.

[0268] The processing unit 920 is further configured to process the training samples based on the input transformation parameter to obtain processed training samples.

[0269] The analysis unit 930 is further configured to analyze the processed training samples to obtain preset labels, the preset labels are included in N source domain labels, N is a positive integer, and N is greater than S.

[0270] The processing unit 920 is further configured to determine, based on a mapping relationship between the preset label and a reference label, that the label corresponding to the training sample is the reference label, and the mapping relationship between the reference label and the preset label is included in the target mapping set.

[0271] The processing unit 920 is further configured to update the input transformation parameter based on a loss value of the training label and the reference label to obtain an updated input transformation parameter.

[0272] Optionally, the obtaining unit 910 is further configured to obtain N source domain labels.

[0273] The obtaining unit 910 is further configured to obtain transmission costs of the mapping relationships, the transmission cost of the mapping relationship includes a transmission cost of a mapping relationship between a first label and a second label, the first label is any one of the S target domain labels, the second label is any one of the N source domain labels, the second label corresponds to at most one of the S target domain labels, and the first label corresponds to one of the N source domain labels.

[0274] The analysis unit 930 is further configured to analyze, based on the S target domain labels, the N source domain labels and the transmission costs of the mapping relationships, a mapping relationship set with a minimum transmission cost to obtain the target mapping set.

[0275] Optionally, the analysis unit 930 is specifically configured to:

[0276] constructing an N*S-dimensional mapping matrix, each row of N rows including at most one element 1 and at least S-1 elements 0, each column of S columns including one element 1 and N-1 elements 0, the mapping matrix being used to represent a mapping relationship between the first label and the second label;

[0277] constructing an N*S-dimensional transmission cost matrix according to the transmission cost of the mapping relationship, an element at a first coordinate in the transmission cost matrix being a transmission cost of a mapping relationship corresponding to the first coordinate in the mapping matrix;

[0278] based on a Frobenius inner product of the mapping matrix and the transmission cost matrix, taking a mapping relationship set with the minimum transmission cost as a target mapping set.

[0279] Optionally, the obtaining unit 910 is specifically configured to:

[0280] analyzing the preset label to obtain a distribution of an analysis result of the reference sample in the N source domain labels, the reference sample being at least two samples corresponding to a third label, the third label being included in the S target domain labels;

[0281] constructing an N*S-dimensional distribution matrix based on the distribution of the analysis result of the reference sample in the S source domain labels;

[0282] obtaining a transmission cost of a second coordinate according to the distribution matrix, the second coordinate representing a corresponding relationship between the first label and the second label, the second coordinate being a coordinate in the distribution matrix.

[0283] Optionally, the analyzing unit 930 is specifically configured to:

[0284] analyzing the processed training sample to obtain a first distribution probability, the first distribution probability being a probability of a label corresponding to the processed training sample being each label in the S source domain labels;

[0285] based on the first distribution probability, taking a label with the highest probability in the S source domain labels as the preset label;

[0286] the analyzing unit 930 is specifically configured to:

[0287] analyzing the first distribution probability and the training label to obtain a distribution of an analysis result of the reference sample in the N source domain labels, the reference sample being at least two samples corresponding to a third label, the third label being included in the S target domain labels;

[0288] constructing an N*S-dimensional distribution matrix based on the distribution of the analysis result of the reference sample in the source domain labels;

[0289] obtaining a transmission cost of a second coordinate according to the distribution matrix, the second coordinate representing a corresponding relationship between the first label and the second label.

[0290] Optionally, the transmission cost of the second coordinate is a sum of element values corresponding to all coordinates except the second coordinate in a column in which the second coordinate is located in the distribution matrix, or the transmission cost of the second coordinate is a sum of element values corresponding to all coordinates except the second coordinate in a row in which the second coordinate is located in the distribution matrix.

[0291] Referring to FIG. 10, FIG. 10 is a structural schematic diagram of a training device of a data processing model provided by the present application;

[0292] A training device 100 of an image processing model comprises:

[0293] An acquisition unit 1010 is configured to acquire a training sample and a training label, the training label being included in S target domain labels, S being a positive integer;

[0294] A processing unit 1020 is configured to process the training sample based on an input transformation parameter to obtain a processed training sample;

[0295] An analysis unit 1030 is configured to analyze the processed training sample to obtain a preset label, the preset label being included in N source domain labels, N being a positive integer, and N being greater than S;

[0296] The processing unit 1020 is further configured to determine, based on a mapping relationship between the preset label and a reference label, that a label corresponding to the training sample is the reference label, the mapping relationship between the preset label and the reference label being included in a target mapping set, the target mapping set being a mapping relationship set with a minimum transmission cost, and the transmission cost of the target mapping set being a sum of transmission costs corresponding to each mapping relationship in the target mapping set, the mapping relationship set being composed of mapping relationships between each label in the target domain label and a source domain label;

[0297] The processing unit 1020 is further configured to update the input transformation parameter based on a loss value of the training label and the reference label to obtain an updated input transformation parameter.

[0298] Optionally, the acquisition unit 1010 is further configured to acquire the N source domain labels.

[0299] The acquisition unit 1010 is further configured to acquire a transmission cost of a mapping relationship, the transmission cost of the mapping relationship including a transmission cost corresponding to a mapping relationship between a first label and a second label, the first label being any one of the S target domain labels, the second label being any one of the N source domain labels, the second label corresponding to at most one of the S target domain labels, and the first label corresponding to one of the N source domain labels.

[0300] The analysis unit 1030 is further configured to analyze, based on the S target domain labels, the N source domain labels, and the transmission cost of the mapping relationship, a mapping relationship set with a minimum transmission cost to obtain the target mapping set.

[0301] Optionally, the analysis unit 1030 is specifically configured to:

[0302] construct an N*S-dimensional mapping matrix, each row of N rows includes at most one element 1 and at least S-1 elements 0, each column of S columns includes one element 1 and N-1 elements 0, and the mapping matrix is used to represent the mapping relationship between the first label and the second label;

[0303] construct an N*S-dimensional transmission cost matrix according to the transmission cost of the mapping relationship, and the element of the first coordinate in the transmission cost matrix is the transmission cost of the mapping relationship corresponding to the first coordinate in the mapping matrix;

[0304] based on the Frobenius inner product of the mapping matrix and the transmission cost matrix, the mapping relationship set with the minimum transmission cost is taken as the target mapping set.

[0305] Optionally, the obtaining unit 1010 is specifically configured to:

[0306] analyze the preset label, obtain the distribution of the analysis result of the reference sample in the N source domain labels, the reference sample is at least two samples corresponding to the third label, and the third label is included in the S target domain labels;

[0307] based on the distribution of the analysis result of the reference sample in the N source domain labels, construct an N*S-dimensional distribution matrix;

[0308] according to the distribution matrix, calculate the transmission cost of the second coordinate, and the second coordinate represents the corresponding relationship between the first label and the second label, and the second coordinate is a coordinate in the distribution matrix.

[0309] Optionally, the analysis unit 1030 is specifically configured to:

[0310] analyze the processed training sample, obtain the first distribution probability, and the first distribution probability is the probability that the label corresponding to the processed training sample is each label in the S source domain labels;

[0311] the obtaining unit 1010 is specifically configured to:

[0312] analyze the first distribution probability and the training label, obtain the distribution of the analysis result of the reference sample in the N source domain labels, the reference sample is at least two samples corresponding to the third label, and the third label is included in the S target domain labels;

[0313] based on the distribution of the analysis result of the reference sample in the source domain label, construct an N*S-dimensional distribution matrix;

[0314] according to the distribution matrix, calculate the transmission cost of the second coordinate, and the second coordinate represents the corresponding relationship between the first label and the second label.

[0315] Optionally, the transmission cost of the second coordinate is the sum of the element values corresponding to all coordinates except the second coordinate in the column in which the second coordinate is located in the distribution matrix, or the transmission cost of the second coordinate is the sum of the element values corresponding to all coordinates except the second coordinate in the row in which the second coordinate is located in the distribution matrix.

[0316] Next, the computer device provided by the embodiments of the present application is described. Referring to FIG. 11, FIG. 11 is a structural schematic diagram of a computer device provided by an embodiment of the present application. The computer device 1100 includes a processor 1101, a memory 1102, a communication interface 1103 and a bus 1104. The processor 1101, the memory 1102 and the communication interface 1103 communicate through the bus 1104, and can also communicate through wireless transmission and other means. The memory 1102 stores program codes, and the processor 1101 can invoke the program codes stored in the memory 1102 to execute the operations performed in the embodiments shown in FIG. 1, FIG. 3 or FIG. 7, which will not be described herein.

[0317] It should be understood that, in the embodiments of the present application, the processor 1101 can be a CPU, and the processor 1101 can also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0318] The memory 1102 can include read-only memory and random access memory, and provide instructions and data for the processor 1101. The memory 1102 can also include non-volatile random access memory. For example, the memory 1102 can also store device type information.

[0319] The memory 1102 can be volatile memory or nonvolatile memory, or can include both volatile and nonvolatile memory. By way of illustration, and not limitation, nonvolatile memory can be read-only memory (ROM), programmable ROM (PROM), erasable PROM (EPROM), electrically EPROM (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), which acts as external cache. By way of illustration and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double-data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), Synchlink DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0320] The bus 1104 can include a data bus, a power bus, a control bus, and a state signal bus, among others. For the sake of clarity, the various buses are illustrated in the figure as the bus 1104. The bus 1104 can be a Peripheral Component Interconnect Express (PCIe) bus, or an extended industry standard architecture (EISA) bus, a unified bus (Ubus or UB), a compute express link (CXL), a cache coherent interconnect for accelerators (CCIX), or the like. The bus 1104 can be divided into an address bus, a data bus, a control bus, and the like.

[0321] The computer device 1100 can also include one or more communication interfaces, one or more operating systems, such as Windows Server TM , Mac OS XTM Unix TM Linux TM FreeBSD TM and the like.

[0322] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the system, device and unit described above can refer to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0323] In several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interface, device or unit, and can be electrical, mechanical or other forms.

[0324] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, that is, they can be located in one place, or can be distributed on a plurality of network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0325] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0326] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

Claims

1. A method of image processing, characterized by, The method comprises: obtaining a target sample; processing the target sample based on an updated input transformation parameter to obtain a processed target sample; analyzing the processed target sample to obtain a pre-training label, the pre-training label being a source domain label; determining that a label corresponding to the target sample is a target label according to a mapping relationship between the pre-training label and the target label, the mapping relationship between the pre-training label and the target label being included in a target mapping set, the target mapping set being a mapping relationship set with a minimum transmission cost, the transmission cost of the mapping relationship set being a sum of transmission costs corresponding to each mapping relationship in the mapping relationship set, the mapping relationship set being composed of mapping relationships between each label in a target domain label and a source domain label, and the target label being a target domain label.

2. The method of claim 1, wherein, Before the determining that the label corresponding to the target sample is the target label according to the mapping relationship between the pre-training label and the target label, the method further comprises: obtaining a training sample and a training label, the training label being included in S target domain labels, S being a positive integer; processing the training sample based on an input transformation parameter to obtain a processed training sample; analyzing the processed training sample to obtain a preset label, the preset label being included in N source domain labels, N being a positive integer, and N being greater than S; determining that a label corresponding to the training sample is a reference label based on a mapping relationship between the preset label and the reference label, the mapping relationship between the reference label and the preset label being included in the target mapping set; updating the input transformation parameter based on a loss value of the training label and the reference label to obtain the updated input transformation parameter.

3. The method of claim 2, wherein, Before the determining that the label corresponding to the training sample is the reference label based on the mapping relationship between the preset label and the reference label, the method further comprises: obtaining the N source domain labels; obtaining a transmission cost of a mapping relationship, the transmission cost of the mapping relationship including a transmission cost of a mapping relationship between a first label and a second label, the first label being any one of the S target domain labels, the second label being any one of the N source domain labels, the second label corresponding to at most one of the S target domain labels, and the first label corresponding to one of the N source domain labels; analyzing a mapping relationship set with a minimum transmission cost based on the S target domain labels, the N source domain labels, and the transmission cost of the mapping relationship to obtain the target mapping set.

4. The method of claim 3, wherein, The analyzing the mapping relationship set with the minimum transmission cost based on the target domain labels, the N source domain labels, and the transmission cost of the mapping relationship to obtain the target mapping set comprises: constructing an N*S-dimensional mapping matrix, each row in the N rows including at most one element 1 and at least S-1 elements 0, each column in the S columns including one element 1 and N-1 elements 0, and the mapping matrix being used to represent the mapping relationship between the first label and the second label; According to the transmission cost of the mapping relationship, a transmission cost matrix of N*S dimensions is constructed, an element of a first coordinate in the transmission cost matrix being a transmission cost of a mapping relationship corresponding to the first coordinate in the mapping matrix; A mapping relationship set with minimum transmission cost is taken as the target mapping set based on a Frobenius inner product of the mapping matrix and the transmission cost matrix.

5. The method according to claim 3 or 4, characterized in that, The transmission cost of the mapping relationship includes: The distribution of the analysis result of the reference sample in the N source domain labels is obtained by analyzing the preset label, the reference sample being at least two samples corresponding to a third label, the third label being included in the S target domain labels; An N*S dimension distribution matrix is constructed based on the distribution of the analysis result of the reference sample in the S source domain labels; According to the distribution matrix, the transmission cost of a second coordinate is calculated, the second coordinate representing a corresponding relationship between the first label and the second label, the second coordinate being a coordinate in the distribution matrix.

6. The method according to claim 3 or 4, characterized in that, The analysis of the processed training sample to obtain a preset label includes: A first distribution probability is obtained by analyzing the processed training sample, the first distribution probability being a probability that a label corresponding to the processed training sample is each label in the S source domain labels; Based on the first distribution probability, a label with the highest probability in the S source domain labels is taken as a preset label; The construction of an N*S dimension transmission cost matrix includes: The distribution of the analysis result of the reference sample in the N source domain labels is obtained by analyzing the first distribution probability and the training label, the reference sample being at least two samples corresponding to a third label, the third label being included in the S target domain labels; An N*S dimension distribution matrix is constructed based on the distribution of the analysis result of the reference sample in the source domain labels; According to the distribution matrix, the transmission cost of a second coordinate is calculated, the second coordinate representing a corresponding relationship between the first label and the second label.

7. The method according to claim 5 or 6, characterized in that, The transmission cost of the second coordinate is a sum of element values of all coordinates except the second coordinate in a column where the second coordinate is located in the distribution matrix, or the transmission cost of the second coordinate is a sum of element values of all coordinates except the second coordinate in a row where the second coordinate is located in the distribution matrix.

8. A method for training an image processing model, the method comprising: It includes: Obtaining training samples and training labels, the training labels being included in S target domain labels, the S being a positive integer; Processing the training samples based on input transformation parameters to obtain processed training samples; Analyzing the processed training samples to obtain a preset label, the preset label being included in N source domain labels, the N being a positive integer, the N being greater than the S; determining, based on a mapping relationship between the preset label and a reference label, that the label corresponding to the training sample is the reference label, the mapping relationship between the preset label and the reference label being included in a target mapping set, the target mapping set being a mapping relationship set with minimum transmission cost, the transmission cost of the target mapping set being a sum of transmission costs corresponding to each mapping relationship in the target mapping set, the mapping relationship set being composed of mapping relationships between each label in the target domain label and a source domain label; updating the input transformation parameter based on a loss value of the training label and the reference label to obtain an updated input transformation parameter.

9. The method of claim 8, wherein, Before the determining, based on a mapping relationship between the preset label and a reference label, that the label corresponding to the training sample is the reference label, the method further includes: obtaining the N source domain labels; obtaining a transmission cost of a mapping relationship, the transmission cost of the mapping relationship including a transmission cost corresponding to a mapping relationship between a first label and a second label, the first label being any one of the S target domain labels, the second label being any one of the N source domain labels, the second label corresponding to at most one of the S target domain labels, the first label corresponding to one of the N source domain labels; analyzing, based on the S target domain labels, the N source domain labels and the transmission cost of the mapping relationship, a mapping relationship set with minimum transmission cost to obtain the target mapping set.

10. The method of claim 9, wherein, The analyzing, based on the S target domain labels, the N source domain labels and the transmission cost of the mapping relationship, a mapping relationship set with minimum transmission cost to obtain the target mapping set includes: constructing an N*S-dimensional mapping matrix, each row in the N rows including at most one element 1 and at least S-1 elements 0, each column in the S columns including one element 1 and N-1 elements 0, the mapping matrix being used to represent the mapping relationship between the first label and the second label; constructing an N*S-dimensional transmission cost matrix according to the transmission cost of the mapping relationship, an element at a first coordinate in the transmission cost matrix being a transmission cost of a mapping relationship corresponding to the first coordinate in the mapping matrix; based on a Frobenius inner product of the mapping matrix and the transmission cost matrix, taking a mapping relationship set with minimum transmission cost as the target mapping set.

11. The method according to claim 9 or 10, characterized in that, The obtaining a transmission cost of a mapping relationship includes: analyzing the preset label to obtain a distribution of an analysis result of a reference sample in the N source domain labels, the reference sample being at least two samples corresponding to a third label, the third label being included in the S target domain labels; constructing an N*S-dimensional distribution matrix based on the distribution of the analysis result of the reference sample in the N source domain labels; calculating, according to the distribution matrix, a transmission cost of a second coordinate, the second coordinate representing a corresponding relationship between the first label and the second label, the second coordinate being a coordinate in the distribution matrix.

12. The method according to claim 9 or 10, characterized in that, The analyzing the processed training sample to obtain a preset label includes: analyze the processed training samples to obtain a first distribution probability, the first distribution probability being a probability that a label corresponding to the processed training samples is each label in the S source domain labels; the transmission cost of the mapping relationship comprises: analyze the first distribution probability and the training label to obtain a distribution of an analysis result of a reference sample in the N source domain labels, the reference sample being at least two samples corresponding to a third label, and the third label being included in the S target domain labels; based on the distribution of the analysis result of the reference sample in the source domain labels, a distribution matrix of N*S dimensions is constructed; according to the distribution matrix, a transmission cost of a second coordinate is calculated, the second coordinate representing a corresponding relationship between the first label and the second label.

13. The method according to claim 11 or 12, characterized in that, The transmission cost of the second coordinate is the sum of the element values of all coordinates in the column of the second coordinate in the distribution matrix except the second coordinate, or the transmission cost of the second coordinate is the sum of the element values of all coordinates in the row of the second coordinate in the distribution matrix.

14. An apparatus for image processing, characterized by comprise: an acquisition unit configured to acquire a target sample; a processing unit configured to process the target sample based on the updated input transformation parameter to obtain a processed target sample; an analysis unit configured to analyze the processed target sample to obtain a pre-training label, the pre-training label being a source domain label; the processing unit is further configured to determine that a label corresponding to the target sample is a target label according to a mapping relationship between the pre-training label and the target label, the mapping relationship between the pre-training label and the target label being included in a target mapping set, the target mapping set being a mapping relationship set with the minimum transmission cost, the transmission cost of the mapping relationship set being the sum of the transmission costs corresponding to each mapping relationship in the mapping relationship set, the mapping relationship set being composed of mapping relationships between each label in the target domain label and the source domain label, and the target label being a target domain label.

15. A training device for an image processing model, characterized in that, comprise: an acquisition unit configured to acquire a training sample and a training label, the training label being included in S target domain labels, and the S being a positive integer; a processing unit configured to process the training sample based on an input transformation parameter to obtain a processed training sample; an analysis unit configured to analyze the processed training sample to obtain a preset label, the preset label being included in N source domain labels, the N being a positive integer, and the N being greater than the S; the processing unit is further configured to determine that a label corresponding to the training sample is a reference label according to a mapping relationship between the preset label and the reference label, the mapping relationship between the preset label and the reference label being included in a target mapping set, the target mapping set being a mapping relationship set with the minimum transmission cost, the transmission cost of the target mapping set being the sum of the transmission costs corresponding to each mapping relationship in the target mapping set, the mapping relationship set being composed of mapping relationships between each label in the target domain label and the source domain label. The processing unit is further configured to update the input transformation parameter based on a loss value of the training label and the reference label, and obtain an updated input transformation parameter.

16. A computer device, comprising: The communication device comprises a processor coupled with a memory; The memory stores instructions which, when executed on the processor, cause the communication device to implement the method of any one of claims 1 to 13.

17. A computer-readable storage medium, characterized in that, The computer readable storage medium stores instructions which, when executed on the processor, cause the method of any one of claims 1 to 13 to be implemented.

18. A computer program product, characterised in that, The computer program product, when executed on the computer, causes the method of any one of claims 1 to 13 to be implemented.

Images