Image processing model training method and device, storage medium and electronic device

Abstract

The present disclosure belongs to the technical field of image processing, and relates to a training method and device of an image processing model, a storage medium and an electronic device. The method comprises: obtaining a picture training sample and a true value picture corresponding to the picture training sample; inputting the picture training sample into a three-dimensional color lookup model to obtain a model prediction picture, and performing loss calculation on the model prediction picture and the true value picture to obtain a loss calculation result; and adjusting the three-dimensional color lookup model according to the loss calculation result to obtain a target image processing model, wherein the target image processing model is used for image processing on a to-be-processed image to obtain an image processing result corresponding to the to-be-processed image.

Description

Technical Field

[0001] This disclosure relates to the field of image processing technology, and in particular to a training method for an image processing model, an image processing model training device, a computer-readable storage medium, and an electronic device. Background Technology

[0002] With the development of image technology, more image processing needs have emerged, such as dehazing images, transforming blurry images into clear ones, and enhancing image exposure.

[0003] In related technologies, image processing typically employs a one-dimensional color lookup model, which usually corresponds to a one-dimensional color lookup table. However, a one-dimensional color lookup table can only control the output of a single color channel, and each color channel is independent of the others. Furthermore, the data volume of a one-dimensional color lookup table is relatively small. Consequently, the one-dimensional color lookup model cannot provide highly accurate image processing results, thus reducing the user experience.

[0004] Therefore, there is an urgent need in this field to develop a new training method and apparatus for image processing models.

[0005] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0006] The purpose of this disclosure is to provide a training method for an image processing model, a training device for an image processing model, a computer-readable storage medium, and an electronic device, thereby overcoming, to at least a certain extent, the problem of not being able to provide highly accurate image processing results due to related technologies.

[0007] Other features and advantages of this disclosure will become apparent from the following detailed description, or may be learned in part from practice of this disclosure.

[0008] According to a first aspect of the present invention, a method for training an image processing model is provided. The method includes: acquiring image training samples and acquiring ground truth images corresponding to the image training samples; inputting the image training samples into a three-dimensional color lookup model to obtain a model prediction image, and performing loss calculation on the model prediction image and the ground truth image to obtain a loss calculation result; adjusting the three-dimensional color lookup model according to the loss calculation result to obtain a target image processing model; wherein the target image processing model is used to perform image processing on an image to be processed to obtain an image processing result corresponding to the image to be processed.

[0009] In an exemplary embodiment of the present invention, the three-dimensional search model includes a first optimized model, which includes a combination of a basic search model and a first derived model. The first derived model includes a weight model and a plurality of the basic search models. The step of inputting the image training sample into the three-dimensional color search model to obtain a model-predicted image includes: inputting the image training sample into the weight model to extract image features corresponding to the image training sample; determining weight values ​​corresponding to the plurality of basic search models in the first derived model based on the image features; updating the plurality of basic color mapping relationships corresponding to the plurality of basic search models in the first derived model based on the weight values ​​corresponding to the plurality of basic search models in the first derived model, and calculating the update result to obtain a first derived color mapping relationship corresponding to one of the first derived models; determining a first optimized color mapping relationship corresponding to the first optimized model based on the combination relationship corresponding to the combination of the basic search model and the first derived model in the first optimized model, the first derived color mapping relationship, and the basic color search mapping relationship corresponding to the basic search model in the first optimized model; and determining a model-predicted image corresponding to the image training sample based on the first optimized color mapping relationship.

[0010] In an exemplary embodiment of the present invention, there is a linear combination relationship between the basic search model and the first derived model in the first optimization model; the step of adjusting the three-dimensional color search model according to the loss calculation result to obtain the target image processing model includes: adjusting the weight values ​​corresponding to the multiple basic search models in the first derived model and the basic color mapping relationship corresponding to the basic search models in the first derived model according to the loss calculation result to obtain the target image processing model.

[0011] In an exemplary embodiment of the present invention, there is a product combination relationship between the basic search model and the first derived model in the first optimization model; the step of adjusting the three-dimensional color search model according to the loss calculation result to obtain the target image processing model includes: adjusting the weight values ​​corresponding to the multiple basic search models in the first derived model and the multiple basic color mapping relationships corresponding to the multiple basic search models in the first derived model to obtain the training result corresponding to the first derived model; when the training result meets the training termination condition, the first optimization model is trained according to the loss calculation result to obtain the target image processing model.

[0012] In an exemplary embodiment of the present invention, the weight model includes an image size fixed layer, multiple sampling layers, and an output layer connected in sequence, wherein the image size fixed layer is used to fix the size of the image training samples, the sampling layers are used to extract the image features corresponding to the image training samples, and the output layer is used to determine the weight values ​​corresponding to the multiple basic search models in the first derived model based on the image features.

[0013] In an exemplary embodiment of the present invention, there is a product combination relationship between the basic search model and the first derived model in the first optimization model; the step of adjusting the three-dimensional color search model according to the loss calculation result to obtain the target image processing model includes: the three-dimensional color search model includes a combination model, the combination model includes two first optimization models; wherein, the combination relationship of one of the first optimization models in the combination model is a linear combination relationship, and the combination relationship of the other first optimization model in the combination model is a product combination relationship.

[0014] In an exemplary embodiment of the present invention, the three-dimensional color lookup model includes a second optimization model, which includes a combination of multiple second derived models and a basic lookup model; the step of inputting the image training sample into the three-dimensional color lookup model to obtain the model prediction image includes: inputting the image training sample into multiple second derived models to extract image features corresponding to the image training sample; determining a second derived color mapping relationship between the pixel color value and the target pixel color value in the image training sample based on the image features; determining a second optimized color mapping relationship corresponding to the second optimization model based on the combination relationship corresponding to the combination of multiple second derived models and the basic lookup model, multiple second derived color mapping relationships corresponding to the multiple second derived models, and a basic color mapping relationship corresponding to the basic lookup model; and determining the model prediction image corresponding to the image training sample based on the second optimized color mapping relationship.

[0015] In an exemplary embodiment of the present invention, the three-dimensional color lookup model includes a combined model, which includes two second optimized models; wherein, in the combined model, a plurality of second derived models of one of the second optimized models and a basic lookup model are in a linear combination relationship, and in the combined model, a plurality of second derived models of the other second optimized model and a basic lookup model are in a product combination relationship.

[0016] In an exemplary embodiment of the present invention, the combined model includes a combination of a first optimization model and a second optimization model; the first optimization model includes a basic search model and a first derived model, the first derived model includes a weight model and a plurality of the basic search models; the method further includes: if the first optimization model is a linear combination of a basic search model and the first derived model, then the second optimization model is a product combination of a basic search model and a plurality of the second derived models; if the first optimization model is a product combination of a basic search model and the first derived model, then the second optimization model is a linear combination of a basic search model and a plurality of the second derived models.

[0017] In an exemplary embodiment of the present invention, there is a linear combination relationship between the basic search model and the plurality of second derived models in the second optimization model; the step of adjusting the three-dimensional color search model according to the loss calculation result to obtain the target image processing model includes: adjusting the second derived color mapping relationship according to the loss calculation result to obtain the target image processing model.

[0018] In an exemplary embodiment of the present invention, there is a product combination relationship between the basic search model and the second derived model in the second optimization model; the step of adjusting the three-dimensional color search model according to the loss calculation result to obtain the target image processing model includes: adjusting the second derived color mapping relationship according to the loss calculation result to obtain a training result corresponding to the second derived model; if the training result satisfies the training termination condition, training the second optimization model according to the loss calculation result to obtain the target image processing model.

[0019] In an exemplary embodiment of the present invention, the second derived model includes an image size fixing layer, a matrix transformation layer, multiple sampling layers, and an output layer connected in sequence. The image size fixing layer is used to fix the size of the image training samples, the matrix transformation layer is used to transform the matrix output by the image size fixing layer, the sampling layer is used to extract the image features corresponding to the image training samples, and the output layer is used to output the second derived color mapping relationship corresponding to the image features.

[0020] In an exemplary embodiment of the present invention, the three-dimensional color lookup model includes one or more third optimization models, wherein the third optimization model is any one of the basic lookup model, the first optimization model, the second optimization model, and the combined model; the first optimization model includes a combination of one of the basic lookup models and a first derived model, the first derived model includes a weight model and multiple basic lookup models, and the second optimization model includes a combination of multiple second derived models and one basic lookup model; the step of inputting the image training samples into the three-dimensional color lookup model to obtain the model prediction image includes: obtaining multiple downsampling ratios, and sampling the image training samples according to the multiple downsampling ratios to obtain multiple downsampling results; wherein the downsampling ratios include integer ratios; and determining the upsampling ratio according to image processing requirements. The upsampling rate is used to sample the training samples of the image to obtain an upsampling result; wherein the upsampling rate includes a decimal rate; the downsampling result is input into one or more of the third optimization models to obtain a first model output result corresponding to the downsampling result; the upsampling result is input into one or more of the third optimization models to obtain a second model output result corresponding to the upsampling result; the magnitudes of the upsampling rates are compared to obtain a first rate comparison result, and the magnitudes of the downsampling rates are compared to obtain a second rate comparison result; based on the first rate comparison result and the second rate comparison result, the input-output relationship between the first model output result and the second model output result is determined, so as to obtain the model-predicted image based on the input-output relationship.

[0021] In an exemplary embodiment of the present invention, the method further includes: inputting the image training samples into a model to be learned to obtain ground truth images corresponding to the image training samples; wherein the model to be learned includes any one of the basic search model, the first optimization model, the second optimization model, the third optimization model, and an open-source model; inputting the image training samples into a target optimization model to obtain the model predicted images; wherein the target optimization model includes any one of the basic search model, the first optimization model, the second optimization model, and the combined model; performing loss calculation on the model predicted images and the ground truth images, and adjusting the target optimization model according to the loss calculation results to obtain the target optimization model having the same function as the model to be learned.

[0022] In an exemplary embodiment of the present invention, the three-dimensional color lookup model includes a basic lookup model; the step of inputting the image training sample into the three-dimensional color lookup model to obtain a model prediction image includes: inputting the image training sample into the basic lookup model; wherein, the basic lookup model is used to determine the pixel color values ​​in the image training sample, and determine the target pixel color value that has a basic color mapping relationship with the pixel color value; determine the target pixel corresponding to the target pixel color value, and use the image composed of the target pixel as the model prediction image.

[0023] According to a second aspect of the present invention, an image processing model training apparatus is provided. The apparatus includes: an acquisition module configured to acquire image training samples and acquire ground truth images corresponding to the image training samples; a loss calculation module configured to input the image training samples into a three-dimensional color lookup model to obtain a model prediction image, and perform loss calculation on the model prediction image and the ground truth image to obtain a loss calculation result; and an adjustment module configured to adjust the three-dimensional color lookup model according to the loss calculation result to obtain a target image processing model; wherein the target image processing model is used to perform image processing on an image to be processed to obtain an image processing result corresponding to the image to be processed.

[0024] According to a third aspect of the present invention, an electronic device is provided, comprising: a processor and a memory; wherein the memory stores computer-readable instructions, which, when executed by the processor, implement a training method for an image processing model of any of the above exemplary embodiments.

[0025] According to a fourth aspect of the present invention, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the training method of the image processing model in any of the above exemplary embodiments.

[0026] According to a fifth aspect of the present invention, an image processing method is provided, comprising: acquiring an image to be processed and an image processing requirement; and inputting the image to be processed and the image processing requirement into the target image processing model of the above method to obtain an image processing result.

[0027] As can be seen from the above technical solutions, the image processing model training method, image processing model training device, computer storage medium, and electronic device in the exemplary embodiments of the present invention have at least the following advantages and positive effects:

[0028] In the methods and apparatus provided by the exemplary embodiments of this disclosure, on the one hand, the target image processing model is the result of training a three-dimensional color lookup model, which avoids the fact that the image processing results in the prior art are based on a one-dimensional color lookup table, thus ensuring the accuracy of the image processing results; on the other hand, since the three-dimensional color lookup model is three-dimensional, the three-dimensional color lookup model has a larger data capacity than the one-dimensional color lookup table in the prior art, which meets different image processing needs and expands the application scenarios of image processing.

[0029] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0030] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure. It is obvious that the drawings described below are merely some embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0031] Figure 1 The schematic diagram illustrates the flowchart of the training method for the image processing model in the embodiments of this disclosure;

[0032] Figure 2 This schematic diagram illustrates the structure of the three-dimensional color lookup model in an embodiment of the present disclosure.

[0033] Figure 3 This illustration shows a schematic diagram of the process for obtaining the model-predicted image in an embodiment of this disclosure;

[0034] Figure 4 This illustration shows a schematic diagram of the process for obtaining the model-predicted image in an embodiment of this disclosure;

[0035] Figure 5 This schematic diagram illustrates the structure of a first optimization model according to an embodiment of the present disclosure;

[0036] Figure 6 This schematic diagram illustrates the structure of another first optimization model in an embodiment of the present disclosure;

[0037] Figure 7 This schematic diagram illustrates the structure of the weight model in an embodiment of the present disclosure;

[0038] Figure 8 This illustration shows a schematic diagram of the process for obtaining the model-predicted image in an embodiment of this disclosure;

[0039] Figure 9This schematic diagram illustrates a structural schematic of the second optimization model in an embodiment of the present disclosure;

[0040] Figure 10 This schematic diagram illustrates another structural diagram of the second optimization model in an embodiment of the present disclosure;

[0041] Figure 11 This schematic diagram illustrates the structure of the second derived model in an embodiment of the present disclosure;

[0042] Figure 12 This schematic diagram illustrates a structural schematic of a combined search model in an embodiment of the present disclosure;

[0043] Figure 13 This schematically illustrates another structural diagram of the combined search model in an embodiment of the present disclosure;

[0044] Figure 14 The schematic diagram illustrates the process of obtaining the combined model in an embodiment of this disclosure;

[0045] Figure 15 This illustration shows a schematic diagram of the process for obtaining the model-predicted image in an embodiment of this disclosure;

[0046] Figure 16 This schematic diagram illustrates the model structure when the three-dimensional color lookup model is the third optimized model in an embodiment of this disclosure.

[0047] Figure 17 This schematic diagram illustrates the model structure when the three-dimensional color lookup model is the third optimized model in an embodiment of this disclosure.

[0048] Figure 18 This schematic diagram illustrates the process of adjusting the three-dimensional color lookup model to obtain the target image processing model in an embodiment of this disclosure.

[0049] Figure 19 This schematic diagram illustrates the process of adjusting the parameters in the three-dimensional color lookup model to obtain the target image processing model in an embodiment of this disclosure.

[0050] Figure 20 This schematic diagram illustrates the process of obtaining a target optimization model with the same function as the model to be learned in an embodiment of this disclosure;

[0051] Figure 21 The schematic diagram illustrates the model structure of a training method for an image processing model according to an embodiment of the present disclosure;

[0052] Figure 22 The schematic diagram illustrates the model structure of another image processing model training method in an embodiment of this disclosure;

[0053] Figure 23 The schematic diagram illustrates a flowchart of an image processing method according to an embodiment of the present disclosure;

[0054] Figure 24 This illustration schematically depicts a model training apparatus for image processing in an embodiment of the present disclosure;

[0055] Figure 25 An electronic device for a training method for an image processing model is schematically illustrated in an embodiment of the present disclosure;

[0056] Figure 26 The illustration schematically shows a computer-non-transient readable storage medium for a training method of an image processing model according to an embodiment of the present disclosure. Detailed Implementation

[0057] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided to make this disclosure more comprehensive and complete, and to fully convey the concept of the example embodiments to those skilled in the art. The described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a full understanding of embodiments of this disclosure. However, those skilled in the art will recognize that the technical solutions of this disclosure can be practiced with one or more of the specific details omitted, or other methods, components, apparatus, steps, etc., can be employed. In other instances, well-known technical solutions are not shown or described in detail to avoid obscuring various aspects of this disclosure.

[0058] The terms “a,” “an,” “the,” and “the” are used in this specification to indicate the presence of one or more elements / components / etc.; the terms “including” and “having” are used to indicate an open-ended inclusion and to mean that there may be other elements / components / etc. in addition to the listed elements / components / etc.; the terms “first” and “second” are used only as markings and are not a limitation on the number of objects.

[0059] Furthermore, the accompanying drawings are merely illustrative of this disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities.

[0060] In view of the problems existing in related technologies, this disclosure proposes a training method for an image processing model. Figure 1 A flowchart illustrating the training method for an image processing model is shown, such as... Figure 1As shown, the training method for an image processing model includes at least the following steps:

[0061] Step S110. Obtain image training samples and obtain ground truth images corresponding to the image training samples.

[0062] Step S120. Input the image training samples into the three-dimensional color lookup model to obtain the model prediction image, and perform loss calculation on the model prediction image and the ground truth image to obtain the loss calculation result.

[0063] Step S130. Adjust the three-dimensional color lookup model according to the loss calculation results to obtain the target image training model. The target image training model is used to perform image processing on the image to be processed in order to obtain the image processing result corresponding to the image to be processed.

[0064] In the methods and apparatus provided by the exemplary embodiments of this disclosure, on the one hand, the target image processing model is the result of training a three-dimensional color lookup model, which avoids the fact that the image processing results in the prior art are based on a one-dimensional color lookup table, thus ensuring the accuracy of the image processing results; on the other hand, since the three-dimensional color lookup model is three-dimensional, the three-dimensional color lookup model has a larger data capacity than the one-dimensional color lookup table in the prior art, which meets different image processing needs and expands the application scenarios of image processing.

[0065] The following section details each step of the training method for image processing models.

[0066] In step S110, image training samples are obtained, and ground truth images corresponding to the image training samples are obtained.

[0067] In the exemplary embodiments of this disclosure, image training samples refer to sample images used to train the subsequent 3D color lookup model. It is worth noting that the image training samples need to be sufficiently complex and diverse. Specifically, image training samples can include images of various brightness levels, images with various content, and images from each frame of various videos. This exemplary embodiment does not impose any special limitations on this. Ground truth images are also images used to train the 3D color lookup model. There is a one-to-one correspondence between ground truth images and image training samples. Furthermore, ground truth images are images obtained after correcting problems present in the image training samples. For example, if the image training sample is a blurry image of scene A, then the ground truth image is a clear image of scene A obtained by sharpening the image of scene A.

[0068] For example, if 10,000 training images are obtained, 5,000 of which are images and the other 5,000 are images from each frame of a video, then 10,000 ground truth images are obtained. Furthermore, there is a one-to-one correspondence between the 10,000 ground truth images and the 10,000 training images.

[0069] In this exemplary embodiment, obtaining image training samples and corresponding ground truth images ensures the subsequent training of the 3D color lookup model, thereby ensuring the training of an accurate 3D color lookup model and improving the accuracy of image processing results.

[0070] In step S120, the image training samples are input into the three-dimensional color lookup model to obtain the model prediction image, and the loss is calculated on the model prediction image and the ground truth image to obtain the loss calculation result.

[0071] In an exemplary embodiment of this disclosure, the model predicts the image as the result obtained by inputting image training samples into a three-dimensional color lookup model.

[0072] For an image, the color value of each pixel in the image has three color channels. Specifically, the three color channels are the red channel, the green channel, and the blue channel. The three-dimensional color lookup model refers to a model composed of three-dimensional color lookup tables, where the three-dimensional color lookup table refers to the color lookup table related to the above three color channels.

[0073] It is worth noting that since the 3D color lookup table is related to all three color channels, the output of the 3D color lookup table is affected by the values ​​of the three color channels. In addition, the 3D color lookup table has a huge capacity; for example, a 64-order 3D lookup table has 260,000 color output values. Furthermore, since the 3D color lookup table is essentially a numerical matrix, the process of calculating the output color using the 3D color lookup table is a differentiable process, and thus a 3D color lookup model can be created based on the 3D color lookup table.

[0074] After inputting training images into a 3D color lookup model to obtain predicted images, the loss is calculated by comparing the predicted images with the ground truth images. Specifically, the loss calculation can be performed using the mean absolute error formula, the average loss function, the mean squared loss function, a color-related loss function, a smoothing loss function, or a perceptual loss function. This exemplary embodiment does not impose any specific limitations on this method. It is worth noting that the specific loss function used for loss calculation depends on the specific image processing requirements. For example, if the image processing requirement is to obtain color restoration results, a color-related loss function can be used for loss calculation.

[0075] For example, Figure 2 A schematic diagram of the structure of the three-dimensional color lookup model in this exemplary embodiment is shown, such as... Figure 2 As shown, image 210 is a training image, model 220 is a 3D color lookup model, image 230 is a model-predicted image, image 240 is a ground truth image, and result 250 is the loss calculation result. Specifically, the loss calculation result is obtained by performing loss calculation on the model-predicted image and the ground truth image. After obtaining the loss calculation result, the parameters in the 3D color lookup model are adjusted according to the loss calculation result to obtain the target image training model.

[0076] In an optional embodiment, Figure 3 The diagram illustrates the process of obtaining the model's predicted image during the training of the image processing model. The 3D color lookup model includes a basic lookup model, such as... Figure 3 As shown, the method includes at least the following steps: In step S310, the image training samples are input into the basic lookup model; wherein, the basic lookup model is used to determine the pixel color values ​​in the image training samples and to determine the target pixel color values ​​that have a basic color mapping relationship with the pixel color values.

[0077] Specifically, the three-dimensional color lookup model can be a basic lookup model, that is, a model directly composed of three-dimensional color lookup tables. Specifically, the basic lookup model can be as follows: Figure 2 As shown in 220.

[0078] When the image training sample is input into the basic search model, the basic search model first determines the pixel color value corresponding to the image training sample, that is, determines the value of each pixel in the three color channels in the image training sample, and then determines the target pixel color value corresponding to the pixel color value based on the basic color mapping relationship.

[0079] Specifically, the basic color mapping relationship is shown in formulas (1) and (2).

[0080]

[0081]

[0082] Where i, j, and k correspond to the spatial coordinates of the three color channels of a pixel in the image training sample, respectively. Based on this, This represents the values ​​of the three color channels of the pixel at spatial coordinates (i, j, k), where N is the maximum value that i, j, and k can take, and c represents the three color channels. Specifically, r represents the red channel, g represents the green channel, and b represents the blue channel. Based on the basic color mapping relationship, This represents the target pixel color value, determined according to the basic color mapping relationship, which corresponds to the three channel color values ​​of a pixel in the image training sample.

[0083] For example, inputting image training samples into such Figure 2 In the basic search model shown in Model 220, the basic search model determines the pixel color value of each pixel in the image training sample, that is, the three-channel color value of each pixel, and then determines the target pixel color value corresponding to the pixel color value based on the basic color mapping relationship shown in Formula (2).

[0084] In step S320, the target pixel corresponding to the target pixel color value is determined, and the image composed of the target pixels is used as the model prediction image.

[0085] After determining the color value of the target pixel, a target pixel with that color value is identified, and the image composed of the target pixels is used as the image predicted by the model.

[0086] For example, if an image training sample has 1000 pixels, after determining the color values ​​of 1000 target pixels that have a basic color mapping relationship with the color values ​​of these 1000 pixels, the target pixels that have these 1000 target pixel values ​​are determined, and the image composed of these 1000 target pixels is used as the image predicted by the model.

[0087] In this exemplary embodiment, the basic lookup model is a type of three-dimensional color lookup model. The image training samples are then input into the basic lookup model, laying the foundation for obtaining the model training results and ensuring that the subsequent image processing results are highly accurate.

[0088] In an optional embodiment, Figure 4The diagram illustrates the process of obtaining the predicted image from the training method of the image processing model. The 3D search model includes a first optimization model, which is a combination of a basic search model and a first derived model. The first derived model includes a weight model and multiple basic search models, such as... Figure 4 As shown, the method includes at least the following steps: In step S410, the image training samples are input into the weight model to extract the image features corresponding to the image training samples.

[0089] The three-dimensional search model can be a first optimized model, and the first optimized model includes a combination of a basic search model and a first derived model. Specifically, it can be a linear combination of a basic search model and a first derived model, or a product combination of a basic search model and a first derived model. This exemplary embodiment does not impose any special limitations on this.

[0090] The first derived model includes multiple basic search models and a weight model, wherein the weight model is used to assign weights to the multiple basic search models in the first derived model.

[0091] At this point, the image training samples are input into the weight model, which extracts the image features of the image training samples. The image features can be the content features of the image, the texture features of the image, the color features of the image, or any kind of feature of the image. This exemplary embodiment does not impose any special limitations on this.

[0092] Figure 5 A schematic diagram of the structure of a first optimization model is shown, such as Figure 5 As shown, image 510 is the image training sample, model 520 is the first optimization model, model 521 is the basic search model, model 522 is the first derived model, the first derived model 522 contains model 530 (weight model) and multiple basic search models 540, value 550 is the weight value output by the weight model, relation 560 describes the linear combination relationship between the basic search model 521 and the first derived model 522 in the first optimization model, and relation 570 describes the model prediction image 580 obtained by inputting the image training sample into the first optimization model.

[0093] Figure 6 A schematic diagram of another first optimization model is shown, such as Figure 6 As shown, it is worth noting that relation 610 describes the product combination relationship between the basic search model 521 and the first derived model 522 at this time.

[0094] based on Figure 5 or Figure 6The image training samples are input into the weight model in the first optimization model, and the weight model will extract the image features corresponding to the image training samples.

[0095] For example, inputting image training samples into such Figure 5 In the first optimization model shown, the weight model in the first optimization model extracts the image features of the image training samples.

[0096] In step S420, the weight values ​​corresponding to the multiple basic search models in the first derived model are determined based on the image features.

[0097] In this model, after extracting image features, the weight model generates weight values ​​that are consistent with the number of basic search models in the first derived model, and these weight values ​​correspond one-to-one with the multiple basic search models in the first derived model.

[0098] For example, such as Figure 5 or Figure 6 As shown, after inputting the image training samples into the weight model 530, weight values ​​550 corresponding one-to-one with the basic search model 540 will be obtained.

[0099] In step S430, the multiple basic color mapping relationships corresponding to the multiple basic lookup models in the first derived model are updated according to the weight values ​​corresponding to the multiple basic lookup models in the first derived model, and the updated results are calculated to obtain the first derived color mapping relationship corresponding to a first derived model.

[0100] In the first derivative model, there are multiple basic search models. By using the weight values ​​corresponding to the multiple basic search models, the multiple basic color mapping relationships corresponding to the multiple basic search models in the first derivative model can be updated to obtain the first derivative color mapping relationship corresponding to the first derivative model.

[0101] For example, such as Figure 5 As shown, there is a first derived model in the first optimization model 520, and there are n basic lookup models in the first derived model. Based on this, there are n basic color mapping relationships. Specifically, these n basic color lookup models are RLUT1, RLUT2, ..., RLUTn, and RLUT1 corresponds to basic lookup model 1, RLUT2 corresponds to basic lookup model 2, and so on until the last one, RLUTn, corresponds to basic lookup model n.

[0102] In addition, there are n weight values, specifically w1, w2, ..., wn. Weight value w1 corresponds to basic search model 1, weight value w2 corresponds to basic search model 2, and so on, until the last weight value wn corresponds to basic search model n. Based on this, the first derived color mapping relationships are obtained as the product of w1 and RLUT1, the product of w2 and RLUT2, ..., the product of wn and RLUTn. Figure 6 The process and Figure 5 similar.

[0103] In step S440, based on the combination relationship between a basic search model and a first derived model corresponding to the first optimization model, the first derived color mapping relationship, and the basic color mapping relationship between a basic search model and a first optimization model, the first optimized color mapping relationship corresponding to the first optimization model is determined.

[0104] Among them, there is a combination relationship between a basic tea axis model and a first derived model in the first optimization model. Based on this combination relationship, the first derived color mapping relationship, and the basic color mapping relationship corresponding to a basic search model in the first optimization model, the first optimized color mapping relationship corresponding to the first optimization model can be determined.

[0105] For example, regarding Figure 5 For example, relation 560 describes the linear combination relationship between a basic search model and a first derived model in the first optimization model. In addition, there are n basic color mapping relationships in the first derived model. Specifically, these n basic color search models are RLUT1, RLUT2, ..., RLUTn, and RLUT1 corresponds to basic search model 1, RLUT2 corresponds to basic search model 2, and so on until the last one, RLUTn, corresponds to basic search model n.

[0106] In addition, there are n weight values, specifically w1, w2, ..., wn. Weight value w1 corresponds to basic search model 1, weight value w2 corresponds to basic search model 2, and so on, until the last weight value wn corresponds to basic search model n.

[0107] Based on this, the first optimized color mapping relationship is determined to be BLUT+w1×BLUT1+w2×BLUT2+…+wn×BLUTn.

[0108] For Figure 6 For example, relation 610 describes the product combination relationship between a basic search model and a first derived model in the first optimization model. Then the first optimization color mapping relationship is BLUT×w1×BLUT1×w2×BLUT2×…×wn×BLUTn.

[0109] In step S450, the model prediction image corresponding to the image training sample is determined according to the first optimized color mapping relationship.

[0110] The first optimized color mapping relationship is the color mapping lookup relationship corresponding to the first optimized model. Based on this, the image training samples can be input into the first optimized model to obtain the model's predicted image.

[0111] In this exemplary embodiment, the three-dimensional color lookup model can be a first optimized model, and a weight model exists in the first optimized model. The weight model can assign weights to the basic lookup model in the first derived model, thereby dynamically changing the first optimized color mapping relationship corresponding to the first optimized model through the weights, increasing the flexibility in determining the first optimized model.

[0112] In an optional embodiment, the weight model includes an image size fixed layer, multiple sampling layers, and an output layer connected in sequence. The image size fixed layer is used to fix the size of the image training samples, the sampling layers are used to extract image features corresponding to the image training samples, and the output layer is used to determine the weight values ​​corresponding to the multiple basic search models in the first derived model based on the image features.

[0113] The image size fixed layer refers to a layer structure that fixes the size of the image training samples to a specific size. For example, the size of the image training samples can be fixed to 256×256 or 512×512. This exemplary embodiment does not make any special limitation on this. Specifically, the method of fixing the size of the image training samples can be the nearest neighbor interpolation algorithm, the bilinear interpolation algorithm, the bicubic interpolation algorithm, or any other interpolation algorithm. This exemplary embodiment does not make any special limitation on this.

[0114] The sampling layer is used to extract features from the training samples of the image. Specifically, the sampling layer can consist of two convolutional layers, an activation function layer, and a global normalization layer. The second convolutional layer can also be replaced by nearest neighbor interpolation, bilinear interpolation, bicubic interpolation, or any other interpolation algorithm.

[0115] The output layer is used to output weight values ​​that correspond one-to-one with the multiple base lookup models in the first derived model based on the image features.

[0116] For example, Figure 7 The schematic diagram illustrates the structure of the weighting model, as follows: Figure 7As shown, image 710 is a training image sample, layer 720 is a fixed image size layer, layer 730 is a sampling layer, layer 740 is an output layer, and value 750 is a weight value. Layer 730 can specifically be composed of convolutional layer 731, convolutional layer 732, activation function layer 733, and global normalization layer 734. Furthermore, the convolutional kernel of layer 731 can be 3×3, and the stride of layer 732 can be 1.

[0117] In this exemplary embodiment, a weight model is provided to ensure that when image training samples are input into the weight model, weight values ​​corresponding one-to-one with the base search model in the first derived model can be obtained.

[0118] In an optional embodiment, Figure 8 The diagram illustrates the process of obtaining the predicted image from the training method of the image processing model. The 3D color lookup model includes a second optimization model, which is a combination of multiple second derived models and a basic lookup model, such as... Figure 8 As shown, the method includes at least the following steps: In step S810, the image training samples are input into multiple second derivative models to extract image features corresponding to the image training samples.

[0119] The three-dimensional search model can also be a second optimized model. Specifically, the difference between the second optimized model and the first optimized model is that the second optimized model replaces the first derived model in the first optimized model with multiple second derived models. It is worth noting that the second derived model does not include the weight model.

[0120] Based on this, the image training samples are input into multiple second-derived models to extract the image features of the image training samples.

[0121] Figure 9 A schematic diagram of one structure of the second optimization model is shown, such as Figure 9 As shown, image 910 is an image training sample, model 920 consists of multiple second-derived models, model 930 is a basic search model, relation 940 describes the linear combination relationship between multiple second-derived models and the basic search model, relation 950 is the input of image training samples into the second-derived models, and image 960 is the model prediction result.

[0122] Figure 10 The schematic diagram illustrates another structural diagram of the second optimization model, such as... Figure 10 As shown, relation 1010 describes the product combination relationship between multiple second derived models and the basic search model.

[0123] For example, based on Figure 9 or Figure 10By inputting the image training samples into multiple second-derived models, image features can be extracted from the image training samples.

[0124] In step S820, a second derived color mapping relationship between pixel color values ​​in the image training samples and target pixel color values ​​is determined based on image features.

[0125] The second derived color mapping relationship is a color mapping relationship derived from image features.

[0126] For example, in such Figure 9 or Figure 10 In this context, RLutNet1 corresponding to the second derived model 1, RLutNet1 corresponding to the second derived model 2, ..., RLutNetn corresponding to the nth derived model are the second derived color mapping relationships.

[0127] In step S830, based on the combination relationship corresponding to the combination of multiple second derived models and the basic search model, the multiple second derived color mapping relationships corresponding to the multiple second derived models, and the basic color mapping relationship corresponding to the basic search model, the second optimized color mapping relationship corresponding to the second optimized search model is determined.

[0128] The second optimized color mapping relationship can be determined based on the combination relationship between multiple second derived models and the basic search model, the multiple second derived color mapping relationships corresponding to multiple second derived models, and the basic color mapping relationship corresponding to the basic search model.

[0129] For example, regarding Figure 9 The second color mapping relationship is BLUT2 = BLUT + RLutNet1 + RLutNet2 + ... + RLutNetn, where BLUT2 is the second color mapping relationship.

[0130] For Figure 10 The second color mapping relationship is BLUT2 = BLUT × RLutNet1 × RLutNet2 × … × RLutNetn, where BLUT2 is the second color mapping relationship.

[0131] In step S840, the model prediction image corresponding to the image training sample is determined according to the second optimized color mapping relationship.

[0132] The second optimized color mapping relationship refers to the color mapping relationship corresponding to the second optimized search model. When the image training sample is input into the second optimized model, the model predicts the image based on the second optimized color mapping relationship.

[0133] For example, such as Figure 9 or Figure 10 As shown, the training image sample 910 is input into the second optimization model to obtain the model's predicted image 960.

[0134] In this exemplary embodiment, the second derived model improves the intelligence and flexibility of the model without relying on the weight model.

[0135] In an optional embodiment, the second derived model includes an image size fixing layer, a matrix transformation layer, multiple sampling layers, and an output layer connected in sequence. The image size fixing layer is used to fix the size of the image training samples, the matrix transformation layer is used to transform the matrix output by the image size fixing layer, the sampling layer is used to extract image features corresponding to the image training samples, and the output layer is used to output the second derived color mapping relationship corresponding to the image features.

[0136] The image size fixed layer refers to a layer structure that fixes the size of the image training samples to a specific size. For example, the size of the image training samples can be fixed to 256×256 or 512×512. This exemplary embodiment does not make any special limitation on this. Specifically, the method of fixing the size of the image training samples can be the nearest neighbor interpolation algorithm, the bilinear interpolation algorithm, the bicubic interpolation algorithm, or any other interpolation algorithm. This exemplary embodiment does not make any special limitation on this.

[0137] The matrix transformation layer is used to transform the matrix output of the fixed-size image layer. For example, if the image training sample is input into the fixed-size image layer, the matrix [B, C, H, W] is obtained, where C is the number of color channels, B is the batch size, H is the height of the image training sample, and W is the width of the image training sample. Inputting the matrix [B, C, H, W] into the matrix transformation layer can obtain [C, B, H, W].

[0138] The sampling layer is used to extract features from the image training samples. Specifically, the sampling layer can consist of a convolutional layer, an activation function layer, a global normalization layer, and a downsampling layer. The convolutional kernel of the convolutional layer can be 3×3. The downsampling layer can be replaced by a convolutional layer, nearest neighbor interpolation algorithm, bilinear interpolation algorithm, bicubic interpolation algorithm, or any interpolation algorithm. The number of downsampling layers is related to the number of bits in the three-dimensional color lookup table corresponding to the basic lookup model and the specific size of the image size fixed layer design. For example, when the specific size of the image size fixed layer design is 256×256 and the number of bits in the three-dimensional color lookup table is 32, 3 downsampling layers can be set. When the specific size of the image size fixed layer design is 512×512 and the number of bits in the three-dimensional color lookup table is 32, 4 downsampling layers can be set.

[0139] The output layer determines the bit depth of the second optimization model based on the feature information output by the sampling layer, so as to obtain the first derived color mapping relationship corresponding to the bit depth. Specifically, the output layer can be composed of a convolutional layer. When the bit depth of the second optimization model to be obtained is 32 bits, the parameters of the convolutional layer are set to 3×3×32. When the bit depth of the second optimization model to be obtained is 64 bits, the parameters of the convolutional layer are set to 3×3×64.

[0140] For example, Figure 11 A schematic diagram of the structure of the second derived model is shown, such as Figure 11 As shown, image 1110 is a training sample image, layer 1120 is a fixed image size layer, layer 1130 is a matrix transformation layer, layer 1140 is a sampling layer, layer 1150 is an output layer, and model 1160 is the output second derived model, which can be used to obtain the second derived color mapping relationship. The sampling layer 1140 includes a convolutional layer 1141, an activation function layer 1142, a global normalization layer 1143, and a downsampling layer 1144.

[0141] In this exemplary embodiment, a second derived lookup model is provided to ensure that after inputting image training samples into the second derived lookup model, a second derived color mapping model corresponding to the second derived lookup model can be obtained.

[0142] In an optional embodiment, the three-dimensional search model includes a combined model, which includes two first optimized models; wherein, the combination relationship of one of the optimized models in the combined model is a linear combination relationship, and the combination relationship of the other first optimized model in the combined model is a product combination relationship.

[0143] The first optimization model includes a basic search model and a first derived model. Furthermore, based on the combination relationship, it can be divided into two types of first optimization models: one is a first optimization model A, in which the basic search model and the first derived model have a linear combination relationship, and the other is a first optimization model B, in which the basic search model and the first derived model have a product combination relationship.

[0144] Based on this, a combined model can be two first optimization models with different combination relationships. That is, a combined model can be a combination of a first optimization model with a linear combination relationship and a first optimization model with a product combination relationship, and there is no restriction on the order of combination models.

[0145] For example, Figure 12 A schematic diagram of a combined search model is shown, such as Figure 12As shown, model 1210 is the first derived model, model 1220 is the basic search model, and there is a linear combination relationship between model 1210 and model 1220. Thus, model 1210 and model 1220 form the first optimization model A with a linear combination relationship, and model 1230 can be the first optimization model B with a product combination relationship.

[0146] Figure 13 The schematic diagram illustrates another structural representation of the combined model, such as... Figure 13 As shown, model 1310 can be the first optimization model A with a linear combination relationship, model 1320 is the basic search model, model 1330 is the first derived model, and the first derived model and model 1320 have a product combination relationship, that is, model 1320 and model 1330 form the first optimization model B.

[0147] In addition, the combined model can also be a second optimization model with different combination relationships. That is, the combined model can be a combination of a second optimization model with a linear combination relationship and a second optimization model with a product combination relationship, and there is no restriction on the order of combination of the combined models.

[0148] It is worth noting that the subsequent training processes for the linear combination model (first optimization model A and second optimization model C) and the product combination model (first optimization model B and second optimization model D) are different. Because of this difference, the linear combination model is more inclined to extract local features from the image training samples, while the product combination model is more inclined to extract global features. Consequently, the training results of the model corresponding to the linear combination model are more suitable for image processing tasks such as handling image details and super-resolution tasks, while the training results of the model corresponding to the product combination model are more suitable for image processing tasks such as image exposure, image dehazing, and image color correction.

[0149] In this exemplary embodiment, the three-dimensional color lookup model includes a combined model, which can be two first optimized models, and the combination relationship between these two first optimized models is different, thereby expanding the types of image tasks to which the three-dimensional color lookup model is applicable and thus expanding the application scenarios of the three-dimensional color lookup model.

[0150] In an optional embodiment, the three-dimensional color lookup model includes a combination model, which includes two second optimized models; wherein, one of the second optimized models in the combination model has a linear combination relationship of multiple second derived models and a basic lookup model, and the other second optimized model in the combination model has a product combination relationship.

[0151] Among them, the combined model can also be a second optimization model with different combination relationships. That is, the combined model can be a combination of a second optimization model with a linear combination relationship and a second optimization model with a product combination relationship, and there is no restriction on the order of the combination models.

[0152] For example, such as Figure 12 As shown, model 1210 can be multiple first derived models, model 1220 is a basic search model, and there is a linear combination relationship between model 1210 and model 1220. Thus, model 1210 and model 1220 form a second optimization model C with a linear combination relationship, and model 1230 can be a second optimization model D with a product combination relationship.

[0153] In addition, such as Figure 13 As shown, model 1310 can be a second optimization model C with a linear combination relationship, model 1320 is a basic search model, and model 1330 includes multiple second derived models. The multiple second derived models are in a product combination relationship with model 1320, that is, model 1320 and model 1330 together form the second optimization model D.

[0154] In this exemplary embodiment, the combined model includes a second optimized model with different combined relationships, which expands the types of image tasks to which the three-dimensional color lookup model is applicable, thereby expanding the application scenarios of the three-dimensional color lookup model.

[0155] In an optional embodiment, Figure 14 The diagram illustrates the process of obtaining the combined model in the training method of the image processing model. The combined model includes a combination of a first optimization model and a second optimization model. The first optimization model includes a basic search model and a first derived model. The first derived model includes a weight model and multiple basic search models, such as... Figure 14 As shown, the method includes at least the following steps: In step S1410, if the first optimization model is a linear combination of a basic search model and a first derived model, then the second optimization model is a product combination of a basic search model and multiple second derived models.

[0156] The combined model may include a first optimization model and a second optimization model. When the first optimization model in the combined model is a linear combination of a basic search model and a first derived model, the second optimization model is a product combination of a basic model and multiple second derived models. The combination order of the first optimization model and the second optimization model is not restricted. The linear combination model refers to the linear combination relationship between the basic search model and the first derived model in the first optimization model, and the product combination model refers to the product combination relationship between the basic search model and multiple second derived models in the second optimization model.

[0157] For example, such as Figure 12 As shown, when model 1210 is the first derived model, the first derived model 1210 and the basic search model 1220 form the first optimization model A with a linear combination relationship. At this time, model 1230 is the second optimization model D. That is, in the second optimization model D, there is a product combination relationship between the basic search model and multiple second derived models.

[0158] For example, such as Figure 13 As shown, when model 1310 is the first optimization model A, that is, model 1310 is a linear combination model of a basic search model and a first derived model, that is, when there is a linear combination relationship between a basic search model and a first derived model, model 1330 includes multiple second derived models, and model 1330 has a product combination relationship with basic search model 1320, that is, model 1320 and model 1330 form the second optimization model D.

[0159] In step S1420, if the first optimization model is a product combination of a basic search model and a first derived model, then the second optimization model is a linear combination of a basic search model and multiple second derived models.

[0160] The combined model may include a first optimization model and a second optimization model. When the first optimization model in the combined model is a product combination of a basic model and a first derived model, the second optimization model is a linear combination of a basic model and multiple second derived models.

[0161] For example, such as Figure 12 As shown, when model 1210 is the first derived model, and the first derived model 1210 and the basic search model 1220 form the first optimized model B with a product combination relationship, model 1230 is the second optimized model C. That is, in the second optimized model C, there is a linear combination relationship between the basic search model and multiple second derived models.

[0162] For example, such as Figure 13As shown, when model 1310 is the first optimization model B, that is, model 1310 is a product combination model of a basic search model and a first derived model, that is, when there is a linear combination relationship between a basic search model and a first derived model, model 1330 includes multiple second derived models, and model 1330 has a linear combination relationship with basic search model 1320, that is, model 1320 and model 1330 form the second optimization model C.

[0163] Similarly, there is a one-to-one correspondence between the third optimization model and the number of downsampling operations. That is, the downsampling result corresponding to downsampling factor A needs to be input into one third optimization model, the downsampling result corresponding to downsampling factor B needs to be input into another third optimization model, and so on, until all downsampling results corresponding to all downsampling factors are input into the corresponding third optimization model.

[0164] For example, if there are four downsampling results, namely downsampling result A, downsampling result B, downsampling result C, and downsampling result D, and if there is only one third optimization model, then inputting downsampling result A into the third optimization model will yield the first model output result A1, inputting downsampling result B into the third optimization model will yield the first model output result B1, inputting downsampling result C into the third optimization model will yield the first model output result C1, and inputting downsampling result D into the third optimization model will yield the first model output result D1.

[0165] If there are 4 third optimization models, the downsampling result A is input into the first third optimization model to obtain the first model output result A1, the downsampling result B is input into the second third optimization model to obtain the first model output result B1, the downsampling result C is input into the third third optimization model to obtain the first model output result C1, and the downsampling result D is input into the fourth third optimization model to obtain the first model output result D1.

[0166] In step S1540, the upsampling result is input into one or more third optimization models to obtain the second model output result corresponding to the upsampling result.

[0167] The second model output refers to the result obtained by inputting the upsampling result into the third optimization model. Specifically, if there is only one third optimization model, each upsampling result needs to be input into this third optimization model, and a corresponding second model output result will be obtained. If there are multiple third optimization models, the number of models in the multiple third optimization models is the same as the number of upsampling times. There is a one-to-one correspondence between the third optimization model and the number of upsampling times. That is, the upsampling result corresponding to the upsampling factor E needs to be input into one third optimization model, the upsampling result corresponding to the upsampling factor F needs to be input into another third optimization model, and so on, until all the upsampling results corresponding to all upsampling factors are input into the corresponding third optimization model.

[0168] For example, if there are two upsampling results, namely upsampling result E and upsampling result F, and if there is only one third optimization model, then the upsampling result E is input into the third optimization model to obtain the second model output result E1, and the upsampling result F is input into the third optimization model to obtain the second model output result F1.

[0169] If there are two third optimization models, the upsampling result A is input into the first third optimization model to obtain the second model output result e1, and the downsampling result F is input into the second third optimization model to obtain the second model output result f1.

[0170] In step S1550, the magnitude of the upsampling factor is compared to obtain the first factor comparison result, and the magnitude of the downsampling factor is compared to obtain the second factor comparison result.

[0171] The first comparison result is a comparison of the upsampling magnification, and the second comparison result is a comparison of the downsampling magnification.

[0172] For example, if the downsampling factor is 2, 4, 8, and 16, and the upsampling factor is 2 and 1.15, then the first factor comparison result is 16 > 8, 8 > 4, and 4 > 2. The second factor comparison result is 2 > 1.15.

[0173] In step S1560, based on the first magnification comparison result and the second magnification comparison result, the input-output relationship between the first model output result and the second model output result is determined, so as to obtain the model prediction image based on the input-output relationship.

[0174] Based on the first and second ratio comparison results, the input-output relationship between the first and second model outputs can be determined. Specifically, inputting the downsampled result with the highest ratio into the third optimization model will yield the first model output. At this point, the first model output and the first model output corresponding to the downsampled results of adjacent ratios can be used together as the next input to the third optimization model, and so on, until all downsampled results are input into the third optimization model.

[0175] Based on this, the first model output result obtained by inputting the last downsampling result into the third optimization model and the first upsampling result are used as the input of the new third optimization model to obtain the second model output result. The second model output result and the next upsampling result are then input into the third optimization model again, and so on, until all upsampling results are input into the third optimization model.

[0176] Figure 16 This schematic diagram illustrates the model structure when the 3D color lookup model is the third optimized model, as shown below. Figure 16 As shown, model 1610 consists of multiple third optimization models, value 1620 represents the upsampling factor, and value 1630 represents the downsampling factor. Specifically, the downsampling factors are 2, 4, 8, and 16, respectively. Based on this, the downsampling result A corresponding to downsampling factor 16 needs to be input into third optimization model 1, the downsampling result B corresponding to downsampling factor 8 needs to be input into third optimization model 2, the downsampling result C corresponding to downsampling factor 4 needs to be input into third optimization model 3, the downsampling result D corresponding to downsampling factor 2 needs to be input into third optimization model 4, the upsampling result E corresponding to upsampling factor 2 needs to be input into third optimization model 5, and the upsampling result F corresponding to upsampling factor 4 needs to be input into third optimization model 6.

[0177] The comparison process involves two steps: first, comparing the upsampling magnifications to obtain the first magnification comparison result; and second, comparing the downsampling magnifications to obtain the second magnification comparison result. Since 16 is greater than 8, 8 is greater than 4, 4 is greater than 2, and 2 is greater than 1.5, the first output of the third optimization model 1 and the downsampling result B are used as the input of the third optimization model 2. The first output of the third optimization model 2 and the downsampling result C are used as the input of the third optimization model 3, and so on, until the first output of the third optimization model 5 and the upsampling result F corresponding to the upsampling magnification of 1.5 are input into the third optimization model 6. At this point, the output of the third optimization model 6 is the target prediction image.

[0178] Figure 17 This schematic diagram illustrates the model structure when the 3D color lookup model is the third optimized model, as shown below. Figure 17As shown, there is only one first optimization model 1710, and the remaining input-output relationships are... Figure 16 same.

[0179] It is worth noting that, for Figure 16 First, we still need to obtain the ground truth image Z corresponding to the training image. Then, the downsampling result corresponding to the downsampling ratio of 16 is input into the third optimization model 1 to obtain the output result A1 of the first model. At this time, we need to calculate the loss 1 by comparing the output result A1 of the first model with the ground truth image Z. Similarly, when obtaining the output result B1 of the first model corresponding to the second optimization model 2, we still need to calculate the loss 2 by comparing the output result B1 of the first model with the ground truth image Z. And so on, until the loss 5 corresponding to the third optimization model 5 is calculated.

[0180] Based on this, we now have loss1, loss2, loss3, loss4, loss0, and loss5. Adding these together gives us the result... Figure 15 The calculation results are shown after performing loss calculations on the model.

[0181] Similarly, for Figure 17 The model shown is the same as Figure 16 The difference is that there is only one third optimization model.

[0182] For example, inputting image training samples Figure 16 The model structure shown can be used to obtain the image processing results at the corresponding magnification.

[0183] In this exemplary embodiment, on the one hand, the upsampling factor is determined according to the image processing requirements, and the upsampling factor includes a decimal factor, so model prediction images of any resolution can be flexibly obtained. On the other hand, the third optimization model can be any one of the basic search model, the first optimization model, the second optimization model, and the combined model. Therefore, different third optimization models can be set according to the image processing requirements to meet the needs of different image processing, thereby expanding the scope of application of the three-dimensional color search model.

[0184] In step S130, the three-dimensional color lookup model is adjusted according to the loss calculation results to obtain the target image processing model. The target image processing model is used to perform image processing on the image to be processed in order to obtain the image processing result corresponding to the image to be processed.

[0185] In this exemplary embodiment, the three-dimensional search model can be adjusted based on the loss calculation results to obtain the target image processing model. The image to be processed is then input into the target image processing model to obtain the image processing result corresponding to the image to be processed.

[0186] For example, the loss calculation result can be obtained by using the color loss function to calculate the image training samples and ground truth images. This result can be used to adjust the three-dimensional color search model to obtain the target image processing model A. Inputting an image with unclear colors into the target image processing model A will produce the image processing result, which is an image with restored colors.

[0187] In an optional embodiment, there is a linear combination relationship between a base search model and a derived model in the first optimization model; the target image processing model is obtained by adjusting the three-dimensional color search model according to the loss calculation result, including adjusting the weight values ​​corresponding to the multiple base search models in the first derived model and the base color mapping relationship corresponding to the base search models in the first derived model according to the loss calculation result, so as to obtain the target image processing model.

[0188] In the first optimization model, there are two types of combination relationships between the basic search model and the first derived model: linear combination relationship and product combination relationship.

[0189] For the first optimization model with linear combination relationship, the parameters of the basic search model in the first derived model are usually fixed, that is, the basic color mapping relationship corresponding to the basic search model in the first derived model is fixed. The target image processing model can be obtained by adjusting the weight values ​​of the multiple basic search models in the first derived model and the basic search model with linear combination relationship with the first derived model according to the loss calculation results.

[0190] For example, such as Figure 5 As shown, based on the loss calculation results, it is necessary to adjust the weight value 550 and the basic color mapping relationship in the first derived model 522 corresponding to the basic search model 540.

[0191] In this exemplary embodiment, when the basic search model and the first derived model are linearly combined, only the weight values ​​and the basic color mapping relationship in the first derived model corresponding to the basic search model can be adjusted. Based on this, according to different training processes, a target image processing model that is more adapted to local image features is obtained.

[0192] In an optional embodiment, a product combination relationship exists between a basic search model and a first derived model in the first optimization model. Figure 18The diagram illustrates the process of adjusting the 3D color lookup model to obtain the target image processing model in the training method of the image processing model. Figure 18 As shown, the method includes at least the following steps: In step S1810, the weight values ​​corresponding to the multiple basic search models in the first derived model and the multiple basic color mapping relationships corresponding to the multiple basic search models in the first derived model are adjusted to obtain the training results corresponding to the first derived model.

[0193] For the product combination relationship, the training process can be divided into two stages. In the first stage, only the first derived model in the first optimization model is trained to obtain the training result corresponding to the first derived model. In the second stage, the first derived model and a basic search model in the first optimization model are treated as a whole and trained to obtain the final target image processing model.

[0194] Specifically, in the first stage, the parameters of the basic search model in the first optimization model are kept unchanged, and the weight values ​​of the weight model in the first derived model and the basic color mapping relationship in the first derived model corresponding to the basic search model are adjusted to obtain the training results corresponding to the first derived model.

[0195] For example, such as Figure 6 As shown, the weight values ​​in the weight model 530 are adjusted according to the loss calculation results, and the basic color mapping relationship corresponding to the basic search model in the first derived model 522 is adjusted to obtain the training results corresponding to the first derived model 522.

[0196] In step S1820, when the training result meets the training termination condition, the first optimization model is trained according to the loss calculation result to obtain the target image processing model.

[0197] The training termination condition can be a convergence condition. Specifically, it can be the training termination condition corresponding to the first derived model. When the training result meets the training termination condition, it proves that the first stage of training has ended and the second stage of training needs to begin.

[0198] Specifically, in the second stage of training, the first optimization model needs to be trained based on the loss calculation results. That is, the basic search model in the first derived model and the first optimization model needs to be trained so that the model training results are more suitable for processing global features of the image.

[0199] For example, such as Figure 6As shown, the weight values ​​in the weight model 530 are adjusted according to the loss calculation results, and the basic color mapping relationship corresponding to the basic search model in the first derived model 522 is adjusted to obtain the training results corresponding to the first derived model 522.

[0200] When the training results meet the training termination condition, the first optimization model 520 is trained as a whole based on the loss calculation results. At this time, the basic search model 521 also needs to be adjusted until the target image processing model is obtained.

[0201] In this exemplary embodiment, when the combination of the basic search model and the first derived model is a product combination, it is necessary to first adjust the weight values ​​and the basic color mapping relationship in the first derived model corresponding to the basic search model, and then adjust the first optimization model as a whole. Based on this, according to different training processes, a target image processing model that is more adapted to global image features is obtained.

[0202] In an optional embodiment, there is a linear combination relationship between a basic search model and multiple second derived models in the second optimization model; adjusting the three-dimensional color search model according to the loss calculation results to obtain the target image processing model includes: adjusting the second derived color mapping relationship according to the loss calculation results to obtain the target image processing model.

[0203] In the second optimization model, there are two types of combination relationships between the basic search model and the second derived model: linear combination relationship and product combination relationship. For the linear combination relationship, the parameters of the basic search model in the second optimization model are usually fixed. Only the second derived color mapping relationship between the second optimization model and the multiple second derived models needs to be adjusted according to the loss calculation results.

[0204] For example, such as Figure 9 As shown, based on the loss calculation results, it is necessary to adjust the multiple second-derived color mapping relationships corresponding to multiple second-derived models 920.

[0205] In an optional embodiment, when the combination of the base search model and the second derived model is a linear combination, only the second derived color mapping relationship can be adjusted. Based on this, according to different training processes, a target image processing model that is more adapted to local image features is obtained.

[0206] In an optional embodiment, Figure 19 The diagram illustrates the process of adjusting the parameters of the 3D color lookup model to obtain the target image processing model in the training method of the image processing model. A product combination relationship exists between the basic lookup model and the second derived model in the second optimization model, such as... Figure 19As shown, the method includes at least the following steps: In step S1910, the second derived color mapping relationship is adjusted according to the loss calculation result to obtain the training result corresponding to the second derived model.

[0207] In the second optimization model, there are two types of combination relationships between the basic search model and the second derived model: linear combination and product combination. For the product combination relationship, the training process is divided into two stages. In the first training stage, the parameters of the basic search model in the second optimization model are kept unchanged. Then, based on the loss calculation results, only the second derived color mapping relationship corresponding to the second derived model is adjusted to obtain the training result corresponding to the second derived model.

[0208] For example, such as Figure 9 As shown, the second derived color mapping relationship corresponding to the second derived model 920 is adjusted according to the loss calculation results.

[0209] In step S1920, if the training result meets the training termination condition, the second optimization model is trained according to the loss calculation result to obtain the target image processing model.

[0210] The training termination condition can be a convergence condition. Specifically, it can be the convergence condition of the second derived color mapping relationship in the first training phase. When the training result meets the training termination condition, the second training phase needs to be started. That is, based on the loss calculation result, the second optimization model is adjusted as a whole so that the obtained target image processing model is more suitable for processing global image features.

[0211] For example, such as Figure 9 As shown, based on the loss calculation results, the second derived color mapping relationship corresponding to the second derived model 920 is adjusted to obtain the training results.

[0212] When the training results meet the training termination condition, the second optimization model is trained as a whole based on the loss calculation results. At this time, the parameters in the basic search model 930 also need to be adjusted until the model training results are obtained, that is, the target image processing model is obtained.

[0213] In this exemplary embodiment, when the combination of the basic search model and the second derived model is a product combination, the second derived color mapping relationship needs to be adjusted first, and then the second optimized model as a whole needs to be adjusted. Based on this, according to different training processes, a target image processing model that is more adapted to global image features is obtained.

[0214] In an optional embodiment, Figure 20The diagram illustrates the process of obtaining a target optimization model with the same functionality as the model to be learned in the training method of an image processing model. Figure 20 As shown, the method includes at least the following steps: In step S2010, the image training samples are input into the model to be learned to obtain the ground truth images corresponding to the image training samples; wherein, the model to be learned includes any one of the basic search model, the first optimization model, the second optimization model, the third optimization model, and the source model.

[0215] In this process, training images are input into the model to be learned to obtain ground truth images. It is worth noting that the model to be learned can be any one of the following: the basic search model, the first optimization model, the second optimization model, the third optimization model, and the source model. Among them, the open source model refers to a model that has been publicly disclosed and can be used directly.

[0216] For example, inputting image training samples into such Figure 15 In the third optimization model shown, the corresponding ground truth image can be obtained.

[0217] In step S2020, the image training samples are input into the target optimization model to obtain the model's predicted image; wherein, the target optimization model includes any one of the basic search model, the first optimization model, the second optimization model, and the combined model.

[0218] The target optimization model refers to any one of the basic search model, the first optimization model, the second optimization model, and the combined model.

[0219] By inputting the image into the target optimization model, the model's predicted image can be obtained.

[0220] For example, inputting image training samples into such Figure 12 The combined model shown can be used to obtain the corresponding output results, namely the model-predicted image.

[0221] In step S2030, loss is calculated on the model-predicted image and the ground truth image. Based on the loss calculation results, the target optimization model is adjusted to obtain a target optimization model with the same function as the model to be learned.

[0222] in, Figure 21 A schematic diagram of the model structure for an image processing model training method is shown, as follows: Figure 21As shown, image 2110 is the image training sample, model 2120 is the model to be learned, model 2130 is the basic search model, i.e., the target optimization model, and result 2140 is the loss calculation result. Specifically, the loss calculation result 2140 is obtained by calculating the loss based on the model's predicted image and the ground truth image. After obtaining the loss calculation result, the model mapping relationship in the basic search model is adjusted. For the basic search model, its model color mapping relationship is the basic color mapping relationship. Through the training process described above, a target optimization model with the same function as the model to be learned 2120 can be obtained.

[0223] For example, Figure 22 A schematic diagram of the model structure for training another image processing model is shown, as follows. Figure 22 As shown, model 2210 is the second optimized model. Based on the loss calculation results, the model mapping relationship of the second optimized model is adjusted. For the second optimized model, its model mapping relationship is the second derived color mapping relationship and the basic color mapping relationship. Through the training process described above, a second optimized model with the same function as the model to be learned, 2220, can be obtained.

[0224] In this exemplary embodiment, a target optimization model with the same function as the model to be learned is obtained. When the model to be learned is a complex model, a target optimization model with a simple model structure can be obtained through the above process, and the target optimization model has the same function as the model to be learned.

[0225] In the methods and apparatus provided by the exemplary embodiments of this disclosure, on the one hand, the target image processing model is the result of training a three-dimensional color lookup model, which avoids the fact that the image processing results in the prior art are based on a one-dimensional color lookup table, thus ensuring the accuracy of the image processing results; on the other hand, since the three-dimensional color lookup model is three-dimensional, the three-dimensional color lookup model has a larger data capacity than the one-dimensional color lookup table in the prior art, which meets different image processing needs and expands the application scenarios of image processing.

[0226] Furthermore, in an exemplary embodiment of this disclosure, an image processing method is also provided. Figure 23 A flowchart illustrating the image processing method is shown, such as... Figure 23 As shown, the image processing method includes at least the following steps:

[0227] Step 2310. Obtain the image to be processed and the image processing requirements.

[0228] Step 2320. Input the image to be processed and the image processing requirements into the target image processing model of the above method to obtain the image processing result.

[0229] In the methods and apparatus provided by the exemplary embodiments of this disclosure, the image to be processed and the image requirements are input into the target image processing model of the above method. Since the target image processing model can be a three-dimensional color lookup model, it not only avoids the fact that the image processing results in the prior art are based on a one-dimensional color lookup table, thus ensuring the accuracy of the image processing results, but also meets different image processing requirements and expands the application scenarios of image processing.

[0230] The following section provides a detailed explanation of each step in the image processing method.

[0231] In step S2310, the image to be processed and the image processing requirements are obtained.

[0232] Here, the image to be processed refers to the image that needs to be input into the target image processing model to obtain the image processing result, and the image processing requirement refers to the processing requirement corresponding to the problem of the image to be processed. For example, if the image to be processed is an unclear image, the image processing requirement can be the image processing requirement to make it clearer.

[0233] For example, obtain the image to be processed, XX, and find that the image processing requirement is color correction.

[0234] In step S2320, the image to be processed and the image processing requirements are input into the target image processing model of the above method to obtain the image processing result.

[0235] The target image processing model in the above method can be obtained by training a basic search model, by training a first optimized model, by training a second optimized model, by training a third optimized model, or by training a combined model.

[0236] For example, input the image processing requirements (such as color correction) of the image to be processed into a program like this: Figure 5 The first optimization model shown is used to obtain the image processing results.

[0237] The training method of the image processing model in this embodiment will be described in detail below with reference to an application scenario.

[0238] Image processing requires super-resolution output at any magnification of 2.3. In this case, the 3D color lookup model can be as follows: Figure 16 The third optimization model shown here, based on which the image training samples are input into... Figure 16 In this case, the input of the third optimization model 6 is the image processing result.

[0239] In this application scenario, on the one hand, the target image processing model is the result of training a three-dimensional color lookup model, which avoids the fact that the image processing results in the prior art are based on a one-dimensional color lookup table, thus ensuring the accuracy of the image processing results; on the other hand, since the three-dimensional color lookup model is three-dimensional, it has a larger data capacity than the one-dimensional color lookup table in the prior art, which meets different image processing needs and expands the application scenarios of image processing.

[0240] Furthermore, in an exemplary embodiment of this disclosure, an image processing model training apparatus is also provided. Figure 24 A schematic diagram of the image processing device is shown, such as... Figure 24 As shown, the image processing model training device 2400 may include: an acquisition module 2410, a loss calculation module 2420, and an adjustment module 2430. Wherein:

[0241] The acquisition module 2410 is configured to acquire image training samples and acquire ground truth images corresponding to the image training samples; the loss calculation module 2420 is configured to input the image training samples into the three-dimensional color lookup model to obtain the model prediction image, and perform loss calculation on the model prediction image and the ground truth image to obtain the loss calculation result; the adjustment module 2430 is configured to adjust the three-dimensional color lookup model according to the loss calculation result to obtain the target image training model, wherein the target image training model is used to perform image processing on the image to be processed to obtain the image processing result corresponding to the image to be processed.

[0242] The specific details of the image processing model training device 2400 have been described in detail in the corresponding image processing model training method, so they will not be repeated here.

[0243] It should be noted that although several modules or units of the image processing model training apparatus 2400 are mentioned in the detailed description above, this division is not mandatory. In fact, according to embodiments of this disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0244] Furthermore, in an exemplary embodiment of this disclosure, an electronic device capable of implementing the above-described method is also provided.

[0245] The following reference Figure 25 To describe an electronic device 2500 according to such an embodiment of the present invention. Figure 25 The electronic device 2500 shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of the present invention.

[0246] like Figure 25 As shown, the electronic device 2500 is manifested in the form of a general-purpose computing device. The components of the electronic device 2500 may include, but are not limited to: at least one processing unit 2510, at least one storage unit 2520, a bus 2530 connecting different system components (including storage unit 2520 and processing unit 2510), and a display unit 2540.

[0247] The storage unit stores program code that can be executed by the processing unit 2510, causing the processing unit 2510 to perform the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of the present invention.

[0248] Storage unit 2520 may include readable media in the form of volatile storage units, such as random access memory (RAM) 2521 and / or cache memory 2522, and may further include read-only memory (ROM) 2523.

[0249] Storage unit 2520 may also include a program / utility 2524 having a set (at least one) program module 2525, such program module 2525 including but not limited to: operating system, one or more application programs, other program modules and program data, each or some combination of these examples may contain the reality of the network environment.

[0250] Bus 2530 can represent one or more of several types of bus structures, including memory cell bus or memory cell controller, peripheral bus, graphics acceleration port, processing unit, or local bus using any of the multiple bus structures.

[0251] Electronic device 2500 can also communicate with one or more external devices 2570 (e.g., keyboard, pointing device, Bluetooth device, etc.), and with one or more devices that enable a user to interact with electronic device 2500, and / or with any device that enables electronic device 2500 to communicate with one or more other computing devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 2550. Furthermore, electronic device 2500 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 2560. As shown, network adapter 2560 communicates with other modules of electronic device 2500 via bus 2530. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 2500, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0252] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, terminal device, or network device, etc.) to execute the methods according to the embodiments of this disclosure.

[0253] In exemplary embodiments of this disclosure, a computer-readable storage medium is also provided, on which a program product capable of implementing the methods described above is stored. In some possible embodiments, various aspects of the invention may also be implemented as a program product comprising program code that, when the program product is run on a terminal device, causes the terminal device to perform the steps of the various exemplary embodiments of the invention described in the "Exemplary Methods" section above.

[0254] refer to Figure 26 As shown, a program product 2600 for implementing the above-described method according to an embodiment of the present invention is described. It may employ a portable compact disc read-only memory (CD-ROM) and include program code, and may run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited thereto. In this document, the readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, apparatus, or device.

[0255] The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0256] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium, capable of sending, propagating, or transmitting programs for use by or in conjunction with an instruction execution system, apparatus, or device.

[0257] The program code contained on the readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0258] Program code for performing the operations of this invention can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0259] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and embodiments are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the claims.

Claims

1. A training method for an image processing model, characterized in that, The method includes: Obtain image training samples and obtain ground truth images corresponding to the image training samples; The training samples of the images are input into the three-dimensional color lookup model to obtain the model's predicted images, and the loss is calculated on the model's predicted images and the ground truth images to obtain the loss calculation results; The target image processing model is obtained by adjusting the three-dimensional color lookup model based on the loss calculation results; wherein, the target image processing model is used to perform image processing on the image to be processed in order to obtain an image processing result corresponding to the image to be processed; The three-dimensional color lookup model includes one or more third optimization models. The model predicts the image by: acquiring multiple downsampling ratios and sampling the image training samples according to the multiple downsampling ratios to obtain multiple downsampling results; wherein the downsampling ratios include integer ratios; determining an upsampling ratio according to image processing requirements and sampling the image training samples according to the upsampling ratios to obtain upsampling results; wherein the upsampling ratios include decimal ratios; inputting the downsampling results into one or more of the third optimization models to obtain a first model output result corresponding to the downsampling results; inputting the upsampling results into one or more of the third optimization models to obtain a second model output result corresponding to the upsampling results; comparing the magnitudes of the upsampling ratios to obtain a first ratio comparison result, and comparing the magnitudes of the downsampling ratios to obtain a second ratio comparison result; based on the first ratio comparison result and the second ratio comparison result, determining the input-output relationship between the first model output result and the second model output result, and obtaining the model predicts the image based on the input-output relationship.

2. The training method for the image processing model according to claim 1, characterized in that, The three-dimensional color lookup model includes a first optimization model, which includes a combination of a basic lookup model and a first derived model. The first derived model includes a weight model and multiple basic lookup models. The step of inputting the image training samples into the three-dimensional color lookup model to obtain the model prediction image includes: The image training samples are input into the weight model to extract image features corresponding to the image training samples; Based on the image features, determine the weight values ​​corresponding to the multiple basic search models in the first derived model; The multiple basic color mapping relationships corresponding to the multiple basic search models in the first derivative model are updated according to the weight values ​​corresponding to the multiple basic search models in the first derivative model, and the update results are calculated to obtain the first derivative color mapping relationship corresponding to one of the first derivative models. Based on the combination relationship corresponding to the combination of the basic search model and the first derived model of the first optimization model, the first derived color mapping relationship, and the basic color mapping relationship corresponding to the basic search model in the first optimization model, the first optimized color mapping relationship corresponding to the first optimization model is determined. Based on the first optimized color mapping relationship, determine the model prediction image corresponding to the image training sample.

3. The training method for the image processing model according to claim 2, characterized in that, There is a linear combination relationship between the basic search model and the first derived model in the first optimization model; The step of adjusting the three-dimensional color lookup model based on the loss calculation results to obtain the target image processing model includes: Based on the loss calculation results, the weight values ​​corresponding to the multiple basic search models in the first derived model and the basic color mapping relationships corresponding to the basic search models in the first derived model are adjusted to obtain the target image processing model.

4. The training method for the image processing model according to claim 2, characterized in that, There is a product combination relationship between the basic search model and the first derived model in the first optimization model; The step of adjusting the three-dimensional color lookup model based on the loss calculation results to obtain the target image processing model includes: The weight values ​​corresponding to the multiple base search models in the first derived model and the multiple base color mapping relationships corresponding to the multiple base search models in the first derived model are adjusted to obtain the training results corresponding to the first derived model. When the training results meet the training termination condition, the first optimized model is trained based on the loss calculation results to obtain the target image processing model.

5. The training method for the image processing model according to claim 2, characterized in that, The weight model includes a fixed-size image layer, multiple sampling layers, and an output layer connected in sequence. The fixed-size image layer is used to fix the size of the image training samples. The sampling layers are used to extract the image features corresponding to the image training samples. The output layer is used to determine the weight values ​​corresponding to the multiple basic search models in the first derived model based on the image features.

6. The training method for the image processing model according to claim 2, characterized in that, The three-dimensional color lookup model includes a combination model, which includes two first optimization models; wherein, the combination relationship of one of the first optimization models in the combination model is a linear combination relationship, and the combination relationship of the other first optimization model in the combination model is a product combination relationship.

7. The training method for the image processing model according to claim 1, characterized in that, The three-dimensional color lookup model includes a second optimization model, which is a combination of multiple second derivative models and a basic lookup model. The step of inputting the image training samples into the three-dimensional color lookup model to obtain the model prediction image includes: The image training samples are input into multiple second derivative models to extract image features corresponding to the image training samples; Based on the image features, a second derived color mapping relationship is determined between the pixel color values ​​in the image training samples and the target pixel color values; Based on the combination relationship corresponding to the combination of multiple second derived models and the basic search model, the multiple second derived color mapping relationships corresponding to multiple second derived models, and the basic color mapping relationship corresponding to the basic search model, the second optimized color mapping relationship corresponding to the second optimized model is determined; Based on the second optimized color mapping relationship, the model prediction image corresponding to the image training sample is determined.

8. The training method for the image processing model according to claim 7, characterized in that, The three-dimensional color lookup model includes a combined model, which includes two second optimized models; wherein, in the combined model, multiple second derived models and a basic lookup model of one of the second optimized models are in a linear combination relationship, and multiple second derived models and a basic lookup model of the other second optimized model are in a product combination relationship.

9. The training method for the image processing model according to claim 8, characterized in that, The combined model includes a combination of a first optimization model and a second optimization model; the first optimization model includes a basic search model and a first derived model, and the first derived model includes a weight model and multiple basic search models. The method further includes: If the first optimization model is a linear combination of the basic search model and the first derived model, then the second optimization model is a product combination of the basic search model and multiple second derived models. If the first optimization model is a product combination of the basic search model and the first derived model, then the second optimization model is a linear combination of the basic search model and multiple second derived models.

10. The training method for the image processing model according to claim 7, characterized in that, There is a linear combination relationship between the basic search model and the multiple derived models in the second optimization model; The step of adjusting the three-dimensional color lookup model based on the loss calculation results to obtain the target image processing model includes: Based on the loss calculation results, the second derived color mapping relationship is adjusted to obtain the target image processing model.

11. The training method for the image processing model according to claim 7, characterized in that, There is a product combination relationship between the basic search model and the second derived model in the second optimization model; The step of adjusting the three-dimensional color lookup model based on the loss calculation results to obtain the target image processing model includes: Based on the loss calculation results, the second derived color mapping relationship is adjusted to obtain the training results corresponding to the second derived model; If the training results meet the training termination condition, the second optimization model is trained based on the loss calculation results to obtain the target image processing model.

12. The training method for the image processing model according to claim 7, characterized in that, The second derived model includes an image size fixed layer, a matrix transformation layer, multiple sampling layers, and an output layer connected in sequence. The image size fixed layer is used to fix the size of the image training samples. The matrix transformation layer is used to transform the matrix output by the image size fixed layer. The sampling layers are used to extract the image features corresponding to the image training samples. The output layer is used to output the second derived color mapping relationship corresponding to the image features.

13. The training method for the image processing model according to any one of claims 6, 8, and 9, characterized in that, The third optimization model is any one of the basic search model, the first optimization model, the second optimization model, and the combined model; the first optimization model includes a combination of the basic search model and the first derived model, the first derived model includes a weight model and multiple basic search models, and the second optimization model includes a combination of multiple second derived models and the basic search model.

14. The training method for the image processing model according to claim 13, characterized in that, The method further includes: The image training samples are input into the model to be learned to obtain the ground truth images corresponding to the image training samples; wherein, the model to be learned includes any one of the basic search model, the first optimized model, the second optimized model, the third optimized model, and the open source model; The image training samples are input into the target optimization model to obtain the model's predicted image; wherein, the target optimization model includes any one of the basic search model, the first optimization model, the second optimization model, and the combined model; Loss is calculated on the model-predicted image and the ground truth image. Based on the loss calculation results, the target optimization model is adjusted to obtain the target optimization model with the same function as the model to be learned.

15. The training method for the image processing model according to claim 1, characterized in that, The three-dimensional color lookup model includes a basic lookup model; The step of inputting the image training samples into the three-dimensional color lookup model to obtain the model prediction image includes: The image training samples are input into the basic lookup model; wherein, the basic lookup model is used to determine the pixel color values ​​in the image training samples and to determine the target pixel color values ​​that have a basic color mapping relationship with the pixel color values; The target pixel corresponding to the color value of the target pixel is determined, and the image composed of the target pixel is used as the model prediction image.

16. A training device for an image processing model, characterized in that, include: The acquisition module is configured to acquire image training samples and acquire ground truth images corresponding to the image training samples; The loss calculation module is configured to input the image training samples into a three-dimensional color lookup model to obtain the model prediction image, and to perform loss calculation on the model prediction image and the ground truth image to obtain the loss calculation result. The adjustment module is configured to adjust the three-dimensional color lookup model according to the loss calculation result to obtain a target image processing model; wherein, the target image processing model is used to perform image processing on the image to be processed to obtain an image processing result corresponding to the image to be processed; The three-dimensional color lookup model includes one or more third optimization models. The model predicts the image by: acquiring multiple downsampling ratios and sampling the image training samples according to the multiple downsampling ratios to obtain multiple downsampling results; wherein the downsampling ratios include integer ratios; determining an upsampling ratio according to image processing requirements and sampling the image training samples according to the upsampling ratios to obtain upsampling results; wherein the upsampling ratios include decimal ratios; inputting the downsampling results into one or more of the third optimization models to obtain a first model output result corresponding to the downsampling results; inputting the upsampling results into one or more of the third optimization models to obtain a second model output result corresponding to the upsampling results; comparing the magnitudes of the upsampling ratios to obtain a first ratio comparison result, and comparing the magnitudes of the downsampling ratios to obtain a second ratio comparison result; based on the first ratio comparison result and the second ratio comparison result, determining the input-output relationship between the first model output result and the second model output result, and obtaining the model predicts the image based on the input-output relationship.

17. An electronic device, characterized in that, include: processor; Memory for storing the executable instructions of the processor; The processor is configured to execute the training method of the image processing model according to any one of claims 1-15 by executing the executable instructions.

18. A computer non-transient readable storage medium storing a computer program thereon, characterized in that, When the computer program is executed by the processor, it implements the training method of the image processing model according to any one of claims 1-14.

19. An image processing method, characterized in that, include: Obtain the image to be processed and the image processing requirements; The image to be processed and the image processing requirements are input into the target image processing model as described in any one of claims 1-15 to obtain the image processing result.

Images