Image processing model training and image processing methods and devices

Abstract

This invention relates to an image processing model training method and apparatus, belonging to the field of image processing technology. The image processing model training method includes: training an image color encoder based on multiple sets of training data, each set of training data including a set of photoelectric data corresponding to a color space parameter and a luminance encoding curve; acquiring a training image dataset, the training image dataset including multiple training image datasets with different luminance encoding curves and / or color space parameters; inputting the training image dataset into the image color encoder to obtain conditional feature codes for the luminance encoding curve and color space parameters of the training image dataset; inputting the conditional feature codes and the training image dataset into the image processing model to be trained to obtain the processing result of the image processing model on the training image dataset; and training the parameters of the image processing model based on the processing result to obtain a trained image processing model.

Claims

1. An image processing model training method, executed by an electronic device, comprising the following steps: An image color encoder is trained using multiple sets of training data to obtain a trained image color encoder. Each set of training data includes a set of photoelectric data corresponding to a color space parameter and a luminance encoding curve. Different sets of training data include different luminance encoding curves corresponding to the photoelectric data, and / or different sets of training data include different color space parameters. Each color space parameter corresponds to one color space. A training image dataset is obtained, which includes multiple training images, each including training images with different luminance encoding curves and / or color space parameters. The training images in the training image dataset are input into the image color encoder to obtain conditional feature codes for the luminance encoding curve and color space parameters of the training images. The conditional feature codes and the training images are then input into the image processing model to be trained to obtain the processing result of the image processing model on the training images. Based on the processing result, the parameters of the image processing model are trained to obtain a trained image processing model.

2. The image processing model training method as described in claim 1, wherein before training the image color encoder based on multiple sets of training data, the method further includes: Determine the G-type luminance encoding curve and the H-type color space parameters, where G and H are both integers greater than 1; Based on each of the luminance coding curves, generate N pairs of photoelectric values ​​to obtain G sets of photoelectric data. Each set of photoelectric data includes N pairs of photoelectric values, and each pair of photoelectric values ​​includes a luminance value and a corresponding electrical signal value, where N is an integer greater than 1. Based on the G sets of photoelectric data corresponding to the G types of luminance coding curves and H types of color space parameters, generate multiple sets of training data.

3. The image processing model training method as described in claim 2, wherein N corresponding photoelectric value pairs are generated based on each of the luminance encoding curves, including: Obtain N brightness sampling points; Based on the brightness range supported by the first brightness encoding curve, N brightness sampling points are converted into N brightness values, wherein the first brightness encoding curve is any one of G types of brightness encoding curves; the N brightness values ​​are input into the photoelectric conversion function OEFT corresponding to the first brightness encoding curve to obtain N electrical signal values ​​that correspond one-to-one with the N brightness values, and the N brightness values ​​and N electrical signal values ​​constitute a set of photoelectric data; based on the one-to-one correspondence between the N brightness values ​​and the N electrical signal values, a photoelectric mapping table corresponding to the first brightness encoding curve is generated.

4. The image processing model training method as described in claim 2, wherein multiple sets of training data are generated based on G sets of photoelectric data corresponding to G types of luminance coding curves and H types of color space parameters, including: The N photoelectric value pairs corresponding to each brightness encoding curve are used as N-dimensional vector data, and each color space parameter is used as M-dimensional vector data. The N photoelectric value pairs corresponding to each brightness encoding curve are concatenated with each color space parameter to obtain multiple sets of training data. Each set of training data is an N+M dimensional vector data.

5. The image processing model training method as described in claim 2, wherein before training the image color encoder based on multiple sets of training data, it further includes at least one of the following: perturbing the parameters of at least one of the G types of luminance encoding curves to obtain P luminance encoding curves, where P is an integer greater than G; wherein, The process involves generating N pairs of photoelectric values ​​for each type of luminance encoding curve, including: generating N pairs of photoelectric values ​​for each of P luminance encoding curves to obtain P sets of photoelectric data; perturbing the electrical signal values ​​in the N pairs of photoelectric values ​​corresponding to a second luminance encoding curve for any one of the G sets of photoelectric data to obtain at least two sets of photoelectric data corresponding to the second luminance encoding curve, where the second luminance encoding curve is any one of the currently obtained luminance encoding curves; perturbing at least one of the H color space parameters to obtain Q color space parameters, where Q is an integer greater than H; and generating multiple sets of training data based on the G sets of photoelectric data corresponding to the G luminance encoding curves and the H color space parameters, including: generating multiple sets of training data based on the multiple sets of photoelectric data obtained after perturbation and the multiple color space parameters.

6. The image processing model training method according to any one of claims 1 to 5, wherein the conditional feature encoding and the training image data are input into the image processing model to be trained to obtain the processing result of the image processing model on the training image data, comprising: The training image data is input into the first network layer of the image processing model to extract the feature map of the training image data, and the feature map of the training image data is input into the next network layer. The first network layer is the first network layer in the image processing model. The conditional feature encoding is input into multiple feature fusion layers of the image processing model to obtain the feature adjustment parameters corresponding to each feature fusion layer. The multiple feature fusion layers are located in different network layers in the image processing model, and the input data of each feature fusion layer also includes the feature map output by the previous network layer. Feature fusion processing is performed by each feature fusion layer based on the corresponding feature adjustment parameters and the input feature map to obtain the fused feature, and the fused feature is input into the next network layer. The processing result of the training image data output by the second network layer of the image processing model is obtained. The second network layer is the last network layer in the image processing model.

7. The image processing model training method as described in claim 6, wherein the feature fusion layer includes a third network layer and a fourth network layer, a dot product layer and a concatenation layer; the conditional feature encoding is input into multiple feature fusion layers of the image processing model respectively to obtain the feature adjustment parameters corresponding to each feature fusion layer, including: The conditional feature encoding is then input into the third network layer and the fourth network layer, respectively. The conditional feature is encoded and mapped into C scale parameters through the third network layer, where C is the number of channels in the feature map and C is an integer greater than 1; the conditional feature is encoded and mapped into C displacement parameters through the fourth network layer; each feature fusion layer performs feature fusion processing based on the corresponding feature adjustment parameters and the input feature map to obtain the fused feature, including: performing a dot product operation between the C scale parameters and the C channel features of the input feature map through the dot product layer; The concatenation layer concatenates the C displacement parameters with the feature map output by the dot product layer.

8. An image processing method, performed by an electronic device, comprising the steps of: Obtain the image to be processed; The image color encoder encodes the conditional feature code of the image to be processed based on the actual brightness encoding curve and color space parameters used in the image to be processed. The image color encoder is the image color encoder as described in any one of claims 1 to 7. The conditional feature code of the image to be processed and the image to be processed are input into the image processing model to obtain the target image after processing of the image to be processed output by the image processing model. The image processing model is trained by the image processing model training method described in any one of claims 1 to 7.

9. An image processing model training device, which mainly comprises: The first training module is used to train an image color encoder based on multiple sets of training data to obtain a trained image color encoder. Each set of training data includes a set of photoelectric data corresponding to a color space parameter and a luminance coding curve. Different sets of training data include different luminance coding curves corresponding to the photoelectric data, and / or different sets of training data include different color space parameters. One color space parameter corresponds to one color space. The first acquisition module is used to acquire a training image dataset, which includes multiple training image datasets, and the multiple training image datasets include training image datasets with different luminance coding curves and / or color space parameters. The second training module is used to input the training image datasets from the training image dataset into the image color encoder to obtain conditional feature codes for the luminance coding curve and color space parameters of the training image datasets. The conditional feature codes and the training image datasets are then input into an image processing model to be trained to obtain the processing result of the image processing model on the training image datasets. Based on the processing result, the parameters of the image processing model are trained to obtain a trained image processing model.

10. An image processing apparatus, comprising: The second acquisition module is used to acquire the image to be processed. An image encoding module is used to encode the conditional feature encoding of the image to be processed based on the actual brightness encoding curve and color space parameters used in the image to be processed by an image color encoder, wherein the image color encoder is the image color encoder as described in any one of claims 1 to 7; an image processing module is used to input the conditional feature encoding of the image to be processed and the image to be processed into an image processing model to obtain the target image after processing of the image to be processed output by the image processing model, wherein the image processing model is trained by the image processing model training method as described in any one of claims 1 to 7.

Images