Image processing method, training method and device

Abstract

The application discloses an image processing method and device and a training method, relates to the field of image processing, and particularly relates to the fields of computer vision, deep learning and artificial intelligence. The specific implementation scheme is as follows: for each pixel point in a to-be-processed image, according to a first color value of the pixel point, a voxel index of the pixel point in a three-dimensional lookup table and a relative coordinate of the pixel point in a target voxel corresponding to the voxel index are determined; according to the voxel index, first color information of each vertex of the target voxel is acquired; a weight generation model obtained by joint training of the three-dimensional lookup table is used to determine a first weight of the vertex according to the relative coordinate; according to the first color information and the first weight, second color information of the pixel point after color mapping is determined; and a target image is generated according to the second color information of each pixel point.

Description

Technical Field

[0001] This application relates to the field of image processing, and more particularly to the fields of artificial intelligence such as computer vision and deep learning, specifically to an image processing method, training method, and apparatus. Background Technology

[0002] Color mapping refers to the process of converting the colors of an input image from a source color gamut to a target color gamut, and it is widely used in photography, display, printing, medical imaging, and computational photography. Non-parametric methods based on lookup tables are gradually becoming mainstream. Lookup tables, by discretely sampling the color space and storing the corresponding mapping results, can approximate arbitrarily complex nonlinear color transformation functions. Summary of the Invention

[0003] This application provides an image processing method, a training method, and an apparatus. The specific solutions are as follows:

[0004] According to one aspect of this application, an image processing method is provided, comprising: For each pixel in the image to be processed, the voxel index of the pixel in the three-dimensional lookup table and its relative coordinates in the target voxel corresponding to the voxel index are determined based on the first color value of the pixel. Based on the voxel index, obtain the first color information of each vertex of the target voxel; A weight generation model jointly trained with a 3D lookup table is used to determine the first weight of a vertex based on its relative coordinates. Based on the first color information and the first weight, determine the second color information of the color-mapped pixels; The target image is generated based on the second color information of each pixel.

[0005] According to another aspect of this application, a training method is provided, comprising: The first determining module is used to determine the voxel index of each pixel in the three-dimensional lookup table and its relative coordinates in the target voxel corresponding to the voxel index, based on the first color value of each pixel in the image to be processed. The acquisition module is used to obtain the first color information of each vertex of the target voxel based on the voxel index; The second determination module is used to determine the first weight of the vertex based on the relative coordinates using a weight generation model jointly trained with a 3D lookup table. The third determining module is used to determine the second color information of the color-mapped pixels based on the first color information and the first weight. The generation module is used to generate the target image based on the second color information of each pixel.

[0006] According to another aspect of this application, an image processing apparatus is provided, comprising: The first determining module is used to determine the voxel index of each pixel in the three-dimensional lookup table and its relative coordinates in the target voxel corresponding to the voxel index, based on the first color value of each pixel in the image to be processed. The acquisition module is used to obtain the first color information of each vertex of the target voxel based on the voxel index; The second determination module is used to determine the first weight of the vertex based on the relative coordinates using a weight generation model jointly trained with a 3D lookup table. The third determining module is used to determine the second color information of the color-mapped pixels based on the first color information and the first weight. The generation module is used to generate the target image based on the second color information of each pixel.

[0007] According to another aspect of this application, a training apparatus is provided, comprising: The first determining module is used to determine the first voxel index of each pixel in the initial three-dimensional lookup table and the relative coordinates of the target voxel corresponding to the first voxel index, based on the color value of each pixel in the sample image. The acquisition module is used to obtain the first color information of each vertex of the target voxel based on the first voxel index; The second determining module is used to generate a model using initial weights and determine the weights of vertices based on their relative coordinates. The third determining module is used to determine the second color information of the color-mapped pixels based on the first color information and weights. The generation module is used to generate a color-mapped image based on the second color information of each pixel. The training module is used to jointly train the initial 3D lookup table and the initial weight generation model based on the difference between the color-mapped image and the expected image corresponding to the sample image, so as to obtain the trained 3D lookup table and weight generation model.

[0008] According to another aspect of this application, an electronic device is provided, comprising: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the method described in the above embodiments.

[0009] According to another aspect of this application, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause the computer to perform the method described in the above embodiments.

[0010] According to another aspect of this application, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps of the method described in the above embodiments.

[0011] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this application, nor is it intended to limit the scope of this application. Other features of this application will become readily apparent from the following description. Attached Figure Description

[0012] The accompanying drawings are provided for a better understanding of this solution and do not constitute a limitation of this application. Wherein: Figure 1 A schematic flowchart of an image processing method provided in an embodiment of this application; Figure 2 A schematic flowchart illustrating an image processing method provided in another embodiment of this application; Figure 3 A schematic flowchart illustrating an image processing method provided in another embodiment of this application; Figure 4 A schematic flowchart illustrating a training method provided in an embodiment of this application; Figure 5 This is a schematic diagram of the structure of an image processing apparatus provided in an embodiment of this application; Figure 6 This is a schematic diagram of the structure of a training device provided in one embodiment of this application; Figure 7 This is a block diagram of an electronic device used to implement the image processing method of the embodiments of this application. Detailed Implementation

[0013] The following description, in conjunction with the accompanying drawings, illustrates exemplary embodiments of this application, including various details to aid understanding. These should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this application. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0014] It should be noted that the acquisition, storage, use, and processing of data in this application comply with the relevant provisions of national laws and regulations and do not violate public order and good morals.

[0015] The image processing method, training method, apparatus, electronic device, and storage medium of this application are described below with reference to the accompanying drawings.

[0016] Figure 1 This is a schematic flowchart of an image processing method provided in an embodiment of this application.

[0017] The image processing method of this application embodiment can be executed by the image processing device of this application embodiment, which can be configured in an electronic device.

[0018] Among them, electronic devices can be any device with computing capabilities, such as personal computers, mobile terminals, servers, etc. Mobile terminals can be hardware devices with various operating systems, touch screens and / or displays, such as in-vehicle devices, mobile phones, tablets, personal digital assistants, wearable devices, etc.

[0019] like Figure 1 As shown, the image processing method includes: Step 101: For each pixel in the image to be processed, determine the voxel index of the pixel in the three-dimensional lookup table and its relative coordinates in the target voxel corresponding to the voxel index based on the first color value of the pixel.

[0020] In this application, the image to be processed can be the original image, or it can be an image obtained by normalizing the color values ​​of the pixels in the original image, and there is no limitation on this.

[0021] For example, if the original image uses the RGB color space, the values ​​of the R, G, and B color channels of the pixels in the original image can be normalized to the range of [0,1] to obtain the image to be processed.

[0022] For example, if the color space of the image to be processed is RGB space, then the first color value may include the components on the three color channels R, G and B. The first color value can be in the range of [0,255] or in the range of [0,1], without limitation.

[0023] In this application, the three-dimensional lookup table can be obtained by joint training with the weight generation model. That is, the initial three-dimensional lookup table and the initial weight generation model can be jointly trained to obtain the trained three-dimensional lookup table and the weight generation model.

[0024] Taking the RGB color space as an example, the RGB color space can be uniformly quantized into N×N×N grid points. Each grid point stores the target color or color increment. Eight adjacent grid points can form a small cube, or a voxel, where N is an integer greater than 1. Then, the three-dimensional lookup table can be a learnable lookup table tensor of size N×N×N×3, where N represents the dimension of the three-dimensional lookup table and 3 represents the three color channels R, G, and B.

[0025] In this application, for each pixel in the image to be processed, coordinate mapping and voxel localization can be performed based on the first color value of the pixel to determine the voxel index of the pixel in the three-dimensional lookup table, as well as the relative coordinates of the pixel in the target voxel corresponding to the voxel index.

[0026] For example, a voxel index can be understood as a grid point index, that is, a voxel index identifies a grid point. A voxel index can also be used to identify the voxel to which a pixel belongs in a 3D lookup table. This voxel can be called a target voxel. The target voxel has 8 vertices, and the grid point identified by the voxel index is one of the 8 vertices of the target voxel. The relative coordinates of a pixel in the target voxel can be understood as the position coordinates of the pixel in the target voxel.

[0027] It should be noted that the RGB color space mentioned above is just an example, and other color spaces can also be used. If the other color space does not have three color channels, the corresponding three-dimensional lookup table should be replaced with a lookup table with the same number of channels as the other color space.

[0028] Step 102: Obtain the first color information of each vertex of the target voxel based on the voxel index.

[0029] In this application, the first color information can be a color value or a color increment, and there is no limitation on it.

[0030] For example, since each grid point in the 3D lookup table stores color information such as color value or color increment, the eight vertices of the target voxel can be determined based on the voxel index, and the first color information of these eight vertices can be obtained from the 3D lookup table.

[0031] For example, the color information stored in the grid points of the 3D lookup table can be obtained from training.

[0032] Step 103: Using the weight generation model jointly trained with the 3D lookup table, the first weight of the vertex is determined based on the relative coordinates.

[0033] In this application, the weight generation model can be jointly trained with a three-dimensional lookup table. The relative coordinates of the pixel in the three-dimensional lookup table can be input into the weight generation model to obtain the first weight of each vertex of the target voxel to which the pixel belongs, as output by the model.

[0034] Taking the RGB color space as an example, the first weight of each of the eight vertices of the target voxel can be obtained.

[0035] For example, the first weights of each vertex of the target voxel can be the same or the same, and there is no limitation on this.

[0036] Step 104: Determine the second color information of the color-mapped pixels based on the first color information and the first weight.

[0037] In this application, the second color information of the pixel after color mapping can be obtained by weighted summation based on the first color information and first weight of each vertex.

[0038] For example, if the color information stored at each grid point in the 3D lookup table is a color value, then both the first color information and the second color information are color values; if the color information stored at each grid point in the 3D lookup table is a color increment, then both the first color information and the second color information are color increments.

[0039] Step 105: Generate the target image based on the second color information of each pixel.

[0040] In this application, a target image can be generated based on the second color information of each pixel in the image to be processed, combined with the type of the second color information.

[0041] In some embodiments, if the second color information is a color value, then the target image can be directly obtained based on the second color information of each pixel in the image to be processed. That is, the color value of each pixel in the image to be processed is mapped to the corresponding second color information to obtain the target image.

[0042] In some embodiments, if the second color information is a color increment, the target image is obtained by residual mapping based on the first color value and the second color information of each pixel in the image to be processed.

[0043] The image processing method of this application embodiment can be applied to image style processing, image filter processing, etc., and is not limited thereto.

[0044] As an application example, this method can be applied to image filters on mobile devices or in the cloud. For instance, it can be used to apply the image processing method of this application to user-taken photos or album images in real time or offline.

[0045] As an application example, this method can be applied to camera ISP (Image Signal Processing) or post-processing. For instance, after white balance, exposure, and color style processing, color mapping can be performed using the image processing method of this application embodiment to output a more stable and consistent stylized image.

[0046] As an application example, it can be used for video color grading. For instance, the same 3D lookup table parameters can be applied to each frame of a video to ensure temporal consistency, and the weight generation model can generate weights based on the coordinates of pixels to adapt to different color gamuts.

[0047] As an application example, this method can be used for batch style migration and material unification. For instance, in material library management, content production, and advertising, the image processing method described in this application can be used to perform consistent style mapping on a large number of images.

[0048] In this embodiment, by mapping the first color value of each pixel in the image to be processed to the target voxel in the three-dimensional lookup table, and using a weight generation model jointly trained with the three-dimensional lookup table, the interpolation weights of each vertex are dynamically generated based on the relative coordinates within the voxel, thereby achieving adaptive fusion of the color information of the vertices and adaptive fusion of different color gamut regions. This can reduce mapping errors under complex color distributions, reduce problems such as banding, color cast and detail loss, and improve the quality of the target image.

[0049] Figure 2 This is a schematic flowchart of an image processing method provided in another embodiment of this application.

[0050] like Figure 2 As shown, the image processing method includes: Step 201: For each pixel in the image to be processed, determine the continuous voxel coordinates of the pixel in the three-dimensional lookup table based on the first color value of the pixel and the dimension of the three-dimensional lookup table.

[0051] In this application, the first color value of a pixel can be mapped to the index range of the three-dimensional lookup table based on the first color value of the pixel and the dimension of the three-dimensional lookup table, thereby obtaining the continuous voxel coordinates of the pixel in the three-dimensional lookup table. In other words, based on the dimension of the three-dimensional lookup table, the first color value of the pixel is mapped to continuous voxel coordinates in the three-dimensional lookup table.

[0052] To save storage space, the dimensions of a 3D lookup table are usually less than 256. Therefore, the first color value may not necessarily be mapped to a grid point in the 3D lookup table, but may be mapped to a voxel. Thus, the continuous voxel coordinates in this application are floating-point continuous coordinates.

[0053] In some embodiments, the index range of the three-dimensional lookup table can be determined based on the dimensions of the three-dimensional lookup table, and the continuous voxel coordinates can be determined based on the first color value and the upper limit of the index range.

[0054] For example, if the dimension of a three-dimensional lookup table is N, then the index range of the three-dimensional lookup table is [0, N-1], where N is an integer greater than 1.

[0055] Taking the RGB color space as an example, if the image to be processed is the original image, then the product of the ratio of the first color value to 255 and the upper limit of the index range can be used as the continuous voxel coordinates. For example, the continuous voxel coordinates can be calculated using the following formula (1): (1) in, Represents the coordinates of a continuum element. This represents the unnormalized first color value. Indicates the upper limit of the index range.

[0056] Taking the RGB color space as an example, if the image to be processed is an image with normalized pixel color values, that is, the first color value is in the range of [0,1], the product of the first color value and the upper limit of the index range can be used as the continuous voxel coordinates. For example, the continuous voxel coordinates can be calculated using the following formula (2): (2) in, This represents the first color value obtained after normalization.

[0057] For example, if a pixel's first color value is (0.3, 0.6, 0.1), and the dimension N=33 of the 3D lookup table, then the corresponding continuous voxel coordinates of that pixel are... .

[0058] Therefore, by determining the index range of the three-dimensional lookup table based on its dimensions, and by determining the continuous voxel coordinates of the pixel in the three-dimensional lookup table based on the first color value of the pixel and the upper limit of the index range, the first color value of the pixel is mapped to the index range of the three-dimensional lookup table, which facilitates color mapping using the three-dimensional lookup table and improves the accuracy of color mapping.

[0059] Step 202: Round down the continuous voxel coordinates to obtain the voxel index.

[0060] In this application, the integer part of the continuous voxel coordinates is obtained by rounding down, and this integer part is used as the voxel index. For example, the voxel index can be the largest integer less than or equal to the continuous voxel coordinates.

[0061] For example, the voxel index can be obtained using the following formula (3): (3) in, Indicates voxel index, Represents the coordinates of a continuum element. The function's purpose is to round a floating-point number down to the largest integer not greater than that floating-point number.

[0062] Step 203: Determine the relative coordinates based on the difference between the continuous floating-point coordinates and the voxel index.

[0063] In this application, the difference between continuous floating-point coordinates and voxel indices can be used as the relative coordinates of a pixel within a target voxel. It is evident that the relative coordinates are the fractional part of the continuous voxel coordinates. For example, the value of the relative coordinates can be in the range [0,1].

[0064] For example, the continuous voxel coordinates corresponding to a pixel are voxel index relative coordinates .

[0065] Step 204: Obtain the first color information of each vertex of the target voxel based on the voxel index.

[0066] Step 205: Using the weight generation model jointly trained with the 3D lookup table, the first weight of the vertex is determined based on the relative coordinates.

[0067] Step 206: Determine the second color information of the color-mapped pixels based on the first color information and the first weight.

[0068] Step 207: Generate the target image based on the second color information of each pixel.

[0069] In this application, steps 204-207 can be implemented in any of the embodiments of this application, so they will not be described in detail here.

[0070] In this embodiment, the first color value of each pixel in the image to be processed is linearly mapped to continuous voxel coordinates according to the dimensions of the three-dimensional lookup table. Then, the coordinates are rounded down to obtain an integer voxel index. The difference between the continuous coordinates and the voxel index is used to obtain the relative coordinates within the voxel. This accurately describes the spatial positional relationship of the first color value of the pixel in the discrete grid of the three-dimensional lookup table. This coordinate decomposition mechanism can not only maintain the geometric consistency from the color space to the index space of the three-dimensional lookup table, but also provide standardized and normalized local positional information for subsequent interpolation operations. On the one hand, it can ensure that each pixel in the image to be processed is accurately assigned to the corresponding voxel, avoiding color mapping errors caused by cross-voxel mislocation. On the other hand, the generated relative coordinates, as differentiable continuous variables, can effectively drive the weight generation model to perform end-to-end learning, improve the sensitivity and adaptability of the interpolation weights to local color gamut changes, and thus enhance the accuracy, smoothness and robustness of the overall color mapping, while being compatible with the deployment requirements of three-dimensional lookup tables of arbitrary resolution.

[0071] Figure 3 This is a schematic flowchart of an image processing method provided in another embodiment of this application.

[0072] like Figure 3 As shown, the image processing method includes: Step 301: For each pixel in the image to be processed, determine the voxel index of the pixel in the three-dimensional lookup table and its relative coordinates in the target voxel corresponding to the voxel index based on the first color value of the pixel.

[0073] In this application, step 301 can be implemented in any of the embodiments of this application, so it will not be described in detail here.

[0074] Step 302: Determine the index position of the vertex in the 3D lookup table based on the voxel index.

[0075] In this application, the index position of each vertex of the target voxel in the three-dimensional lookup table can be determined by traversing the offset of each color channel direction in the color space of the three-dimensional lookup table based on the voxel index.

[0076] For example, the index location can be understood as the storage location of color information in a three-dimensional lookup table.

[0077] Taking the RGB color space of the 3D lookup table as an example, the 8 vertices of the target voxel are the voxel index and its 7 adjacent grid points. By traversing the 0 or 1 offsets in the R, G, and B directions, the index positions of the 8 vertices in the 3D lookup table are determined.

[0078] For example, the voxel index of the target voxel is Then the index positions of the 8 vertices in the 3D lookup table are: , , , ..., .

[0079] Step 303: Obtain the first color information of the vertex from the 3D lookup table according to the index position.

[0080] In this application, since each grid point in the three-dimensional lookup table stores corresponding color information, the first color information of each vertex can be obtained from the three-dimensional lookup table based on the index position of each vertex of the target voxel.

[0081] For example, the first color information can be a color value or a color increment, etc.

[0082] Step 304: Using the weight generation model jointly trained with the 3D lookup table, the first weight of the vertex is determined based on the relative coordinates.

[0083] In this application, step 304 can be implemented in any of the embodiments of this application, so it will not be described in detail here.

[0084] Step 305: Determine the second color information of the color-mapped pixels based on the first color information and the first weight.

[0085] In this application, step 305 can be implemented in any of the embodiments of this application, so it will not be described in detail here.

[0086] To improve the accuracy of interpolated colors, in some embodiments, the first weight of each vertex can be normalized according to the first weight of each vertex of the target voxel to which the pixel belongs, so as to obtain the normalized second weight of each vertex. Then, the first color information is weighted according to the second weight of each vertex to obtain the second color information of the pixel after color mapping.

[0087] For example, the weight normalization can be performed using the following formula (4), and the weighted summation can be performed using formula (5) to obtain the second color information: (4) (5) in, Indicates the first The second weight of each vertex Indicates the first The first weight of each vertex , The values ​​are 1, 2, ..., 8. This represents the second color information of the pixel. Indicates the first The first color information of each vertex.

[0088] For example, the first weight of each vertex can be normalized using the following method: the first weights of all vertices of the target voxel can be added together to obtain a weight sum. Based on this weight sum, the first weights can be normalized to obtain the normalized second weight of each vertex. For example, the ratio of the first weight of each vertex to the weight sum can be used as the second weight of each vertex.

[0089] For example, the second weight can be calculated using the following formula (6): (6) in, This represents the sum of the first weights of the 8 vertices.

[0090] Therefore, by normalizing the first weight of each vertex of the target voxel, the original weight is transformed into a second weight that satisfies the probability distribution characteristics, thereby ensuring that the weighted interpolation result has good numerical stability and physical meaning in the color space. On this basis, the first color information stored in the voxel vertex is weighted and fused using the normalized second weight. This not only effectively avoids color overexposure, offset or numerical overflow caused by unnormalized weights, but also enables the interpolation process to more accurately reflect the continuous change characteristics of the local color field, significantly improving the smoothness and accuracy of color mapping. In particular, it can effectively suppress color level breakage and artifacts in high saturation or gradient regions, ultimately achieving high-quality, robust pixel-level color correction or stylized mapping.

[0091] Step 306: Generate the target image based on the second color information of each pixel.

[0092] In this application, step 306 can be implemented in any of the embodiments of this application, so it will not be described in detail here.

[0093] In some embodiments, if each grid point in the three-dimensional lookup table stores color increments, that is, the second color information is a color increment, the first color value of each pixel in the image to be processed can be added to the color increment corresponding to that pixel to obtain the second color value of that pixel. Since the second color value may exceed the color value range, the second color value of each pixel can be cropped to obtain the target image.

[0094] For example, the first color value can be added to the component on the same color channel in the color increment to obtain the second color value.

[0095] For example, the components in the R channel of the first color value can be added to the components in the R channel of the color increment, the components in the G channel of the first color value can be added to the components in the G channel of the color increment, and the components in the B channel of the first color value can be added to the components in the B channel of the color increment.

[0096] For example, the second color value of each pixel can be cropped as follows: determine the color value range corresponding to the image to be processed; if the second color value of any pixel is not within the color value range, crop the second color value of that pixel to the color value range, thereby obtaining the target image.

[0097] Taking the RGB color space of the three-dimensional lookup table as an example, if the image to be processed is the original image, then the color value range is [0, 255], and if the image to be processed is a normalized image, then the color value range is [0, 1].

[0098] For example, if any component of a color channel in the second color value is not within the color value range, the second color value can be considered not to be within the color value range. That is, if any component of a color channel in the second color value is not within the color value range, the component of that color channel can be cropped into the color value range.

[0099] For example, when cropping, the components of the color channel that are not within the color value range in the second color value can be cropped to the upper or lower limit of the color value range that is closest to them.

[0100] For example, if the color value range is [0,1], and the first color value of a pixel is (0.3, 0.6, 0.1), then the color increment of that pixel is (0.2, 0.6, 0.1). The first color value (0.1, 0.5) is added to the component of the same color channel in the color increment to obtain the second color value (0.5, 0.5, 0.6). Within this color value range, no cropping is needed. If the color increment of this pixel is (0.8, ... If the values ​​are 0.3, 1.2), then the second color value is (1.1, 0.3, 1.3). The components in both the R and B channels exceed 1, so they need to be cropped. Since the closest value to the R channel component 1.1 is the upper limit of the color value range [0,1], the R channel component 1.1 can be cropped to 1. The closest value to the B channel component 1.3 is the upper limit of the color value range [0,1], so the B channel component 1.1 can be cropped to 1.0. Thus, the cropped color value of this pixel is (1.0, 0.3, 1.0).

[0101] Therefore, by first determining the color value range of the image to be processed, and then performing a clipping operation on the second color value that exceeds the range after color mapping, color overflow caused by residual superposition can be effectively prevented, thus improving image quality and enhancing stability.

[0102] In this embodiment, by storing color increments in a three-dimensional lookup table, the second color value of each pixel is obtained by adding its first color value to the corresponding color increment. This method can not only accurately correct local color deviations or enhance specific visual effects, but also achieve fine adjustments while maintaining the overall tone and brightness of the original image. Furthermore, by performing a cropping process on the calculated second color value, it is ensured that all pixel values ​​fall within the valid color range, thus avoiding distortion problems caused by color overflow.

[0103] In this embodiment, by utilizing the voxel index of the target voxel to which the pixel belongs in the 3D lookup table, the index position of each vertex of the target voxel in the 3D lookup table is accurately located. Based on this, the first color information of each vertex is read from the learnable 3D lookup table, providing a structured and spatially consistent local color reference for subsequent interpolation. This voxel geometry-based indexing and lookup mechanism ensures that the color mapping process strictly follows the topological continuity of the color space, avoiding cross-regional misreading or non-neighborhood interference. Thus, while maintaining computational efficiency, it can significantly improve the accuracy and local correlation of the interpolation input data. Especially in high-resolution 3D lookup tables or complex tone mapping tasks, it can lay a reliable data foundation for achieving smooth, artifact-free color transformations, ensuring high-quality pixel-level color correction.

[0104] To implement the above embodiments, this application also proposes a training method. Figure 4 This is a schematic flowchart of a training method provided in an embodiment of this application.

[0105] like Figure 4 As shown, the training method includes: Step 401: Based on the color value of each pixel in the sample image, determine the first voxel index of the pixel in the initial three-dimensional lookup table and the relative coordinates of the pixel within the target voxel corresponding to the first voxel index.

[0106] In this application, the sample image can be the original image or an image obtained by normalizing the color values ​​of the pixels in the original image.

[0107] For example, each grid point in the initial 3D lookup table can store color values, or it can store color increments; there is no limitation on this.

[0108] For example, the explanation of the first voxel index and relative coordinates can be found in the explanation of the voxel index and relative coordinates of the pixel in the three-dimensional lookup table in the above embodiments, so it will not be repeated here.

[0109] In this application, the method of determining the first voxel index of each pixel in the initial three-dimensional lookup table and the relative coordinates of each pixel in the target voxel corresponding to the first voxel index based on the color value of each pixel in the sample image is similar to the method of determining the voxel index of the pixel in the three-dimensional lookup table and the relative coordinates of each pixel in the target voxel corresponding to the voxel index based on the first color value of the pixel in the above embodiments. Any of the above embodiments can be referred to, so it will not be described again here.

[0110] Step 402: Obtain the first color information of each vertex of the target voxel based on the first voxel index.

[0111] In this application, the first color information of each vertex of the target voxel is obtained according to the first voxel index. This can be referred to in any of the above embodiments for obtaining the first color information of each vertex of the target voxel according to the voxel index, so it will not be described again here.

[0112] Step 403: Use the initial weight generation model to determine the weight of the vertex based on the relative coordinates.

[0113] In this application, the method of determining the weight of a vertex based on relative coordinates using an initial weight generation model can be found in the explanation of determining the first weight of a vertex based on relative coordinates using the weight generation model in the above embodiments, so it will not be repeated here.

[0114] Step 404: Determine the second color information of the color-mapped pixels based on the first color information and weights, and generate the color-mapped image based on the second color information of each pixel.

[0115] In this application, the method for determining the second color information of the color-mapped pixel based on the first color information and the weight can be found in any of the above embodiments for determining the second color information of the color-mapped pixel based on the first color information and the first weight, and therefore will not be described again here.

[0116] In this application, the method for generating a color-mapped image based on the second color information of each pixel can be found in any of the above embodiments for generating a target image based on the second color information of each pixel, and therefore will not be described again here.

[0117] Step 405: Based on the difference between the color-mapped image and the expected image corresponding to the sample image, the initial 3D lookup table and the initial weight generation model are jointly trained to obtain the trained 3D lookup table and weight generation model.

[0118] In this application, taking stylization processing as an example, the expected image can be the target style image; taking image correction as an example, the expected image can be the correction result image, etc.

[0119] In some embodiments, the pixel reconstruction loss can be determined based on the difference between the color-mapped image and the expected image. Based on the pixel reconstruction loss, the learnable parameters of the initial 3D lookup table and the model parameters of the weight generation model are jointly adjusted until the training termination condition is met, thus obtaining the trained 3D lookup table and weight generation model.

[0120] For example, the training termination condition could be that the pixel reconstruction loss is less than a preset threshold, or that a preset number of training iterations has been reached, or other conditions, which are not limited to this.

[0121] As one possible implementation, the pixel reconstruction loss can be calculated as follows: the difference between the color-mapped image and the expected image at each pixel can be calculated, and the absolute values ​​of the differences at each pixel can be summed to obtain the pixel reconstruction loss. That is, the L1 norm can be calculated based on the color-mapped image and the expected image, and the L1 norm can be used as the pixel reconstruction loss.

[0122] As another possible implementation, the pixel reconstruction loss can also be calculated as follows: calculate the difference between the color-mapped image and the expected image at each pixel and color channel, square the difference, sum the squared differences over all pixels and color channels, and take the square root of the sum to obtain the pixel reconstruction loss. That is, calculate the L2 norm based on the color-mapped image and the expected image, and use the L2 norm as the pixel reconstruction loss.

[0123] Considering that color mapping through a 3D lookup table may result in color level breaks, in some embodiments, the second voxel index of the adjacent voxels of the target voxel can be determined based on the first voxel index, and the third color information corresponding to the first voxel index can be obtained from the 3D lookup table based on the first voxel index. The fourth color information corresponding to the second voxel index can be obtained from the 3D lookup table based on the second voxel index. The lookup table smoothing regularization value is determined based on the third and fourth color information. The total loss is determined based on the pixel reconstruction loss and the lookup table smoothing regularization value. Based on the total loss, the initial 3D lookup table and the initial weight generation model are jointly trained.

[0124] For example, any of the above methods can be used to determine the pixel reconstruction loss, so they will not be described in detail here.

[0125] For example, the values ​​of each color channel in the first voxel index can be incremented by 1 to obtain the second voxel index.

[0126] For example, the lookup table smoothing regularity is a key regularity term used to constrain the smoothness of the three-dimensional lookup table space.

[0127] For example, the difference between the fourth color information and the third color information on the same color channel can be calculated, and the absolute values ​​of the differences can be summed to obtain the lookup table smoothing regularity value. This lookup table smoothing regularity value can be used to penalize drastic jumps in color values ​​of adjacent grid points in the three-dimensional lookup table, thereby avoiding banding or artifacts in the output image caused by the non-smoothness of the three-dimensional lookup table.

[0128] For example, the smoothing regularity value of the lookup table can be calculated using the following formula (7). : (7) in, This indicates the component of the fourth color information in the R channel. This indicates the component of the third color information in the R channel. This indicates the component of the fourth color information in the G channel. This indicates the component of the third color information in the G channel. This indicates the component of the fourth color information in the B channel. This indicates the component of the third color information in the B channel.

[0129] For example, the pixel reconstruction loss and the lookup table smoothing regularization value can be weighted to obtain the total loss.

[0130] For example, the total loss can be determined using the following formula (8). : (8) in, Indicates pixel reconstruction loss, The weights of the lookup table's smoothing regularization values ​​can be hyperparameters.

[0131] Therefore, by determining the total loss based on pixel reconstruction loss and lookup table smoothing regularization value, and using this to jointly train the 3D lookup table and weight generation model, the local smoothness of the 3D lookup table in the color space can be effectively constrained while ensuring that the output image closely approximates the target style or correction result. Especially in low-resolution 3D lookup tables or high dynamic range content, the smoothing regularization guides the interpolation base data to exhibit continuous changes, which can enhance generalization ability and deployment robustness, ultimately achieving high-quality, stable, and visually natural color mapping effects with limited storage and computational overhead.

[0132] To limit the weights from being too sharp or ringing, in some embodiments, entropy calculation can be performed on the weights of each vertex to obtain a weight distribution regularization value. Based on the pixel reconstruction loss and the weight distribution regularization value, the initial 3D lookup table and the initial weight generation model are jointly trained.

[0133] For example, a weight distribution regularization value can be used to limit the weights of vertices from being too sharp or ringing, and minimizing this value can make the weight distribution smoother.

[0134] For example, the weight distribution regularization value can be calculated using the following formula (9). : (9) in, The first value of the target voxel to which the pixel belongs The normalized weights of the vertices are: This represents a minimum value.

[0135] For example, the pixel reconstruction loss and the weight distribution regularization value can be weighted to obtain the total loss, and the initial 3D lookup table and the initial weight generation model can be jointly trained based on the total loss.

[0136] For example, the total loss can be calculated using the following formula (10). : (10) in, The weights represent the regularization values ​​of the weight distribution and can be hyperparameters.

[0137] Therefore, by jointly training the initial 3D lookup table and the initial weight generation model based on the pixel reconstruction loss and the weight distribution regularization value, we can not only ensure that the output image closely approximates the target style or correction result in terms of content, but also guide the interpolation weight distribution to be more reasonable, avoiding excessive concentration of weights on a single vertex or the occurrence of extremely sparse or sharp responses. This makes the interpolation process more stable and has better generalization ability, reducing color overshoot, ringing effect or detail distortion caused by abnormal weight amplification of local noise or quantization error. It can improve the accuracy and naturalness of color mapping in complex color gamut regions.

[0138] In some embodiments, the total loss can be calculated based on the pixel reconstruction loss, the lookup table smoothing regularization value, and the weight distribution regularization value, and the total loss can be used to jointly train the initial 3D lookup table and the initial weight generation model.

[0139] For example, the total loss can be calculated using the following formula (11). : (11) Therefore, by jointly training the initial 3D lookup table and the initial weight generation model based on pixel reconstruction loss, lookup table smoothing regularization value and weight distribution regularization value, we can not only ensure that the output image approximates the target style or correction result in terms of content, but also effectively constrain the local smoothness of the 3D lookup table in the color space, guide the interpolation weight distribution to be more reasonable, limit the weights to be too sharp or ringing, and improve the accuracy and naturalness of color mapping in complex color gamut regions.

[0140] In this embodiment, by jointly training the initial 3D lookup table and the initial weight generation model based on the difference between the color-mapped image and the expected image corresponding to the sample image, it is possible not only to accurately fit the complex nonlinear color mapping relationship, but also to avoid abrupt output changes in adjacent color regions caused by overfitting the 3D lookup table parameters to the training samples. This significantly suppresses visual defects such as banding, artifacts, and noise amplification, improves the accuracy of color mapping, and achieves higher color reconstruction accuracy and visual consistency under low-resolution 3D lookup tables.

[0141] The training method of this application embodiment can introduce a learnable interpolation weight generation mechanism while maintaining the exportability, deployability, and compatibility of the three-dimensional lookup table with existing color grading links. This enables adaptive weighting of different color gamut regions, which can reduce error diffusion, color gradation breakage, color cast, and detail loss in dark areas, high saturation, and high contrast areas. Under the condition of small-sized three-dimensional lookup table, it can significantly improve color mapping accuracy and visual consistency.

[0142] To implement the above embodiments, this application also proposes an image processing apparatus. Figure 5 This is a schematic diagram of the structure of an image processing apparatus provided in an embodiment of this application.

[0143] like Figure 5 As shown, the image processing apparatus 500 includes: The first determining module 510 is used to determine the voxel index of each pixel in the three-dimensional lookup table and the relative coordinates of the pixel in the target voxel corresponding to the voxel index, based on the first color value of each pixel in the image to be processed. The acquisition module 520 is used to acquire the first color information of each vertex of the target voxel based on the voxel index; The second determination module 530 is used to determine the first weight of the vertex based on the relative coordinates using a weight generation model jointly trained with a 3D lookup table. The third determining module 540 is used to determine the second color information of the color-mapped pixel based on the first color information and the first weight. The generation module 550 is used to generate the target image based on the second color information of each pixel.

[0144] Optionally, the first determining module 510 is used for: Based on the first color value and the dimensions of the three-dimensional lookup table, determine the continuous voxel coordinates of the pixel in the three-dimensional lookup table; Round down the continuous voxel coordinates to obtain the voxel index; The relative coordinates are determined based on the difference between the continuous floating-point coordinates and the voxel index.

[0145] Optionally, the first determining module 510 is used for: Determine the index range of the three-dimensional lookup table based on the dimensions; The coordinates of the continuous voxels are determined based on the first color value and the upper limit of the index range.

[0146] Optionally, module 520 is used for: Determine the index position of the vertex in the 3D lookup table based on the voxel index; Based on the index position, retrieve the first color information of the vertex from the 3D lookup table.

[0147] Optionally, the third determining module 540 is used for: The first weight is normalized to obtain the second weight; The first color information is weighted according to the second weight of each vertex to obtain the second color information.

[0148] Optionally, the second color information is a color increment, generated by module 550, for: The second color value of a pixel is obtained by adding the first color value of the pixel to the corresponding color increment. The target image is obtained by cropping the second color value of each pixel.

[0149] Optionally, the generation module 550 is used for: Determine the range of color values ​​corresponding to the image to be processed; In response to the second color value of any pixel not being within the color value range, the second color value of that pixel is cropped to be within the color value range.

[0150] It should be noted that the explanation of the foregoing image processing method embodiments also applies to the image processing apparatus of this embodiment, and therefore will not be repeated here.

[0151] In this embodiment, by mapping the first color value of each pixel in the image to be processed to the target voxel in the three-dimensional lookup table, and using a weight generation model jointly trained with the three-dimensional lookup table, the interpolation weights of each vertex are dynamically generated based on the relative coordinates within the voxel, thereby achieving adaptive fusion of the color information of the vertices and adaptive fusion of different color gamut regions. This can reduce mapping errors under complex color distributions, reduce problems such as banding, color cast and detail loss, and improve the quality of the target image.

[0152] To implement the above embodiments, this application also proposes a training device. Figure 6 This is a schematic diagram of the structure of a training device provided in an embodiment of this application.

[0153] like Figure 6 As shown, the training device 600 includes: The first determining module 610 is used to determine the first voxel index of each pixel in the initial three-dimensional lookup table and the relative coordinates of the target voxel corresponding to the first voxel index based on the color value of each pixel in the sample image. The acquisition module 620 is used to acquire the first color information of each vertex of the target voxel based on the first voxel index; The second determining module 630 is used to generate a model using initial weights and determine the weights of vertices based on relative coordinates. The third determining module 640 is used to determine the second color information of the color-mapped pixel based on the first color information and weight. The generation module 650 is used to generate a color-mapped image based on the second color information of each pixel. The training module 660 is used to jointly train the initial 3D lookup table and the initial weight generation model based on the difference between the color-mapped image and the expected image corresponding to the sample image, so as to obtain the trained 3D lookup table and weight generation model.

[0154] Optionally, training module 660 is used for: The pixel reconstruction loss is determined based on the difference between the color-mapped image and the expected image; Based on the first voxel index, determine the second voxel index of the adjacent voxels of the target voxel; Based on the third color information corresponding to the first voxel index and the fourth color information corresponding to the second voxel index in the 3D lookup table, determine the smoothing regularization value of the lookup table; Based on pixel reconstruction loss and lookup table smoothing regularization value, the initial 3D lookup table and the initial weight generation model are jointly trained.

[0155] Optionally, training module 660 is used for: The pixel reconstruction loss is determined based on the difference between the color-mapped image and the expected image; The entropy of the weights of each vertex is calculated to obtain the weight distribution regularization value; Based on pixel reconstruction loss and weight distribution regularization, the initial 3D lookup table and the initial weight generation model are jointly trained.

[0156] It should be noted that the explanation of the aforementioned training method embodiment also applies to the training device of this embodiment, so it will not be repeated here.

[0157] In this embodiment, by jointly training the initial 3D lookup table and the initial weight generation model based on the difference between the color-mapped image and the expected image corresponding to the sample image, it is possible not only to accurately fit the complex nonlinear color mapping relationship, but also to avoid abrupt output changes in adjacent color regions caused by overfitting the 3D lookup table parameters to the training samples. This significantly suppresses visual defects such as banding, artifacts, and noise amplification, improves the accuracy of color mapping, and achieves higher color reconstruction accuracy and visual consistency under low-resolution 3D lookup tables.

[0158] According to embodiments of this application, this application also provides an electronic device, a readable storage medium, and a computer program product.

[0159] Figure 7 A schematic block diagram of an example electronic device 700 that can be used to implement embodiments of this application is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the application described and / or claimed herein.

[0160] like Figure 7As shown, device 700 includes a computing unit 701, which can perform various appropriate actions and processes based on a computer program stored in ROM (Read-Only Memory) 702 or a computer program loaded from storage unit 708 into RAM (Random Access Memory) 703. RAM 703 can also store various programs and data required for the operation of device 700. The computing unit 701, ROM 702, and RAM 703 are interconnected via bus 704. I / O (Input / Output) interface 705 is also connected to bus 704.

[0161] Multiple components in device 700 are connected to I / O interface 705, including: input unit 706, such as keyboard, mouse, etc.; output unit 707, such as various types of monitors, speakers, etc.; storage unit 708, such as disk, optical disk, etc.; and communication unit 709, such as network card, modem, wireless transceiver, etc. Communication unit 709 allows device 700 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0162] The computing unit 701 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, CPUs (Central Processing Units), GPUs (Graphics Processing Units), various special-purpose AI (Artificial Intelligence) computing chips, various computing units running machine learning model algorithms, DSPs (Digital Signal Processors), and any suitable processor, controller, microcontroller, etc. The computing unit 701 performs the various methods and processes described above, such as image processing methods. For example, in some embodiments, the image processing method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 708. In some embodiments, part or all of the computer program may be loaded and / or installed on device 700 via ROM 702 and / or communication unit 709. When the computer program is loaded into RAM 703 and executed by the computing unit 701, one or more steps of the image processing methods described above may be performed. Alternatively, in other embodiments, the computing unit 701 may be configured to perform image processing methods by any other suitable means (e.g., by means of firmware).

[0163] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, FPGAs (Field Programmable Gate Arrays), ASICs (Application-Specific Integrated Circuits), ASSPs (Application-Specific Standard Products), SOCs (System-on-Chips), CPLDs (Complex Programmable Logic Devices), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0164] The program code used to implement the methods of this application may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing device, such that when executed by the processor or controller, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0165] In the context of this application, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, RAM, ROM, EPROM (Electrically Programmable Read-Only Memory) or flash memory, optical fiber, CD-ROM (Compact Disc Read-Only Memory), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0166] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (Cathode-Ray Tube) or LCD (Liquid Crystal Display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0167] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include LANs (Local Area Networks), WANs (Wide Area Networks), the Internet, and blockchain networks.

[0168] Computer systems can include clients and servers. Clients and servers are generally geographically separated and typically interact via communication networks. The client-server relationship is established by computer programs running on the respective computers and having a client-server relationship with each other. A server can be a cloud server, also known as a cloud computing server or cloud host, a hosting product within the cloud computing service ecosystem, addressing the shortcomings of traditional physical hosts and VPS (Virtual Private Server) services, such as high management difficulty and weak business scalability. Servers can also be servers for distributed systems or servers incorporating blockchain technology.

[0169] It should be noted that the structure of the electronic device used to implement the training method of the embodiments of this application is similar to the structure of the electronic device described above, so it will not be described again here.

[0170] According to embodiments of this application, this application also provides a computer program product, which, when executed by an instruction processor, performs the image processing method or training method proposed in the above embodiments of this application.

[0171] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this application can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this application can be achieved, and this is not limited herein.

[0172] The specific embodiments described above do not constitute a limitation on the scope of protection of this application. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. An image processing method, comprising: For each pixel in the image to be processed, the voxel index of the pixel in the three-dimensional lookup table and the relative coordinates of the pixel in the target voxel corresponding to the voxel index are determined according to the first color value of the pixel. Based on the voxel index, obtain the first color information of each vertex of the target voxel; A weight generation model jointly trained with the three-dimensional lookup table is used to determine the first weight of the vertex based on the relative coordinates; Based on the first color information and the first weight, determine the second color information of the pixel after color mapping; The target image is generated based on the second color information of each pixel.

2. The method as described in claim 1, wherein, The step of determining the voxel index of the pixel in the three-dimensional lookup table and its relative coordinates within the target voxel corresponding to the voxel index based on the first color value of the pixel includes: Based on the first color value and the dimension of the three-dimensional lookup table, determine the continuous voxel coordinates of the pixel in the three-dimensional lookup table; The voxel index is obtained by rounding down the continuous voxel coordinates. The relative coordinates are determined based on the difference between the continuous floating-point coordinates and the voxel index.

3. The method as described in claim 2, wherein, Determining the continuous voxel coordinates of the pixel in the three-dimensional lookup table based on the first color value and the dimensions of the three-dimensional lookup table includes: Based on the dimensions, determine the index range of the three-dimensional lookup table; The coordinates of the continuous voxels are determined based on the first color value and the upper limit of the index range.

4. The method of claim 1, wherein, The step of obtaining the first color information of each vertex of the target voxel based on the voxel index includes: The index position of the vertex in the 3D lookup table is determined based on the voxel index. Based on the index position, the first color information of the vertex is obtained from the three-dimensional lookup table.

5. The method of claim 1, wherein, Determining the second color information of the pixel after color mapping based on the first color information and the first weight includes: The first weight is normalized to obtain the second weight; The first color information is weighted according to the second weight of each vertex to obtain the second color information.

6. The method of claim 1, wherein, The second color information is a color increment. Generating the target image based on the second color information of each pixel includes: The second color value of the pixel is obtained by adding the first color value of the pixel to the color increment corresponding to the pixel. The target image is obtained by cropping the second color value of each pixel.

7. The method of claim 6, wherein, The cropping process for the second color value of each pixel includes: Determine the range of color values ​​corresponding to the image to be processed; In response to the fact that the second color value of any pixel is not within the range of color values, the second color value of any pixel is cropped to the range of color values.

8. A training method, comprising: Based on the color value of each pixel in the sample image, determine the first voxel index of the pixel in the initial three-dimensional lookup table and its relative coordinates in the target voxel corresponding to the first voxel index; Based on the first voxel index, obtain the first color information of each vertex of the target voxel; An initial weight generation model is used to determine the weight of the vertex based on the relative coordinates; Based on the first color information and the weight, the second color information of the color-mapped pixel is determined, and the color-mapped image is generated based on the second color information of each pixel. Based on the difference between the color-mapped image and the expected image corresponding to the sample image, the initial 3D lookup table and the initial weight generation model are jointly trained to obtain a trained 3D lookup table and weight generation model.

9. The method of claim 8, wherein, The step of jointly training the initial 3D lookup table and the initial weight generation model based on the difference between the color-mapped image and the expected image corresponding to the sample image includes: The pixel reconstruction loss is determined based on the difference between the color-mapped image and the expected image; Based on the first voxel index, determine the second voxel index of the adjacent voxels of the target voxel; Based on the third color information corresponding to the first voxel index and the fourth color information corresponding to the second voxel index in the three-dimensional lookup table, determine the smoothing regularization value of the lookup table; The initial 3D lookup table and the initial weight generation model are jointly trained based on the pixel reconstruction loss and the lookup table smoothing regularization value.

10. The method of claim 8, wherein, The step of jointly training the initial 3D lookup table and the initial weight generation model based on the difference between the color-mapped image and the expected image corresponding to the sample image includes: The pixel reconstruction loss is determined based on the difference between the color-mapped image and the expected image; Entropy is calculated for the weights of each vertex to obtain the weight distribution regularization value; The initial 3D lookup table and the initial weight generation model are jointly trained based on the pixel reconstruction loss and the weight distribution regularization value.

11. An image processing apparatus, comprising: The first determining module is used to determine, for each pixel in the image to be processed, the voxel index of the pixel in the three-dimensional lookup table and the relative coordinates of the pixel in the target voxel corresponding to the voxel index, based on the first color value of the pixel. The acquisition module is used to acquire the first color information of each vertex of the target voxel based on the voxel index; The second determining module is used to determine the first weight of the vertex based on the relative coordinates using a weight generation model jointly trained with the three-dimensional lookup table. The third determining module is used to determine the second color information of the pixel after color mapping based on the first color information and the first weight. The generation module is used to generate a target image based on the second color information of each pixel.

12. A training device, comprising: The first determining module is used to determine the first voxel index of the pixel in the initial three-dimensional lookup table and the relative coordinates of the pixel in the target voxel corresponding to the first voxel index, based on the color value of each pixel in the sample image. The acquisition module is used to acquire the first color information of each vertex of the target voxel based on the first voxel index; The second determining module is used to generate a model using initial weights and determine the weights of the vertices based on the relative coordinates. The third determining module is used to determine the second color information of the pixel after color mapping based on the first color information and the weight. The generation module is used to generate a color-mapped image based on the second color information of each pixel. The training module is used to jointly train the initial 3D lookup table and the initial weight generation model based on the difference between the color-mapped image and the expected image corresponding to the sample image, so as to obtain the trained 3D lookup table and weight generation model.

13. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-10.

14. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-10.

15. A computer program product comprising a computer program that, when executed by a processor, implements the steps of the method according to any one of claims 1-10.

Images