Three-dimensional lookup table reconstruction methods, devices and electronic equipment

Abstract

This application discloses a method, apparatus, and electronic device for reconstructing a three-dimensional lookup table, belonging to the field of image processing technology. The method includes: acquiring a base image dataset and a target image dataset; wherein the matching degree of image content between any base image in the base image dataset and the corresponding target image in the target image dataset is greater than a preset threshold; performing three-dimensional point cloud registration on the base image dataset and the target image dataset to obtain a mapping relationship between the pixels of the base images in the base image dataset and the pixels of the target images in the target image dataset; and reconstructing an initial three-dimensional lookup table based on the mapping relationship to obtain a target three-dimensional lookup table.

Description

Technical Field

[0001] This application belongs to the field of image processing technology, specifically relating to a three-dimensional lookup table reconstruction method, apparatus, and electronic device. Background Technology

[0002] A 3D look-up table (3Dlut) is a tool used for precise control of color mapping, widely applied in film and image processing. Its principle is relatively simple: input a color, and 3Dlut can look up and map it to another color, achieving very high precision in color mapping control.

[0003] Currently, the method for creating 3Dlut involves manually adjusting various color space curves such as RGB, Lab, and HSV color models, or selecting a certain range of pixels for adjustment. After the original image achieves the desired color effect of the target image under the subjective judgment standard of the human eye, a 3Dlut is generated using specific software tools.

[0004] The 3Dlut generated by the above method has low accuracy and is affected by the subjective color judgment results of different people. Often, the 3Dluts adjusted by different people have large differences, and the process is very time-consuming and labor-intensive. Summary of the Invention

[0005] The purpose of this application is to provide a method, apparatus, and electronic device for reconstructing a three-dimensional lookup table. This addresses the problem in existing technologies where the color lookup table is subject to subjective human influence, resulting in significant differences and low accuracy.

[0006] In a first aspect, embodiments of this application provide a method for reconstructing a three-dimensional lookup table, the method comprising:

[0007] Obtain a basic image dataset and a target image dataset; wherein, the matching degree of the image content of any basic image in the basic image dataset with the corresponding target image in the target image dataset is greater than a preset threshold;

[0008] Perform 3D point cloud registration on the base image dataset and the target image dataset to obtain the mapping relationship between the pixels of the base image in the base image dataset and the pixels of the target image in the target image dataset;

[0009] The initial three-dimensional lookup table is reconstructed based on the mapping relationship to obtain the target three-dimensional lookup table.

[0010] Secondly, embodiments of this application provide a three-dimensional lookup table reconstruction apparatus, the apparatus comprising:

[0011] The first acquisition module is used to acquire a basic image dataset and a target image dataset; wherein, the matching degree of the image content of any basic image in the basic image dataset and the corresponding target image in the target image dataset is greater than a preset threshold.

[0012] The first determining module is used to perform three-dimensional point cloud registration on the basic image dataset and the target image dataset to obtain the mapping relationship between the pixels of the basic image in the basic image dataset and the pixels of the target image in the target image dataset.

[0013] The second determining module is used to reconstruct the initial three-dimensional lookup table based on the mapping relationship to obtain the target three-dimensional lookup table.

[0014] Thirdly, embodiments of this application provide an electronic device including a processor and a memory, the memory storing programs or instructions executable on the processor, the programs or instructions, when executed by the processor, implementing the steps of the method described in the first aspect.

[0015] Fourthly, embodiments of this application provide a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the method described in the first aspect.

[0016] Fifthly, embodiments of this application provide a chip, the chip including a processor and a communication interface, the communication interface being coupled to the processor, the processor being used to run programs or instructions to implement the method as described in the first aspect.

[0017] In a sixth aspect, embodiments of this application provide a computer program product stored in a storage medium, which is executed by at least one processor to implement the method described in the first aspect.

[0018] In this embodiment, by performing 3D point cloud registration on the acquired base image dataset and target image dataset, the mapping relationship between the pixels of the base image in the base image dataset and the pixels of the target image in the target image dataset is obtained. Based on this mapping relationship, the initial 3D lookup table is reconstructed to obtain the target 3D lookup table. This method uses paired base and target image datasets for learning and training, eliminating the need to collect special datasets and perform complex annotation work, thus improving the efficiency of determining the target 3D lookup table. Furthermore, no model related to the image content is generated during the process, improving the versatility and ease of use of the target 3D lookup table. In addition, the method of this embodiment can automatically learn the target 3D lookup table, eliminating the influence of subjective color judgment on the color conversion results, ensuring the consistency and accuracy of the color conversion results, and saving labor costs. Attached Figure Description

[0019] Figure 1 This is one of the flowcharts illustrating a three-dimensional lookup table reconstruction method in an exemplary embodiment;

[0020] Figure 2 This is a schematic diagram illustrating the segmentation of a base image and a target image into blocks, as shown in an exemplary embodiment.

[0021] Figure 3 This is an exemplary embodiment illustrating the determination of relevant key points of a pixel in a three-dimensional lookup table;

[0022] Figure 4 This is an exemplary embodiment illustrating the application of a target three-dimensional lookup table;

[0023] Figure 5 This is a second schematic flowchart illustrating a three-dimensional lookup table reconstruction method in an exemplary embodiment;

[0024] Figure 6 This is a schematic diagram of the structure of a three-dimensional lookup table reconstruction device shown in an exemplary embodiment;

[0025] Figure 7 This is a schematic diagram illustrating the structure of an electronic device in an exemplary embodiment;

[0026] Figure 8 This is a schematic diagram of the hardware structure of an electronic device shown in an embodiment of this application. Detailed Implementation

[0027] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0028] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0029] As described in the background section, current technologies suffer from low accuracy and significant color lookup table variations due to human subjectivity when generating 3D lookup tables. To address these issues, this application provides a 3D lookup table reconstruction method, apparatus, and electronic device. By performing 3D point cloud registration on the acquired base image dataset and target image dataset, a mapping relationship is obtained between the pixels of the base image in the base image dataset and the pixels of the target image in the target image dataset. Based on this mapping relationship, the initial 3D lookup table is reconstructed to obtain the target 3D lookup table. This method uses paired base and target image datasets for learning and training, eliminating the need for collecting special datasets and performing complex annotation work, thus improving the efficiency of determining the target 3D lookup table. Furthermore, the process does not generate models related to image content, improving the universality and ease of use of the target 3D lookup table. In addition, the method of this application can automatically learn the target 3D lookup table, eliminating the influence of subjective color judgment on the color conversion results, ensuring the consistency and accuracy of the color conversion results, and saving labor costs.

[0030] The following detailed description, in conjunction with the accompanying drawings, of the three-dimensional lookup table determination provided in the embodiments of this application through specific implementations and application scenarios, will be provided in detail.

[0031] Figure 1 This is a flowchart illustrating a three-dimensional lookup table reconstruction method provided in an embodiment of this application. The execution entity of this three-dimensional lookup table reconstruction method can be a server. It should be noted that the aforementioned execution entity does not constitute a limitation on the embodiments of this application.

[0032] like Figure 1 As shown, the three-dimensional lookup table reconstruction method provided in this application embodiment may include steps 110-130.

[0033] Step 110: Obtain the basic image dataset and the target image dataset.

[0034] The base image dataset can be a dataset containing multiple base images. This dataset can be an image dataset input into 3Dlut.

[0035] The target image dataset can be a dataset containing multiple target images. This dataset can be an image dataset obtained by processing a base image dataset using 3Dlut.

[0036] In some embodiments of this application, each base image in the base image dataset corresponds to one target image in the target image dataset. Furthermore, the matching degree of image content between any base image in the base image dataset and its corresponding target image in the target image dataset is greater than a preset threshold.

[0037] The preset threshold here can be a threshold for the matching degree of image content between any base image in the base image dataset and the corresponding target image in the target image dataset, for example, it can be 50%. That is, the image content of any base image in the base image dataset and the corresponding target image in the target image dataset must be more than 50% consistent, and there must be a difference in color.

[0038] Step 120: Perform 3D point cloud registration on the base image dataset and the target image dataset to obtain the mapping relationship between the pixels of the base image in the base image dataset and the pixels of the target image in the target image dataset.

[0039] In some embodiments of this application, in order to accurately obtain the mapping relationship between the pixels of the base image in the base image dataset and the pixels of the target image in the target image dataset, step 120 may specifically include:

[0040] The basic image in the basic image dataset and the corresponding target image in the target image dataset are divided into blocks.

[0041] The base image and the target image after block processing are paired to obtain P pairs of image region blocks;

[0042] Density clustering and cluster pairing are performed on each pair of image regions;

[0043] Three-dimensional point cloud registration is performed based on paired clusters to obtain rotation and translation matrices.

[0044] Where P is a positive integer.

[0045] Image region pairs can be obtained by pairing the base image and the target image after block processing.

[0046] In one example, refer to Figure 2 , Figure 2 This is a schematic diagram illustrating the segmentation of any base image in the base image dataset. The base image is segmented into M base image region blocks (i.e., Figure 2 Region 1, Region 2, ..., Region m).

[0047] Correspondingly, the segmentation of the target image in the target image dataset can refer to the segmentation method of the base image, which will not be elaborated here.

[0048] In some embodiments of this application, the base image and target image after block processing can be paired, that is, each base image region block corresponds to one target image region block. Then, density clustering processing can be performed on each image region block pair to obtain multiple clusters. Then, the clusters corresponding to each image region block are paired, and three-dimensional point cloud registration is performed on each paired cluster to obtain a rotation and translation matrix.

[0049] In the embodiments of this application, by dividing any base image in the base image dataset and the corresponding target image in the target image dataset into blocks, and pairing the divided base image and target image to obtain P pairs of image region blocks, density clustering and cluster pairing are performed on each pair of image region blocks, and three-dimensional point cloud registration is performed based on the paired clusters to obtain the rotation and translation matrix. In this way, the mapping relationship between the pixels of the base image in the base image dataset and the pixels of the target image in the target image dataset can be accurately obtained.

[0050] In some embodiments of this application, in order to further obtain the mapping relationship between the pixels of the base image in the base image dataset and the pixels of the target image in the target image dataset, the density clustering processing and cluster pairing of each image region block pair may specifically include:

[0051] Spatial mapping is performed on each pixel of the base image region in each image region block pair to obtain the base 3D pixel cloud, and density clustering is performed on the base 3D pixel cloud to obtain K first-class clusters;

[0052] Spatial mapping is performed on each pixel of the target image region block corresponding to the base image region block to obtain the target three-dimensional pixel cloud, and density clustering is performed on the target three-dimensional pixel cloud to obtain K second-class clusters;

[0053] Pair the first and second clusters based on the centroids of the K first-class clusters and the K second-class clusters.

[0054] Where K is a positive integer.

[0055] A basic 3D pixel point cloud can be obtained by spatially mapping each pixel of a basic image region block in each image region block pair.

[0056] The target 3D pixel point cloud can be the point cloud obtained by spatially mapping each pixel of the target image region block corresponding to the base image region block.

[0057] The first type of cluster can be the cluster formed by density clustering of the basic three-dimensional pixel point cloud corresponding to the basic image region block.

[0058] The second type of cluster can be the cluster formed by density clustering of the target three-dimensional pixel point cloud corresponding to the target image region block.

[0059] In some embodiments of this application, spatial mapping is performed on each pixel of the base image region block in each image region block pair to obtain a base three-dimensional pixel cloud, and spatial mapping is performed on each pixel of the target image region block corresponding to the base image region block to obtain a target three-dimensional pixel cloud. This can be achieved by mapping each pixel in the base image region block and the target image region block to a value between 0 and 1 in the RGB space, which is the obtained base three-dimensional pixel cloud and target three-dimensional pixel cloud.

[0060] In one example, taking the base image region block in each image region block as an example, each pixel in the base image region block is mapped in RGB space to a value between 0 and 1. Specifically, this can be achieved as follows: For example, the pixel value range of an 8-bit RGB image (base image region block) is 0-255. Dividing each pixel of the image by the maximum value minus the minimum value will map the pixels of the base image region block to a value between 0 and 1. For example, if a pixel value is 88, then the mapping is 88 / (255-0).

[0061] In one example, the DBSCAN density clustering algorithm is used to perform density clustering on the base image region blocks in each image region block pair, resulting in, for example, 5 first-class clusters. Similarly, the DBSCAN density clustering algorithm is used to perform density clustering on the target image region blocks in each image region block pair, also resulting in 5 second-class clusters.

[0062] Calculate the centroid positions of the five first-class clusters, for example, s1, s2, s3, s4, and s5. Calculate the centroid positions of the five second-class clusters, for example, t1, t2, t3, t4, and t5. For any one of the five first-class clusters, let s1 be the centroid. If the centroid of the five second-class clusters closest to s1 is t3, then the first-class cluster represented by s1 is matched with the second-class cluster represented by t3. Repeat this process until five pairs of pairings are found.

[0063] In the embodiments of this application, a basic three-dimensional pixel cloud is obtained by spatially mapping each pixel of the basic image region block in each image region block pair. The basic three-dimensional pixel cloud is then density-clustered to obtain K first clusters. The target three-dimensional pixel cloud is obtained by spatially mapping each pixel of the target image region block corresponding to the basic image region block. The target three-dimensional pixel cloud is then density-clustered to obtain K second clusters. Then, the first clusters and second clusters are paired according to the centroids of the K first clusters and the K second clusters. In this way, density clustering and cluster pairing can be performed on each image region block pair accurately, thereby accurately obtaining the mapping relationship between the pixels of the basic image in the basic image dataset and the pixels of the target image in the target image dataset.

[0064] In some embodiments of this application, the 3D point cloud registration based on paired clusters to obtain the rotation and translation matrix can specifically be:

[0065] For each paired cluster, point cloud registration is performed on the pixel feature points of the first cluster and the pixel feature points of the second cluster to obtain the rotation and translation matrix between the first cluster and the second cluster.

[0066] In some embodiments of this application, for each paired cluster, the Iterative Closest Point (ICP) point cloud registration method can be used to register the point clouds of each paired cluster, obtaining the rotation and translation matrix between each paired cluster. For example, if the input is two sets of 3D points, denoted as source_p1 and target_p1 respectively, the algorithm treats each set of 3D points as a rigid body, following a standard procedure. The algorithm will ultimately obtain a matrix Rt representing the rotation and translation relationship between the two sets of rigid bodies. Multiplying this matrix Rt by the 3D points in source_p1 yields the optimal corresponding matching point position in target_p1.

[0067] The specific use of the ICP point cloud registration method for point cloud registration is existing technology and will not be elaborated here.

[0068] In some embodiments of this application, before performing point cloud registration using the ICP point cloud registration method, the ransac algorithm can be used to remove outliers to increase the accuracy of point cloud registration.

[0069] In some embodiments of this application, the number of the first and second clusters obtained above may be different. Therefore, one-to-one matching is not possible during matching. To further refine the matching of each image region block pair, before obtaining K first and K second clusters, the aforementioned three-dimensional lookup table reconstruction method may further include:

[0070] The cluster corresponding to the larger value between the first quantity of the first type of cluster and the second quantity of the second type of cluster is taken as the first target cluster.

[0071] Merge the two clusters with the closest centroid distance among the first target clusters until the number of merged clusters is equal to the number of the second target clusters.

[0072] The first target cluster can be the cluster corresponding to the larger value between the first quantity of the first cluster and the second quantity of the second cluster.

[0073] The second target type is the cluster corresponding to the smaller value between the first quantity of the first type of cluster and the second quantity of the second type of cluster.

[0074] In one example, if there are 3 first-class clusters and 7 second-class clusters, then the second-class cluster is the first target cluster, and the first-class cluster is the second target cluster.

[0075] Continuing with the example above, if the number of first-class clusters is 3 and the number of second-class clusters is 7, and the centroid positions in the second-class clusters can be t1, t2, t3, t4, t5, t6, and t7, then the two clusters with the closest centroids among these 7 centroids are merged into 6 clusters. Then, the two clusters with the closest centroids among the merged 6 clusters are merged into 5 clusters. Then, the two clusters with the closest centroids among the 5 clusters are merged, and so on, until the number of clusters is the same as the number of the second target clusters.

[0076] Then, based on the centroids of the first and second types of clusters, which are equal in number, the first and second types of clusters are paired.

[0077] In the embodiments of this application, when the number of the first cluster and the second cluster are not equal, the cluster corresponding to the larger value of the first number of the first cluster and the second number of the second cluster is taken as the first target cluster. The two clusters with the closest centroid distance among the clusters of the first target cluster are merged until the number of the merged clusters is equal to the number of the second target clusters, so as to obtain at least one mutually matching cluster. In this way, a precisely matching cluster can be obtained.

[0078] In some embodiments of this application, in order to obtain an accurate target 3D lookup table, before performing block processing on any base image in the base image dataset and the corresponding target image in the target image dataset, the aforementioned 3D lookup table reconstruction method may further include:

[0079] Extract at least four first key points from the base image in the base image dataset, and at least four second key points from the corresponding target image in the target image dataset;

[0080] Determine the homography matrix that converts at least four first keypoints into at least four second keypoints;

[0081] Alignment processing is performed on the base image and the target image based on the homography matrix.

[0082] The first key point can be a feature point in the base image, and the second key point can be a feature point in the target image.

[0083] In some embodiments of this application, at least four first key points of the base image and at least four second key points of the target image can be extracted by a feature point extraction algorithm. For example, these can be scale-invariant feature transform (SIFT) key points. Then, a homography matrix (H matrix) is calculated to convert the at least four first key points into at least four second key points. The base image and the target image are aligned according to the homography matrix to obtain the aligned base image and target image.

[0084] In some embodiments of this application, the alignment process may be performed multiple times to reduce the impact of large differences in image content between paired images on the alignment result.

[0085] It should be noted that when the alignment process is performed multiple times, the H matrix for each alignment operation is different. The H matrix is ​​related to the amount of image correction between the base image and the target image (the amount of correction refers to how many pixels the base image needs to be rotated, translated, scaled, or have perspective changed to align with the target image). Therefore, the H matrix may be relatively large the first time, and the H matrix will be relatively small the second time when the result of the first alignment is used. Subsequent alignment operations are used to improve the alignment accuracy.

[0086] In the embodiments of this application, by extracting at least four first key points from the base image in the base image dataset and at least four second key points from the corresponding target image in the target image dataset, a homography matrix is ​​determined to convert the at least four first key points into at least four second key points. Then, based on the homography matrix, the base image and the target image are aligned to obtain the aligned base image and target image. This allows for precise alignment of the base image and the target image, thereby obtaining a precise target 3D lookup table between the base image dataset and the target image dataset.

[0087] Step 130: Reconstruct the initial 3D lookup table based on the mapping relationship to obtain the target 3D lookup table.

[0088] The initial 3D lookup table can be a user-defined 3D lookup table or a 3D lookup table generated in previous iterations. The target 3D lookup table can be the final 3D lookup table.

[0089] In some embodiments of this application, in order to accurately obtain the target three-dimensional lookup table, step 130 may specifically include:

[0090] Obtain the initial learning rate;

[0091] Based on the rotation and translation matrix and the initial learning rate, the initial 3D lookup table is reconstructed to obtain the target 3D lookup table.

[0092] The initial learning rate can be the learning rate initially set by the user. This initial learning rate can be set by the user according to their needs, and there is no limitation here. For example, it can be 0.05.

[0093] In the embodiments of this application, the initial three-dimensional lookup table can be reconstructed using a rotation and translation matrix and an initial learning rate to obtain a target three-dimensional lookup table. This results in an accurate target three-dimensional lookup table, which eliminates the influence of subjective color judgment on the color conversion results, ensuring the consistency and accuracy of the color conversion results and saving labor costs.

[0094] In some embodiments of this application, to further refine the target 3D lookup table, the reconstruction of the initial 3D lookup table based on the rotation and translation matrix and the initial learning rate to obtain the target 3D lookup table may specifically include:

[0095] For each matching cluster, the following steps are executed iteratively to obtain the target 3D lookup table:

[0096] Based on the rotation and translation matrix and the initial learning rate, the color correspondence between the base image region blocks and the target image region blocks is learned according to the initial 3D lookup table.

[0097] If the preset learning stopping condition is not met, update the initial 3D lookup table and the initial learning rate, and return to execute the color correspondence between the base image region block and the target image region block based on the rotation and translation matrix and the initial learning rate, until the preset learning stopping condition is met, and obtain the target 3D lookup table.

[0098] The preset learning stop condition can be a pre-set learning stop condition, such as the number of iterations reaching a certain threshold, or the learning rate reaching a certain value, etc. There are no restrictions here.

[0099] In one example, for each pair of matching clusters in each matching region block, the pixels in the corresponding cluster of the base image are traversed, and the relevant keypoints corresponding to that pixel are found in the initial 3D lookup table, such as... Figure 3 As shown, Figure 3The entire cube in the image is equivalent to an initial three-dimensional lookup table. A certain pixel P1 in a certain cluster of the base image has 8 related key points: m1, m2, m3, m4, m5, m6, m7 and m8. Each pixel has 8 related key points. Then, the duplicate related key points in the cluster are removed, and the related key points after removing the duplicates are recorded as M1, M2, ..., Mb (b is a positive integer). Each related key point is homogenized into a 4*1 matrix, and the new position Pb of the related key point is as shown in the following formula (1).

[0100] It should be noted that since the three-dimensional lookup table describes the correspondence between the pixel values ​​of each pixel in the base image and the target image, each position corresponds to a pixel value. The change of pixel value leads to the change of color. Therefore, the change of the position of the pixel in formula (1) is equivalent to the change of color between the base image and the target image.

[0101] p b =α*Rt*Mb (1)

[0102] Where, p b Let Rt be the new position of the relevant keypoint, α be the initial learning rate, Rt be the rotation and translation matrix, and Mb be the initial position of the relevant keypoint.

[0103] After learning the color correspondence between the base image region block and the target image region block according to the above formula (1), if the learning stopping condition is not met, update the initial three-dimensional lookup table and the initial learning rate, and then iterate through each cluster of each matched region block until the learning stopping condition is met to obtain the target three-dimensional lookup table.

[0104] In some embodiments of this application, the three-dimensional lookup table updated each time is the three-dimensional lookup table obtained in the previous loop.

[0105] In some embodiments of this application, the initial learning rate can be updated based on the average centroid distances of the matched clusters in the current iteration. For example, if the centroids of each cluster in the region block corresponding to the cluster in the current iteration are s1, s2, s3, s4, and s5 (the first cluster corresponding to the base image), and t1, t2, t3, t4, and t5 (the second cluster corresponding to the target image), where s1 matches t3, s2 matches t1, s3 matches t2, s4 matches t4, and s5 matches t5, then the distance between the centroids of each cluster pair is calculated, and a preset multiple (e.g., 1 / 20 times) of the average distance between the centroids of each cluster pair can be used as the new learning rate.

[0106] It should be noted that the preset multiplier can be between (0-1), and the value is related to the learning rate. A larger value is more likely to cause oscillations and non-convergence in the learning process when the image difference is large. Empirically, the value is generally between 1 / 10 and 1 / 100.

[0107] In some embodiments of this application, the final 3Dlut table (i.e., the target 3D lookup table) is a lookup table. The application process involves inputting an image, finding the corresponding spatial location of each pixel value in a standard mapping table (this standard mapping table refers to an initialized mapping table, which can be artificially generated, such as a 33×33×33 3Dlut mapping table, where each dimension is a range of (0,1) divided into 33 equal parts). After the image pixels are mapped through this standard mapping table, the spatial location corresponding to each pixel value is found. Then, the value corresponding to the spatial location in the final 3Dlut table is mapped, and this value is the new value of the pixel. By iterating through each pixel, the mapped new image can be obtained, as shown in Figure 4. Figure 4 Only a portion of the three-dimensional lookup table is shown in the image.

[0108] In the embodiments of this application, for each pair of matching clusters, the following steps are executed iteratively to obtain a target 3D lookup table: based on the rotation and translation matrix and the initial learning rate, the color correspondence between the base image region block and the target image region block is learned according to the initial 3D lookup table. If it is determined that the preset learning stopping condition is not met, the initial 3D lookup table and the initial learning rate are updated, and the process is repeated to learn the color correspondence between the base image region block and the target image region block according to the initial 3D lookup table based on the rotation and translation matrix and the initial learning rate, until the preset learning stopping condition is met, thus obtaining a target 3D lookup table. In this way, an accurate target 3D lookup table can be obtained. This table can eliminate the influence of people's subjective color judgment on the color conversion result, ensure the consistency and accuracy of the color conversion result, and save labor costs.

[0109] In some embodiments of this application, to better understand the technical solutions of this application, the embodiments of this application also provide another possible implementation of the three-dimensional lookup table reconstruction method, such as... Figure 5 As shown, the three-dimensional lookup table reconstruction method provided in this application embodiment may include steps 501 to 508.

[0110] Step 501: Obtain the initial 3D lookup table and the initial learning rate.

[0111] This step is the same as obtaining the initial 3D lookup table and initial learning rate in the above embodiments, and will not be repeated here.

[0112] Step 502: Obtain the basic image dataset and the target image dataset.

[0113] This step is the same as step 110 in the above embodiment, and will not be repeated here.

[0114] Step 503: Align the base image in the base image dataset and the corresponding target image in the target image dataset to obtain the aligned base image and target image.

[0115] This step is the same as the alignment process in the above embodiments, and will not be repeated here.

[0116] Step 504: Perform block processing on the aligned base image and target image respectively.

[0117] This step is the same as the step of dividing the aligned base image and target image into blocks in the above embodiment, and will not be repeated here.

[0118] Step 505: Perform pairing processing on the base image and target image after block processing to obtain P pairs of image region blocks.

[0119] This step is consistent with the pairing process of the base image and target image after block processing in the above embodiment, resulting in P pairs of image region blocks, and will not be repeated here.

[0120] Step 506: Perform density clustering and cluster pairing on each image region block pair, and perform three-dimensional electric cloud registration based on the paired clusters to obtain the rotation and translation matrix.

[0121] This step is consistent with the density clustering and cluster pairing of each image region block pair in the above embodiment; the 3D point cloud registration is performed based on the paired clusters to obtain the same rotation and translation matrix, which will not be repeated here.

[0122] Step 507: Based on the rotation and translation matrix and the initial learning rate, reconstruct the initial 3D lookup table and update the initial 3D lookup table.

[0123] Step 508: Dynamically adjust the initial learning rate based on the mean centroid distance in the matching relationship.

[0124] Steps 507 and 508 are the same as the steps for updating the initial 3D lookup table and the initial learning rate in the above embodiments, and will not be repeated here.

[0125] This application proposes an automated method for learning and training 3Dlut tables using paired datasets. Unlike existing methods that rely on manual debugging to create 3Dlut tables, this method automatically learns and generates a set of 3Dlut tables. It utilizes only readily available datasets to acquire the 3Dlut tables, significantly reducing the difficulty of acquisition and the workload of preparation and debugging. It also eliminates the influence of subjective color judgments, ensuring consistency and accuracy. Unlike existing methods that train models to create 3Dlut tables, this invention, while using paired datasets for automatic learning and training, does not generate models related to image content. This improves the versatility and ease of use of the 3Dlut tables. To reduce the impact of local tonal and color adjustments in different scenes on the versatility of the 3Dlut tables, the dataset can be used to classify scenes, training multiple 3Dlut tables for use in different scenes.

[0126] This application's embodiments include two stages: coarse registration (i.e., alignment processing) and precise registration (ICP point cloud registration). Therefore, it has strong robustness to differences in the field of view of paired image data, reduces the requirements for the dataset, and expands its applicability. As long as the collected image data has a sufficiently wide color coverage, the method of this application can achieve color mapping within the color coverage range of the dataset, improving the use and preparation efficiency of the 3Dlut table.

[0127] The three-dimensional lookup table reconstruction method provided in this application can be executed by a three-dimensional lookup table reconstruction device. This application uses the method of determining a three-dimensional lookup table executed by the three-dimensional lookup table reconstruction device as an example to illustrate the three-dimensional lookup table reconstruction device provided in this application.

[0128] Figure 6 This is a schematic diagram of the structure of a three-dimensional lookup table reconstruction device according to an exemplary embodiment.

[0129] like Figure 6 As shown, the three-dimensional lookup table reconstruction device 600 can be applied to electronic devices, and the three-dimensional lookup table reconstruction device 600 may include:

[0130] The first acquisition module 610 is used to acquire a basic image dataset and a target image dataset; wherein, the matching degree of the image content of any basic image in the basic image dataset and the corresponding target image in the target image dataset is greater than a preset threshold.

[0131] The first determining module 620 is used to perform three-dimensional point cloud registration on the basic image dataset and the target image dataset to obtain the mapping relationship between the pixels of the basic image in the basic image dataset and the pixels of the target image in the target image dataset.

[0132] The second determining module 630 is used to reconstruct the initial three-dimensional lookup table based on the mapping relationship to obtain the target three-dimensional lookup table.

[0133] In the embodiments of this application, a first determining module performs 3D point cloud registration on the base image dataset and the target image dataset acquired by the acquisition module to obtain the mapping relationship between the pixels of the base image in the base image dataset and the pixels of the target image in the target image dataset. Then, based on the mapping relationship, the second determining module reconstructs the initial 3D lookup table to obtain the target 3D lookup table. This method uses paired base image datasets and target image datasets for learning and training, eliminating the need to collect special datasets and perform complex annotation work, thus improving the efficiency of determining the target 3D lookup table. Furthermore, the method does not generate a model related to the image content, improving the versatility and ease of use of the target 3D lookup table. In addition, the method of this application embodiment can automatically learn the target 3D lookup table, eliminating the influence of subjective color judgment on the color conversion results, ensuring the consistency and accuracy of the color conversion results, and saving labor costs.

[0134] In some embodiments of this application, in order to accurately obtain the mapping relationship between the pixels of the base image in the base image dataset and the pixels of the target image in the target image dataset, the first determining module 620 may specifically include:

[0135] The first processing unit is used to perform block processing on any basic image in the basic image dataset and the corresponding target image in the target image dataset;

[0136] The first determining unit is used to perform pairing processing on the base image and the target image after block processing to obtain P pairs of image region blocks; where P is a positive integer;

[0137] The second processing unit is used to perform density clustering and cluster pairing on each of the image region block pairs;

[0138] The second determining unit is used for 3D point cloud registration based on paired clusters to obtain rotation and translation matrices.

[0139] In some embodiments of this application, in order to further obtain a more accurate target three-dimensional lookup table, the first determining module 620 may further include:

[0140] An extraction unit is used to extract at least four first key points of the base image in the base image dataset and at least four second key points of the corresponding target image in the target image dataset.

[0141] The third determining unit is used to determine the homography matrix for converting the at least four first key points into the at least four second key points;

[0142] The third processing unit is used to perform alignment processing on the base image and the target image based on the homography matrix.

[0143] In some embodiments of this application, in order to further accurately obtain the mapping relationship between the pixels of the base image in the base image dataset and the pixels of the target image in the target image dataset, the second processing unit is specifically used for:

[0144] Spatial mapping is performed on each pixel of the base image region block in each image region block pair to obtain a base three-dimensional pixel cloud, and density clustering is performed on the base three-dimensional pixel cloud to obtain K first-class clusters.

[0145] Spatial mapping is performed on each pixel of the target image region block corresponding to the basic image region block to obtain the target three-dimensional pixel cloud, and density clustering is performed on the target three-dimensional pixel cloud to obtain K second-class clusters; where K is a positive integer;

[0146] The first clusters and the second clusters are paired based on the centroids of the K first clusters and the K second clusters.

[0147] In some embodiments of this application, in order to further obtain a more accurate target 3D lookup table, the second determining module 630 is specifically used for:

[0148] Obtain the initial learning rate;

[0149] Based on the rotation and translation matrix and the initial learning rate, the initial 3D lookup table is reconstructed to obtain the target 3D lookup table.

[0150] The three-dimensional lookup table reconstruction device in this application embodiment can be an electronic device or a component within an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices besides a terminal. For example, the electronic device can be a mobile phone, tablet computer, laptop computer, PDA, in-vehicle electronic device, mobile internet device (MID), augmented reality (AR) / virtual reality (VR) device, robot, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc. It can also be a server, network attached storage (NAS), personal computer (PC), television (TV), ATM, or self-service machine, etc. This application embodiment does not specifically limit the device.

[0151] The three-dimensional lookup table reconstruction device in this application embodiment can be a device with an operating system. This operating system can be Android, iOS, or other possible operating systems; this application embodiment does not specifically limit it.

[0152] The three-dimensional lookup table reconstruction apparatus provided in this application embodiment can realize all the processes implemented in the above method embodiments, and will not be described again here to avoid repetition.

[0153] Optionally, such as Figure 7 As shown, this application embodiment also provides an electronic device 700, including a processor 701 and a memory 702. The memory 702 stores a program or instructions that can run on the processor 701. When the program or instructions are executed by the processor 701, they implement the various steps of the above-described three-dimensional lookup table reconstruction method embodiment and can achieve the same technical effect. To avoid repetition, they will not be described again here.

[0154] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.

[0155] Figure 8 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of this application.

[0156] The electronic device 800 includes, but is not limited to, components such as: radio frequency unit 801, network module 802, audio output unit 803, input unit 804, sensor 805, display unit 806, user input unit 807, interface unit 808, memory 809, and processor 810.

[0157] Those skilled in the art will understand that the electronic device 800 may also include a power supply (such as a battery) for supplying power to various components. The power supply may be logically connected to the processor 810 through a power management system, thereby enabling functions such as managing charging, discharging, and power consumption through the power management system. Figure 8 The electronic device structure shown does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be elaborated here.

[0158] The processor 810 is used to acquire a basic image dataset and a target image dataset; wherein the matching degree of the image content of any basic image in the basic image dataset with the corresponding target image in the target image dataset is greater than a preset threshold.

[0159] The processor 810 is used to perform 3D point cloud registration on the base image dataset and the target image dataset to obtain the mapping relationship between the pixels of the base image in the base image dataset and the pixels of the target image in the target image dataset; and to reconstruct the initial 3D lookup table based on the mapping relationship to obtain the target 3D lookup table.

[0160] Thus, by performing 3D point cloud registration on the acquired base image dataset and target image dataset, the mapping relationship between the pixels of the base image in the base image dataset and the pixels of the target image in the target image dataset is obtained. Based on this mapping relationship, the initial 3D lookup table is reconstructed to obtain the target 3D lookup table. Using paired base and target image datasets for learning and training eliminates the need to collect special datasets and perform complex annotation work, improving the efficiency of determining the target 3D lookup table. Furthermore, the process does not generate a model related to the image content, enhancing the versatility and ease of use of the target 3D lookup table. In addition, the method of this application embodiment can automatically learn the target 3D lookup table, eliminating the influence of subjective color judgment on the color conversion results, ensuring the consistency and accuracy of the color conversion results, and saving labor costs.

[0161] Optionally, the processor 810 is further configured to perform block processing on any base image in the base image dataset and the corresponding target image in the target image dataset; perform pairing processing on the block-processed base image and the target image to obtain P pairs of image region blocks; where P is a positive integer; perform density clustering processing and cluster pairing on each pair of image region blocks; and perform three-dimensional point cloud registration based on the paired clusters to obtain a rotation and translation matrix.

[0162] Thus, by dividing any base image in the base image dataset and the corresponding target image in the target image dataset into blocks, and then pairing the divided base image and target image, P pairs of image region blocks are obtained. Density clustering and cluster pairing are performed on each pair of image region blocks, and 3D point cloud registration is performed based on the paired clusters to obtain the rotation and translation matrix. In this way, the mapping relationship between the pixels of the base image in the base image dataset and the pixels of the target image in the target image dataset can be accurately obtained.

[0163] Optionally, the processor 810 is further configured to extract at least four first key points of the base image in the base image dataset and at least four second key points of the corresponding target image in the target image dataset; determine to convert the at least four first key points into a homography matrix of the at least four second key points; and perform alignment processing on the base image and the target image based on the homography matrix.

[0164] Thus, by extracting at least four first key points from the base image in the base image dataset and at least four second key points from the corresponding target image in the target image dataset, a homography matrix is ​​determined to convert the at least four first key points into at least four second key points. Then, based on the homography matrix, the base image and the target image are aligned, thereby obtaining the aligned base image and target image. This allows for precise alignment of the base image and the target image, resulting in a precise target 3D lookup table between the base image dataset and the target image dataset.

[0165] Optionally, the processor 810 is further configured to perform spatial mapping on each pixel of the base image region block in each image region block pair to obtain a base three-dimensional pixel point cloud, and perform density clustering on the base three-dimensional pixel point cloud to obtain K first clusters; perform spatial mapping on each pixel of the target image region block corresponding to the base image region block to obtain a target three-dimensional pixel point cloud, and perform density clustering on the target three-dimensional pixel point cloud to obtain K second clusters; wherein K is a positive integer; and pair the first clusters and the second clusters according to the centroids of the K first clusters and the K second clusters.

[0166] Thus, by spatially mapping each pixel of the base image region in each image region pair, a basic 3D pixel cloud is obtained. Density clustering is then performed on the basic 3D pixel cloud to obtain K first-class clusters. Spatial mapping is then performed on each pixel of the target image region corresponding to the base image region to obtain the target 3D pixel cloud. Density clustering is then performed on the target 3D pixel cloud to obtain K second-class clusters. Then, based on the centroids of the K first-class clusters and the K second-class clusters, the first-class clusters and second-class clusters are paired. In this way, density clustering and cluster pairing can be performed accurately on each image region pair, thereby accurately obtaining the mapping relationship between the pixels of the base image in the base image dataset and the pixels of the target image in the target image dataset.

[0167] Optionally, the processor 810 is further configured to obtain an initial learning rate; and to reconstruct the initial 3D lookup table based on the rotation and translation matrix and the initial learning rate to obtain the target 3D lookup table.

[0168] Thus, by using the rotation and translation matrix and the initial learning rate, the initial 3D lookup table can be reconstructed to obtain the target 3D lookup table. This results in an accurate target 3D lookup table, which eliminates the influence of subjective color judgment on the color conversion results, ensuring the consistency and accuracy of the color conversion results and saving labor costs.

[0169] It should be understood that, in this embodiment, the input unit 804 may include a graphics processing unit (GPU) 8041 and a microphone 8042. The GPU 8041 processes image data of still images or videos obtained by an image capture device (such as a camera) in video capture mode or image capture mode. The display unit 806 may include a display panel 8061, which may be configured in the form of a liquid crystal display, an organic light-emitting diode, or the like. The user input unit 807 includes at least one of a touch panel 8071 and other input devices 8072. The touch panel 8071 is also called a touch screen. The touch panel 8071 may include a touch detection device and a touch controller. Other input devices 8072 may include, but are not limited to, physical keyboards, function keys (such as volume control buttons, power buttons, etc.), trackballs, mice, and joysticks, which will not be described in detail here.

[0170] The memory 809 can be used to store software programs and various data. The memory 809 may primarily include a first storage area for storing programs or instructions and a second storage area for storing data. The first storage area may store the operating system, application programs or instructions required for at least one function (such as sound playback, image playback, etc.). Furthermore, the memory 809 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus RAM (DRRAM). The memory 809 in the embodiments of this application includes, but is not limited to, these and any other suitable types of memory.

[0171] Processor 810 may include one or more processing units; optionally, processor 810 integrates an application processor and a modem processor, wherein the application processor mainly handles operations involving the operating system, user interface, and applications, and the modem processor mainly handles wireless communication signals, such as a baseband processor. It is understood that the aforementioned modem processor may also not be integrated into processor 810.

[0172] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described three-dimensional lookup table reconstruction method embodiments and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0173] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0174] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the various processes of the above-described three-dimensional lookup table reconstruction method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0175] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0176] This application provides a computer program product, which is stored in a storage medium and executed by at least one processor to implement the various processes of the three-dimensional lookup table reconstruction method embodiment described above, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0177] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0178] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0179] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A method of three-dimensional lookup table reconstruction, characterized by, The method comprises: obtaining a basic image dataset and a target image dataset; wherein the matching degree of any one basic image in the basic image dataset and the image content of the corresponding target image in the target image dataset is greater than a preset threshold, the basic image dataset is an image dataset input to an initial three-dimensional lookup table, and the target image dataset is an image dataset obtained after the basic image dataset passes through the initial three-dimensional lookup table; performing three-dimensional point cloud registration on the basic image dataset and the target image dataset to obtain a mapping relationship between the pixels of the basic image in the basic image dataset and the pixels of the target image in the target image dataset; reconstructing the initial three-dimensional lookup table based on the mapping relationship to obtain a target three-dimensional lookup table.

2. The method of claim 1, wherein, The three-dimensional point cloud registration on the basic image dataset and the target image dataset to obtain the mapping relationship between the pixels of the basic image in the basic image dataset and the pixels of the target image in the target image dataset comprises: performing block processing on any one basic image in the basic image dataset and the corresponding target image in the target image dataset; performing pairing processing on the basic image and the target image after the block processing to obtain P pairs of image region blocks; wherein P is a positive integer; performing density clustering processing and cluster pairing on each image region block pair; performing three-dimensional point cloud registration based on the paired clusters to obtain a rotation and translation matrix.

3. The method of claim 2, wherein, Before the block processing on any one basic image in the basic image dataset and the corresponding target image in the target image dataset, the method further comprises: extracting at least four first key points of the basic image in the basic image dataset and at least four second key points of the corresponding target image in the target image dataset; determining a homography matrix for converting the at least four first key points into the at least four second key points; performing alignment processing on the basic image and the target image based on the homography matrix.

4. The method of claim 2, wherein, The density clustering processing and cluster pairing on each image region block pair comprise: performing spatial mapping on each pixel point in the basic image region block in each image region block pair to obtain a basic three-dimensional pixel point cloud, and performing density clustering on the basic three-dimensional pixel point cloud to obtain K first clusters; performing spatial mapping on each pixel point in the target image region block corresponding to the basic image region block to obtain a target three-dimensional pixel point cloud, and performing density clustering on the target three-dimensional pixel point cloud to obtain K second clusters; wherein K is a positive integer; pairing the first clusters and the second clusters according to the centroids of the K first clusters and the centroids of the K second clusters.

5. The method of claim 2, wherein, The reconstruction of the initial three-dimensional lookup table based on the mapping relationship to obtain the target three-dimensional lookup table comprises: obtaining an initial learning rate; reconstructing the initial three-dimensional lookup table based on the rotation and translation matrix and the initial learning rate to obtain the target three-dimensional lookup table.

6. A three-dimensional lookup table reconstruction apparatus characterized by comprising: The device comprises: The first acquisition module is configured to acquire a basic image dataset and a target image dataset; wherein a matching degree of any one basic image in the basic image dataset and image content of a corresponding target image in the target image dataset is greater than a preset threshold, the basic image dataset is an image dataset input to an initial three-dimensional lookup table, and the target image dataset is an image dataset obtained after the basic image dataset passes through the initial three-dimensional lookup table; The first determination module is configured to perform three-dimensional point cloud registration on the basic image dataset and the target image dataset to obtain a mapping relationship between pixels of a basic image in the basic image dataset and pixels of a target image in the target image dataset; The second determination module is configured to reconstruct the initial three-dimensional lookup table based on the mapping relationship to obtain a target three-dimensional lookup table.

7. The apparatus of claim 6, wherein, The first determination module includes: The first processing unit is configured to perform block processing on any one basic image in the basic image dataset and a corresponding target image in the target image dataset; The first determination unit is configured to perform pairing processing on the basic image and the target image after the block processing to obtain P pairs of image region blocks; wherein P is a positive integer; The second processing unit is configured to perform density clustering processing and cluster pairing on each image region block pair; The second determination unit is configured to perform three-dimensional point cloud registration based on the paired clusters to obtain a rotation and translation matrix.

8. The apparatus of claim 7, wherein, The first determination module further includes: The extraction unit is configured to extract at least four first key points of a basic image in the basic image dataset and at least four second key points of a corresponding target image in the target image dataset; The third determination unit is configured to determine a homography matrix for converting the at least four first key points into the at least four second key points; The third processing unit is configured to perform alignment processing on the basic image and the target image based on the homography matrix.

9. The apparatus of claim 7, wherein, The second processing unit is specifically configured to: perform spatial mapping on each pixel point of a basic image region block in each image region block pair to obtain a basic three-dimensional pixel point cloud, and perform density clustering on the basic three-dimensional pixel point cloud to obtain K first clusters; perform spatial mapping on each pixel point of a target image region block corresponding to the basic image region block to obtain a target three-dimensional pixel point cloud, and perform density clustering on the target three-dimensional pixel point cloud to obtain K second clusters; wherein K is a positive integer; pair the first clusters and the second clusters according to the centroids of the K first clusters and the centroids of the K second clusters.

10. An electronic device, comprising: A device includes a processor and a memory storing programs or instructions executable on the processor, which, when executed by the processor, implement the steps of the three-dimensional lookup table reconstruction method of any one of claims 1-5.

Images