Binocular view based regional color correction method and system

Abstract

The application provides a binocular view-based regional color correction method and system, left and right views of a target are synchronously collected by a binocular camera system; a three-dimensional point cloud model of the target is reconstructed based on the left and right views; a plurality of color corresponding point pairs are obtained, each point pair comprises a left image pixel point and a corresponding pixel point in a right image, and color values of the point pairs are recorded; a color transformation matrix is calculated by using an optimization algorithm based on the color values of the plurality of color corresponding point pairs, with the aim of minimizing color differences of the corresponding point pairs in the left and right views; the left and right views are sequentially subjected to color transformation matrix correction, weighted fusion and reinhard correction, so that a color difference-free texture map is generated, and the texture map is mapped to the three-dimensional point cloud model, so that a final color difference-free texture map is generated. The application effectively reduces color deviations of the three-dimensional reconstruction left and right views.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to a regional color correction method and system based on binocular view. BACKGROUND

[0002] It is a current mainstream technology to acquire images and reconstruct three-dimensional models through binocular or multi-camera systems. However, due to the differences in sensor characteristics, lens parameters, white balance settings and light environments of different cameras, there will inevitably be color differences between images acquired from different perspectives. When these images are fused into a unified texture map, color differences will cause obvious seams and visual discontinuity, seriously affecting the realism of the model and the accuracy of subsequent applications (such as biometric identification). Currently, the following methods are generally used for image fusion in the field:

[0003] 1. Global histogram matching: This method adjusts the color histogram of one image to match another image to unify the overall tone and brightness. However, this method is completely based on global statistical information, treating the face as a plane, without considering the amplification of color differences in high-curvature areas (such as the nose bridge) and over-correction in low-curvature areas (such as color distortion on the cheeks), and the single color transformation matrix cannot well handle local light changes caused by perspective differences between left and right cameras and differences in skin optical properties. Therefore, the three-dimensional geometry of the target object (such as the face) and the spatially varying light distribution are ignored. When the light is uneven or the object curvature is large, it will lead to "one-size-fits-all" over-correction or under-correction, resulting in non-uniform color distortion.

[0004] 2. Gradient domain fusion: For example, Poisson image editing, this technology creates seamless stitching by preserving image gradients. However, when there are large underlying color and brightness differences between two images, this method may cause color "drift" or loss of details in the fusion area, resulting in unnatural blurring.

[0005] 3. Learning-based color transfer algorithm: This type of method is complex and requires a large amount of paired data to train a deep learning model to learn the color mapping between images. Its generalization ability is limited by the training scene, and the calculation is time-consuming. When dealing with samples with large differences in processing equipment, light or material, it is easy to produce structural distortion or artifacts, making it difficult to meet the requirements of high real-time and high quality in application scenarios.

[0006] Prior art solutions:

[0007] 1. Generative 3D reconstruction method, such as CN120107481A, a single-image 3D head reconstruction method and system based on guided diffusion model, the core of which is to reconstruct a complete 3D head model from a single face photo. Its method is to first reconstruct a 3D model with front texture from the photo, then unfold the texture into a 2D map, and use a guided diffusion network to predict and repair the invisible areas in the map. Finally, the repaired complete texture is reattached to the 3D model. However, although such generative methods can create visually coherent and detailed textures, their technical core is in the "creation" rather than "restoration" of textures. Although the "created" texture is acceptable in the field of virtual people, games, etc. for general entertainment, it is completely unreliable in serious application scenarios that require high fidelity and individual uniqueness

[0008] 2. Global application calibration, such as CN119922394A, the core idea of the method is: under a controlled lighting condition, use a color-accurate reference camera and a camera to be calibrated to simultaneously shoot a standard object, and calculate a fixed, globally applicable color correction matrix for the camera to be calibrated based on the difference between the two imaging. However, this method has a fundamental flaw when applied to complex scenarios such as 3D model texture stitching. The single matrix calculated is applied globally and indiscriminately to the entire image, completely ignoring the spatial color non-uniformity caused by the three-dimensional surface of the object (such as the bridge of the nose and the cheeks of the face) and local lighting changes. When this matrix is directly applied to the stitching seam, even if the overall tone tends to be consistent, the boundary will still have perceptible transition marks due to the difference in local lighting reflection, making it impossible to achieve true visual seamless fusion

[0009] 3. High-order polynomial fitting correction, such as CN120495076A, which uses a high-order polynomial regression model for color correction, and uses the least squares method to perform nonlinear mapping and fitting on each color channel. Although this method has some nonlinear correction ability, its core defect compared to linear matrix transformation is the risk of overfitting caused by high model complexity: high-order polynomials are highly dependent on the distribution characteristics of training samples and are extremely sensitive to sample noise and outliers, making it easy to produce mapping relationships that are too consistent with training data but have decreased generalization performance, resulting in color distortion and detail confusion when correcting new images. Such methods are difficult to achieve stable color migration with limited samples, restricting their application in scenarios that require strict fidelity.

[0010] It can be known from the above that the many prior arts cannot effectively solve the unstable phenomena such as color distortion and detail confusion when correcting a new image. In high-precision application scenarios, more serious defects will be exposed, such as: 1. In medical and cosmetic simulation, color distortion leads to difficulty in identifying subcutaneous blood vessel texture, affecting the accuracy of surgical planning; 2. In virtual reality interaction, the visibility of the splicing boundary destroys the immersion, and significantly affects the user experience; 3. In biometric identification, the texture break caused by color difference leads to insufficient recognition reliability. SUMMARY

[0011] In view of the above situation, the main purpose of the present application is to provide a regional color correction method and system based on binocular view, which fundamentally solves the distortion in high-curvature area and the hard splicing boundary problem through left and right view combination and regional color difference correction.

[0012] The present application provides a regional color correction method based on binocular view, which comprises the following steps:

[0013] Synchronously acquiring the left view and the right view of the target through a binocular camera system;

[0014] Obtaining a plurality of groups of color corresponding point pairs, each group of point pairs comprising a left image pixel point and its corresponding pixel point in the right image, and recording the color values of the point pairs;

[0015] Taking the minimization of the color difference of the corresponding point pairs of the left and right views as the target, a color transformation matrix is calculated based on the color values of the plurality of groups of color corresponding point pairs by using an optimization algorithm;

[0016] Based on the left view and the right view, a three-dimensional point cloud model of the target is reconstructed;

[0017] Based on the reconstructed three-dimensional point cloud model, the surface of the three-dimensional point cloud model is divided into a direct correction area and a transition fusion area, the color values in the left view direct correction area are converted by using the color transformation matrix; the color values in the transition fusion area are converted by using the color transformation matrix, and are weighted and fused with the color values of the right view to obtain a preliminary corrected texture map, and the left view part in the preliminary corrected texture map is point-by-point corrected in brightness value by using the reinhard correction, and is mapped to the three-dimensional point cloud model to generate a final color difference-free texture map.

[0018] The present application also provides a regional color correction system based on binocular view, wherein the system applies the regional color correction method based on binocular view as described above, and the system comprises:

[0019] A data acquisition module is used for:

[0020] Synchronously acquiring the left view and the right view of the target through a binocular camera system;

[0021] a three-dimensional reconstruction module, configured to:

[0022] reconstruct a three-dimensional point cloud model of the target based on the left view and the right view;

[0023] a coefficient matrix calculation module, configured to:

[0024] obtain a plurality of groups of color corresponding point pairs, each group of point pairs comprising a left image pixel point and a pixel point corresponding to the left image pixel point in a right image, and record color values of the point pairs;

[0025] based on the color values of the plurality of groups of color corresponding point pairs, calculate a color transformation matrix by using an optimization algorithm, with the aim of minimizing color differences of the left view and the right view corresponding point pairs;

[0026] a color difference elimination module, configured to:

[0027] based on the reconstructed three-dimensional point cloud model, divide a surface of the three-dimensional point cloud model into a direct correction area and a transition fusion area, convert color values of the left view in the direct correction area using the color transformation matrix; convert color values of the left view in the transition fusion area using the color transformation matrix, and perform weighted fusion with color values of the right view to obtain a preliminary corrected texture map, and then perform point-by-point reinhard correction on the left view part of the preliminary corrected texture map, and map the corrected left view part to the three-dimensional point cloud model to generate a final color difference-free texture map.

[0028] Compared with the prior art, the present application has the following advantages:

[0029] The present application finds color corresponding point pairs in the left and right views, and then divides the texture map into a left view dominant area, a right view dominant area, and a band-shaped transition fusion area between the two areas according to the distribution of feature points and the source of texture. Based on reliable two-dimensional feature point corresponding samples, the optimal linear transformation parameters of each channel are independently solved to realize the conversion of the color space from the left view to the right view. In the transition area, the pixels from different views are independently corrected and mixed. This method effectively reduces the color deviation of the left and right views of the three-dimensional reconstruction, completely eliminates the hard splicing boundary, significantly reduces the overall color difference, realizes visual seamless fusion, improves the visual reality of the three-dimensional model, and is easy to extend to other fields that require accurate color alignment.

[0030] Additional aspects and advantages of the present application will be given in part in the following description, some of which will become apparent from the following description, or will be learned by practice of the embodiments of the present application. BRIEF DESCRIPTION OF DRAWINGS

[0031] Figure 1 a structure diagram of a regionalized color correction system based on binocular views according to the present application;

[0032] Figure 2 For the embodiment 1 verification process, detect feature points and visualize the sampling distribution;

[0033] Figure 3 For the embodiment 1 verification process, dlib face 68 feature point distribution;

[0034] Figure 4 For the embodiment 1 verification process, sub-region correction schematic diagram;

[0035] Figure 5 For the embodiment 1 verification process, uncorrected effect and effect diagram of only setting transition zone;

[0036] Figure 6 For the embodiment 1 verification process, multi-view diagram of the final modeling result;

[0037] Figure 7 For the embodiment 1 verification process, mediapip 468 feature point distribution schematic diagram;

[0038] Figure 8 For the embodiment 1 verification process, final modeling result schematic diagram of mediapip 468 feature points;

[0039] Figure 9 For the embodiment 2 verification process, random point cloud through mapping relationship in two-dimensional image corresponding point sampling distribution diagram;

[0040] Figure 10 For the embodiment 2 verification process, sub-region correction schematic diagram;

[0041] Figure 11 For the embodiment 2 verification process, uncorrected effect and effect diagram of only setting transition zone;

[0042] Figure 12 For the embodiment 2 verification process, multi-view diagram of the final modeling result. DETAILED DESCRIPTION

[0043] Embodiments of the present application are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals represent the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by reference to the drawings are exemplary and are used only to explain the present application, and should not be understood as limiting the present application.

[0044] These and other aspects of embodiments of the present application will become apparent from the following description and the accompanying drawings. In these descriptions and drawings, some specific embodiments of the embodiments of the present application are specifically disclosed to represent some ways of embodying the principles of the embodiments of the present application, but it should be understood that the scope of the embodiments of the present application is not limited thereto.

[0045] Embodiment 1

[0046] Please refer to Figure 1 The embodiment provides a binocular view-based regional color correction method, color corresponding point pairs are obtained in a feature point matching-based manner, without relying on complex three-dimensional geometric information, and the method has high precision and real-time advantages, and can be conveniently migrated to medical imaging, film special effects, virtual reality and other fields which need to perform high-fidelity alignment on surface colors. The method comprises the following steps:

[0047] Step 1: image acquisition. The images of a target are synchronously collected from two different view angles by a binocular camera system, and are denoted as a left image and a right image.

[0048] Step 2: feature point detection. The left and right images are processed respectively by a feature point detection and matching technology, and two groups of two-dimensional pixel point coordinates with corresponding relations are obtained. The color values (the color values can be RGB, LAB or HSV, etc., and the embodiment adopts RGB) of the corresponding points are extracted, and N groups of corresponding point pairs are formed. To increase robustness, the point pairs with a model output confidence lower than a preset threshold value can be screened out, for example. (The feature point technology includes but is not limited to a face key point detection model, SIFT, ORB or SuperPoint, etc.)

[0049] Step 3: calculation of color transformation matrix. Based on the color values of the multiple groups of color corresponding point pairs, an optimization model (such as least square method, robust weighted least square method or RANSAC, etc.) is established, and a color transformation matrix is solved, so that the color values of the N groups of corresponding points are smallest after transformation.

[0050] Step 4: three-dimensional model reconstruction and texture region division. A three-dimensional point cloud model of the target is reconstructed through feature point registration technology. The left image and the right image are spliced to find the center line of the overlapping region of the left image and the right image, the position corresponding to the center line on the three-dimensional point cloud model surface is defined as a transition line, the transition line extends to both sides to form a transition fusion area, and other parts are defined as a direct correction area. The length of the transition line extending to both sides is determined by the color difference in the preset range on both sides of the transition line, and the calculation process of the width of the transition fusion area is related to the following formula:

[0051] ;

[0052] Wherein, is the width of the transition fusion area, is a coefficient, is an intercept, is the average color difference in the preset range of the overlapping region, and the preset range in the embodiment is 1 cm.

[0053] Step 5: Weighted fusion realizes color difference correction to generate the final colorless texture map. If the pixel is from the direct correction area of the left image, its original color value is multiplied by the color transformation matrix for conversion. If the pixel is located in the transition fusion area, its final color is calculated by the following weighted mixing formula:

[0054] ;

[0055] wherein, is the final color value of the pixel point p, is the color value of the pixel point p in the current left view; is the color transformation matrix; is the color value of the pixel point p in the current right view; is the weight coefficient of the pixel point p, which is obtained by a normalized linear distance function of the pixel point p to the boundary of the transition area T, and the value range is [0, 1], which ensures that w tends to 0 near the direct correction area (left image area) in the transition fusion area, and w tends to 1 near the other side (right image area).

[0056] After color transformation matrix correction and weighted fusion, the preliminary corrected map is obtained. Finally, the left view in the colorless texture map is point-by-point corrected in brightness by reinhard, to individually fine-tune the high-light detail area (such as the white of the eye area), without affecting the color balance of the medium and low brightness area such as skin color, so that the color of the left view is infinitely close to the right view, and finally mapped to the three-dimensional point cloud model to obtain the final colorless texture map.

[0057] In the embodiment, the present application realizes color correction based on mature two-dimensional face feature point detection and matching technology. The method uses widely used feature point positioning models (such as Dlib, MediaPipe or HRNet) to automatically extract face key points and establish a robust correspondence across views without relying on complex three-dimensional geometric information. After establishing accurate pixel-level correspondence, the texture map is divided into left image dominant area, right image dominant area and band-shaped transition fusion area between them according to the feature point distribution and texture source.

[0058] Based on reliable 2D feature point corresponding samples, the optimal linear transformation parameters for each channel are independently solved through channel-specific least squares optimization to achieve color space conversion from the left view to the right view. Within the transition zone, pixels originating from different views are independently corrected and blended. This method effectively reduces color deviation between the left and right views in 3D reconstruction, completely eliminates hard-joining boundaries, significantly reduces overall color difference, achieves seamless visual fusion, enhances the visual realism of the 3D model, and is easily extendable to other fields requiring precise color alignment. Feature point technology has been fully validated in fields such as face recognition, expression analysis, and virtual makeup try-on, possessing advantages in high precision and real-time performance. It can be easily transferred to multiple fields requiring high-fidelity surface color alignment, such as medical imaging, film and television special effects, and virtual reality.

[0059] This embodiment constructs a local color mapping model based on the correspondence of feature points. This method significantly improves the accuracy and stability of cross-view color correction, effectively avoids overfitting, eliminates splicing gaps and non-uniform color differences, and significantly enhances the realism and visual consistency of 3D model textures.

[0060] Example 2

[0061] Please see Figure 1 This embodiment provides a regionalized color correction method based on binocular views. It uses a 3D point projection approach to obtain color-corresponding point pairs, effectively solving the technical problems of non-uniform color distortion and unnatural stitching boundaries that occur during multi-view texture fusion due to ignoring 3D geometric information and employing a global processing strategy. The method includes the following steps:

[0062] Step 1: Image and 3D Information Acquisition. Using a binocular camera system, images of the target are simultaneously acquired from two different perspectives, denoted as the left and right images. A 3D point cloud model of the target is then reconstructed using techniques such as feature point registration.

[0063] Step 2: 3D Spatial Corresponding Point Filtering. On the surface of the 3D point cloud model, based on the importance of different regions, N 3D spatial points are filtered out using weighted hierarchical sampling. Specifically, facial feature point detection (including but not limited to the dlib 68-point detection model) is used to divide the regions with high texture variations, such as eyes and mouth, according to the distribution of feature points. Hierarchical sampling is performed according to weights (weights can be 10% for eyes, 10% for eyebrows, 20% for mouth, and 60% for cheeks). After obtaining the weighted sampled 3D spatial points, the corresponding 2D pixel coordinates and color values ​​(color values ​​can be RGB, LAB, or HSV, etc., this embodiment uses RGB) of these 3D spatial points are calculated on the left and right images respectively using camera calibration parameters and projection mapping relationships. N sets of corresponding point pairs are formed and stored in two arrays respectively. The mean color difference is calculated, and outliers with color differences greater than twice the mean are filtered out to ensure the validity of the points.

[0064] Step 3: Calculation of the color transformation matrix. Based on the color values ​​of multiple sets of corresponding color point pairs, establish an optimization model (such as least squares method, robust weighted least squares method, or RANSAC, etc.) with the goal of solving a color transformation matrix that minimizes the difference in color values ​​of N sets of corresponding points after transformation.

[0065] Step 4: Texture Region Segmentation and Correction. The left and right images are stitched together to find the center line of the overlapping area. The position of this center line on the surface of the 3D point cloud model is defined as the transition line. The transition line extends to both sides to form a transition blending zone. The remaining area is defined as the direct correction zone. The length of the transition line extending to both sides is determined by the color difference within a preset range on both sides of the transition line. The calculation process for the width of the transition blending zone follows the following formula:

[0066] ;

[0067] in, It is the width of the transition and fusion zone. It is a coefficient. It is the intercept. It is the average color difference within a preset range on both sides of the transition line. In this embodiment, the preset range length is 1cm.

[0068] Step 5: Weighted blending is used to correct color differences, generating the final color-aberration-free texture map. If a pixel originates from the direct correction area of ​​the left image, its original color value is multiplied by a color transformation matrix for conversion. If the pixel is located in the transition blending area, its final color is calculated using the following weighted blending formula:

[0069] ;

[0070] in, It is the final color value of pixel p. It is the color value of pixel p in the current left view; It is a color transformation matrix; It is the color value of pixel p in the current right view; It is the weight coefficient of pixel p, which is obtained by the normalized linear distance function from pixel p to the boundary of the transition region T. Its value range is [0, 1], ensuring that w approaches 0 when it is close to the direct correction region (left image region) in the transition fusion region, and approaches 1 when it is close to the other side (right image region).

[0071] After color transformation matrix correction and weighted fusion, a preliminary corrected texture is obtained. Finally, the brightness of the left view in the colorless texture map is reinforced point by point to fine-tune the bright details (such as the whites of the eyes) without affecting the color balance of the medium and low brightness areas such as skin color. This makes the color of the left view infinitely close to that of the right view. Finally, it is mapped onto the 3D point cloud model to obtain the final colorless texture map.

[0072] In this embodiment, the present invention constructs a regional color difference correction framework based on three-dimensional geometric constraints. Unlike traditional methods that match on a two-dimensional image plane, this method uses precise corresponding point sampling under three-dimensional geometric constraints, directly utilizing the reconstructed three-dimensional point cloud model. Weighted layered sampling is applied to the surface of the three-dimensional point cloud model according to the importance of different regions. Through a precise camera projection model, the true corresponding pixel and its RGB value in the left and right views are found for each three-dimensional point. This ensures that the calculation of color differences is strictly based on three-dimensional geometric correspondence, fundamentally avoiding two-dimensional matching errors caused by viewing angle differences and surface deformation.

[0073] In terms of region processing, the core basis of "region division" is the original camera view (left or right image) from which each pixel in the final texture map originates and their spatial proximity in the 3D model. Specifically, it is divided into: the left-image dominant region, the right-image dominant region, and a strip-shaped transition blending region located near the texture stitching boundary between the left and right views. This scheme adopts a dual mechanism of channel optimization + transition region hybridization to achieve accurate correction: First, using the obtained precise corresponding point pairs, the optimal linear transformation parameters of the R, G, and B channels are independently solved through channel least squares optimization to transform the color space of the left view to the color space of the right view. Second, in the transition blending region, pixels originating from the left and right views are independently color corrected. Verification shows that this scheme effectively solves the color deviation problem in 3D reconstruction, completely eliminates hard stitching boundaries, significantly reduces overall color difference and non-uniformity, achieves visually invisible seamless blending at the texture stitching boundary, and enhances the visual realism of the 3D model.

[0074] This embodiment improves the accuracy of color correction, eliminating local color differences on 3D curved surfaces caused by variations in lighting and camera settings. Furthermore, it achieves smooth, seamless visual fusion in the transition areas of texture stitching, eliminating stitching gaps and enhancing the visual realism and fidelity of the 3D point cloud model.

[0075] Example 3

[0076] This embodiment discloses an outlier filtering method based on Embodiments 1 and 2. By fusing three-dimensional curvature approximations and color information, it effectively distinguishes between true color differences and matching errors. The specific steps are as follows:

[0077] Obtain the pixel coordinates of each point pair, and calculate the associated 3D coordinates of each point pair based on the pixel coordinates of the point pair and the stereo camera calibration parameters.

[0078] Search for K nearest neighbors centered on the three-dimensional coordinates associated with each point pair to obtain the nearest neighbor set;

[0079] Principal component analysis and covariance calculation are performed sequentially on the nearest neighbor set to obtain the eigenvalues ​​of the covariance matrix.

[0080] The curvature approximation is defined as the normalized result of the eigenvalues ​​of the covariance matrix, thus obtaining the curvature approximation for each point pair;

[0081] The color information, geometric information, and curvature approximation of each point pair are combined to form a joint feature vector;

[0082] The intrinsic distribution pattern of the joint feature vectors is learned using a Gaussian mixture model. After the learning is completed, the model parameters are obtained.

[0083] The likelihood value of each point pair is obtained by calculating the overall probability that each point pair in the joint feature vector belongs to the overall data distribution using the model parameters.

[0084] Calculate the median and standard deviation of the likelihood values ​​for all point pairs, and use the median and standard deviation to construct a dynamic threshold;

[0085] The dynamic threshold is compared with the likelihood values ​​of all point pairs, and point pairs with values ​​less than the dynamic threshold are filtered out.

[0086] In this embodiment of the invention, by fusing three-dimensional curvature approximations and color information, the true color difference and matching errors are effectively distinguished. For example, at the edge of an object (high curvature approximation area), even if color shifts occur due to differences in viewing angle (such as metallic reflection), correct point pairs are still retained; in flat areas (low curvature approximation area), color anomalies (such as noise or occlusion errors) are strictly filtered out. By dynamically learning the distribution law of point pairs through a Gaussian mixture model (GMM), interference from sudden changes in lighting, shadows, reflections, etc., is effectively overcome.

[0087] Example 4

[0088] To overcome the reliance on preset transition band widths and manual boundary divisions during the partitioning process, and to address the issues of unnatural transitions and detail loss caused by large curvature variations or uneven color difference distributions, this embodiment proposes a method that unifies transition blending region division and color blending into the same global optimization framework. This method constructs an energy function and adaptively iteratively solves the problem, ensuring that the position and width of the transition band, as well as the color blending strategy within the region, simultaneously reach global optimum. While minimizing color differences, it preserves texture details that coexist with the surface's geometric features, generating texture maps with a stronger visual realism. The specific steps are as follows:

[0089] Obtain the color-corresponding point pairs in the left and right views and calculate the color transformation matrix. Extract the center line of the left and right view stitching boundary in the 3D point cloud model as the initial transition line, and form an initial rectangular transition blending area on both sides of it with a basic width.

[0090] To globally optimize the color consistency and visual realism of texture maps, we constructed an energy function containing three types of constraints and minimized it through iterative optimization. The construction process of this energy function is as follows:

[0091] The color consistency term calculates the difference between the color of each vertex in the current iteration's texture and the corresponding pixel color in the left and right views after color transformation matrix correction, as well as the difference between it and the corresponding pixel color in the original right view. The core objective of this term is to drive the optimized texture color to be as consistent as possible with the corrected left and right views, thereby effectively eliminating color differences. Its input data comes from the texture color of the current iteration, as well as pre-calculated color data for the corrected left and right views and the original right view.

[0092] For color difference integration, this item assigns different trust weights based on the spatial location of the vertex and performs weighted summation: if a vertex can only be captured by a single camera, then the camera is fully trusted; if the vertex is located in an overlapping transition blending area, its weight is determined by the normalized distance from the vertex to the image boundary, and the smaller the distance, the more the color correction depends on the view.

[0093] The geometric feature preservation term focuses on high-curvature regions and is constructed by accumulating the difference between the current iteration's texture color gradient change and known geometric curvature changes. Inputs to this term include the color gradient of the current iteration's texture computed within its neighborhood, and vertex curvature data pre-extracted from the 3D point cloud model. Its core objective is to encourage clear texture changes even in regions of drastic geometric variations. This effectively suppresses detail loss caused by over-smoothing by penalizing fusion schemes that might blur feature edges by assigning higher weights to high-curvature regions.

[0094] The smoothness constraint term is used to constrain the continuity of color gradients between adjacent vertices within the transition blending region, avoiding abrupt changes or unnatural boundary effects. Its construction process involves calculating the color gradients between adjacent pixels or vertices within the transition band and modeling the gradient change rate by summing the squares or using total variation (TV) regularization. The input data for this term includes the pixel color field of the texture in the current iteration. Its core objective is to encourage smooth spatial variations in the color field, resulting in a continuous and natural color transition in the stitching region, ultimately achieving a seamless blending effect.

[0095] In the initial stage, the left view is used as the initial texture, and each subsequent iteration uses the optimization result of the current iteration as the texture for the next stage. The energy function is iteratively minimized using either the conjugate gradient method or the Gauss-Newton method. During optimization, the boundary position and local width of the transition fusion region are dynamically updated according to the energy decrease direction: automatically widening in areas with high color difference or high curvature, and automatically narrowing in areas with low difference. After each iteration, the pixel fusion result within the transition region is calculated, and the color gradient continuity index is evaluated simultaneously. If the index does not reach a certain value, the transition width is further adjusted in local areas with significant gradient changes, and the boundary curve is refitted in real time. The iteration stops when the continuity index of the previous two iterations converges or the change amplitude is lower than a preset threshold, yielding the final optimized transition fusion region.

[0096] This embodiment simultaneously determines the partitioning scheme and fusion strategy within a unified optimization framework, enabling the width and shape of the transition band to adaptively adjust according to the distribution of 3D geometric features and local color differences. This method avoids the discontinuous fusion and blurred details problems caused by traditional fixed partitioning. While minimizing color differences, it effectively preserves the geometric details and texture features of high-curvature areas of the surface, achieving a smooth and natural transition globally, and ultimately generating a color-aberration-free texture map with higher visual realism and fidelity.

[0097] Example 5

[0098] like Figure 1 As shown, this embodiment proposes a binocular view-based regional color correction system, wherein the system applies the binocular view-based regional color correction method described above, and the system includes:

[0099] The data acquisition module is used for:

[0100] The left and right views of the target are acquired simultaneously using a binocular camera system.

[0101] The 3D reconstruction module is used for:

[0102] Based on the left and right views, a 3D point cloud model of the target is reconstructed.

[0103] The coefficient matrix calculation module is used for:

[0104] Obtain multiple sets of color-corresponding point pairs. Each point pair includes a pixel in the left image and its corresponding pixel in the right image, and record the color value of the point pair.

[0105] With the goal of minimizing the color difference between corresponding point pairs in the left and right views, a color transformation matrix is ​​calculated using an optimization algorithm based on the color values ​​of multiple sets of corresponding point pairs.

[0106] Color difference elimination module, used for:

[0107] Based on the reconstructed 3D point cloud model, the surface of the 3D point cloud model is divided into a direct correction area and a transition blending area. The color values ​​of the part located in the direct correction area of ​​the left view are transformed using a color transformation matrix. The color values ​​of the left view in the transition blending area are transformed using a color transformation matrix and then weighted and blended with the color values ​​of the right view to obtain a pre-corrected texture. The brightness values ​​of the left view part in the pre-corrected texture are then corrected point by point using reinhard and mapped to the 3D point cloud model to generate the final color difference-free texture map.

[0108] To verify the effectiveness of the present invention, the method of Example 1 was tested. Figure 2 As shown, Figure 2 To detect feature points and visualize the sampling distribution, where Figure 2 In this context, 'a' represents the Dlib 68-point facial model. Figure 2 In this context, b represents the Dlib facial 194-point model.

[0109] The corresponding points are obtained using the dlib 68-feature face detection model. The matrix is ​​a 3x3 square matrix in RGB order, with only three linear unknowns for each channel, in the following form:

[0110] ;

[0111] Figure 3 It is the dlib face 68 feature point distribution. First, remove those feature points that are not visible in the left image but visible in the right image due to different camera angles. Then, calculate the color transformation matrix using the least squares method.

[0112] like Figure 4 As shown, the left image is first transformed globally using the color transformation matrix M. The transition region is then individually transformed again using a linear weighting function based on its distance to the transition line y=-60, resulting in a smoother transition. Figure 5 As can be seen from 'a', there are obvious color differences and splicing marks when it is uncorrected. Through... Figure 5 As shown in 'b', although there is a gradual effect when only a transition band is set, the color difference is still visible to the naked eye.Figure 6 It can be seen that after the method of Example 1 was used for correction and the transition zone was set, the modeling result had no obvious stitching marks. From different angles, the face model had uniform color and smooth transition, and the color difference was almost invisible to the naked eye. The measured ΔE was less than 55% of the original.

[0113] The corresponding points are obtained using the MediaPip 468 feature point detection model, and the matrix is ​​a 3x3 square matrix in RGB order. Figure 7 This is the distribution of MediaPip 468 feature points:

[0114] First, remove the points that are not visible in the left image but are visible in the right image due to different camera angles. There are 432 points left. Then, calculate the color transformation matrix using the least squares method. Figure 8 The modeled results show no obvious stitching marks, smooth transitions, and color differences that are almost invisible to the naked eye. Furthermore, the measured ΔE is less than 60% of the original value.

[0115] To verify the effectiveness of the present invention, the method of Example 2 was tested. Figure 9 As shown, first, the images from the left and right cameras are acquired, and the random point cloud is mapped to the corresponding points in the two-dimensional image.

[0116] When 300 random point cloud data points are taken, the matrix is ​​a third-order square matrix in RGB order, each channel has only three first-order unknowns, and the width of the transition region is 1cm.

[0117] The following example uses channel B:

[0118] ;

[0119] in, It is the color value of channel B at any point in the right image. This indicates the color value of channel B at the corresponding point in the left image. This represents the element in the first row and second column of a third-order square matrix. This indicates the G channel color value of the corresponding point in the left image. This indicates the R channel color value of the corresponding point in the left image.

[0120] Alternatively, the matrix can be transformed into a second-order polynomial matrix (adding a constant term and a second-order term to a 3x3 matrix), with ten unknowns per channel and a transition region width of 1cm. This represents...

[0121] Below is the equation for channel B: (The equations for other channels can be deduced similarly)

[0122] ;

[0123] in, It is a constant term. to These are the linear coefficients of the corresponding channels. to These are the coefficients of the squared terms of the corresponding channels. to It is the cross term coefficient of the corresponding channel.

[0124] like Figure 10 As shown, the left image is first transformed globally using an RGB coefficient matrix. Then, the transition region is individually transformed again using a linear weighting function based on its distance to the transition line y=-60, resulting in a smoother transition. Figure 11 As can be seen from 'a', there are obvious color differences and splicing marks when it is uncorrected. Through... Figure 11 As shown in 'b', although there is a gradual effect when only a transition band is set, the color difference is still visible to the naked eye. Figure 12 As can be seen, after correcting and setting the transition zone using the method in Example 2, the modeled result had no obvious splicing marks, the transition was smooth from all angles, and the color difference was almost invisible to the naked eye.

[0125] It should be understood that although the steps in the flowcharts of the various embodiments of the present invention are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the various embodiments may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least a portion of the sub-steps or stages of other steps.

[0126] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0127] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0128] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of this patent should be determined by the appended claims.

Claims

1. A regional color correction method based on binocular vision, characterized in that, The method includes the following steps: The left and right views of the target are acquired simultaneously using a binocular camera system. Based on the left and right views, a 3D point cloud model of the target is reconstructed. Obtain multiple sets of color-corresponding point pairs. Each point pair includes a pixel in the left image and its corresponding pixel in the right image, and record the color value of the point pair. With the goal of minimizing the color difference between corresponding point pairs in the left and right views, a color transformation matrix is ​​calculated using an optimization algorithm based on the color values ​​of multiple sets of corresponding point pairs. Based on the reconstructed 3D point cloud model, the surface of the 3D point cloud model is divided into a direct correction area and a transition blending area. The color values ​​of the part located in the direct correction area of ​​the left view are transformed using a color transformation matrix. The color values ​​of the left view in the transition blending area are transformed using a color transformation matrix and then weighted and blended with the color values ​​of the right view to obtain a pre-corrected texture. The brightness values ​​of the left view part in the pre-corrected texture are corrected point by point using reinhard and mapped to the 3D point cloud model to generate the final color difference-free texture map. The process of dividing the surface of a 3D point cloud model into a direct correction zone and a transition fusion zone involves: stitching the left and right images together to find the center line of the overlapping area; defining the position of this center line on the surface of the 3D point cloud model as the transition line; and extending the transition line to both sides to form the transition fusion zone. The remaining area is defined as the direct correction zone. The length of the transition line extending to both sides is determined by the color difference within a preset range on both sides of the transition line. The corresponding process follows the following relationship: ； in, It is the width of the transition and fusion zone. It is a coefficient. It is the intercept. It is the average color difference within a preset range on both sides of the transition line; Weighted blending is specifically performed as follows: For a pixel within the transition blending region, its final color value is calculated using the following formula: ； in, It is the final color value of pixel p. It is the color value of pixel p in the current left view; It is a color transformation matrix; It is the color value of pixel p in the current right view; It is the weight coefficient of pixel p, and its value ranges from [0, 1]. The weight coefficient of pixel p is calculated by normalization based on the distance from pixel p to the boundary of the transition blending region, ensuring that w approaches 0 when it is close to the direct correction region in the transition blending region, and approaches 1 when it is close to the other side.

2. The regional color correction method based on binocular view according to claim 1, characterized in that, The specific method for obtaining multiple sets of color-corresponding point pairs is as follows: based on the importance of different regions, a set of three-dimensional points is selected on the surface of the three-dimensional point cloud model using weighted hierarchical sampling; through camera calibration parameters, each three-dimensional point is projected onto the left and right views respectively to obtain the corresponding two-dimensional pixel point pairs, thus forming a set of color-corresponding point pairs.

3. The regional color correction method based on binocular view according to claim 2, characterized in that, After obtaining the two-dimensional pixel pairs, the process also includes a step of filtering out outliers: calculating the average color difference of all point pairs and filtering out outlier point pairs whose color difference is greater than twice the average.

4. The regional color correction method based on binocular view according to claim 1, characterized in that... The specific method for obtaining multiple sets of color-corresponding point pairs is as follows: through feature point detection and matching algorithms, two-dimensional feature point pairs are directly detected and matched on the left and right views to form a set of color-corresponding point pairs.

5. The regional color correction method based on binocular view according to claim 4, characterized in that, After obtaining the two-dimensional pixel point pairs, there is also a step of filtering out abnormal points, specifically: filtering out matching point pairs with a matching confidence level lower than a preset threshold.

6. The regionalized color correction method based on binocular view according to any one of claims 1 to 5, characterized in that, The optimization algorithm is one of the following: least squares method, robust weighted least squares method, or RANSAC algorithm.

7. A regionalized color correction system based on binocular vision, characterized in that, The system applies the binocular-view-based regional color correction method as described in claim 6, and the system includes: The data acquisition module is used for: The left and right views of the target are acquired simultaneously using a binocular camera system. The 3D reconstruction module is used for: Based on the left and right views, a 3D point cloud model of the target is reconstructed. The coefficient matrix calculation module is used for: Obtain multiple sets of color-corresponding point pairs. Each point pair includes a pixel in the left image and its corresponding pixel in the right image, and record the color value of the point pair. With the goal of minimizing the color difference between corresponding point pairs in the left and right views, a color transformation matrix is ​​calculated using an optimization algorithm based on the color values ​​of multiple sets of corresponding point pairs. Color difference elimination module, used for: Based on the reconstructed 3D point cloud model, the surface of the 3D point cloud model is divided into a direct correction area and a transition blending area. The color values ​​of the part located in the direct correction area of ​​the left view are transformed using a color transformation matrix. The color values ​​of the left view in the transition blending area are transformed using a color transformation matrix and then weighted and blended with the color values ​​of the right view to obtain a pre-corrected texture. The brightness values ​​of the left view part in the pre-corrected texture are then corrected point by point using reinhard and mapped to the 3D point cloud model to generate the final color difference-free texture map.

Images