Color restoration and evaluation method for images shot by automobile central control screen

Abstract

The invention discloses a color restoration and evaluation method for an image shot by an automobile central control screen, belongs to the technical field of automobile man-machine engineering, and particularly relates to the color restoration and evaluation method for the image shot by the automobile central control screen. The invention aims to solve the problems that the color rendition accuracy of an image shot by an existing automobile central control screen is low, and an existing color rendition effect evaluation technology lacks an effective quantitative evaluation system. The method comprises the following steps: acquiring an image data set containing a standard color card and an automobile central control screen; performing moire suppression on the acquired image containing the standard color card and the automobile central control screen to obtain an image after moire interference suppression; cutting a standard color card image from the image after the moire interference is suppressed; generating a filter file based on the standard color card image; correcting the image after the moire interference is suppressed based on the filter file to obtain a corrected image; and carrying out effect evaluation on the corrected image.

Description

Technical Field

[0001] This invention belongs to the field of automotive human-machine engineering technology, and relates to a method for color reproduction and evaluation of images captured by automotive central control screens. Background Technology

[0002] With the acceleration of automotive intelligence, the intelligent cockpit has become the core carrier for the development of human-machine interaction technology. As a key interactive interface, the color design of the central control display screen directly affects the driver's visual cognitive efficiency and information acquisition accuracy, thus impacting driving safety and the interactive experience. In the field of automotive ergonomics, the visual effect research, ergonomic evaluation, and optimization iteration of central control screen colors all rely on the accurate quantitative analysis of screen colors. However, at present, it faces two major technological bottlenecks: First, the acquisition of raw color information is limited. Raw color data is a core prerequisite for conducting scientific analysis and universal research, but due to industry technical barriers and commercial confidentiality mechanisms, researchers can only obtain images of the central control screen through photography. However, due to the influence of lens optical characteristics, sensor differences, ambient light interference, and screen pixel arrangement, captured images generally suffer from color distortion, color cast, and moiré interference, resulting in data that cannot reflect true colors. This "data acquisition distortion" problem inherently biases research based on captured images. Second, there is a lack of key technologies for evaluating color reproduction effects. Even with the construction of color reproduction models, existing technologies still lack an effective quantitative evaluation system. In pixel-level color difference analysis, due to factors such as shooting angle deviation, lens distortion, screen curvature, and device shake, different images of the same content are prone to geometric misalignment, resulting in inaccurate pixel matching. Calculating color difference directly without registration introduces additional errors, affecting the objectivity of the assessment. Summary of the Invention

[0003] This invention aims to address the problems of low color reproduction accuracy in automotive central control screen images due to color distortion caused by moiré patterns, reflection interference, and equipment color difference in traditional shooting processes, as well as the lack of an effective quantitative evaluation system for existing color reproduction effect evaluation technologies. The invention proposes a method for color reproduction and evaluation of automotive central control screen images.

[0004] The specific process of a method for color reproduction and evaluation of images captured by a car central control screen is as follows:

[0005] Step 1: Collect an image dataset containing a standard color chart and a car infotainment screen; Step 2: Perform moiré suppression on the collected images containing the standard color chart and the car infotainment screen to obtain an image with suppressed moiré interference; Step 3: Crop the standard color chart image from the image obtained in Step 2 with suppressed moiré interference; Generate a filter file based on the standard color chart image; Correct the image obtained in Step 2 with suppressed moiré interference based on the filter file to obtain a corrected image; Step 4: Evaluate the effect of the corrected image.

[0006] The beneficial effects of this invention are as follows: Addressing the difficulties in acquiring original color information and the severe distortion of captured images, this invention innovatively employs a shooting device equipped with a polarizing filter and performing preliminary white balance adjustment. It combines this with an electronic color chart to construct a target image set, utilizes a pre-trained super-resolution moiré removal model to suppress interference, and employs 3D LUT Creator software to achieve color correction and 3D lookup table optimization. Ultimately, it achieves a true reconstruction of the color information of the captured images, effectively solving the color distortion problems caused by moiré patterns, reflection interference, and equipment color differences in traditional shooting methods. This provides a data acquisition method for color research in the field of automotive ergonomics. Regarding color reproduction effect evaluation, this invention constructs a color reproduction evaluation model that includes color difference analysis, automatic registration, and visual visualization. It uses Imatest software for professional color chart color difference evaluation, combined with an improved feature registration technology based on pre-alignment and region filtering, to achieve accurate alignment between the real image and the corrected image, and generates a regional color difference heatmap, effectively solving the evaluation error problem caused by image geometric misalignment. This evaluation system meets the self-verification requirements under experimental conditions, and can quantitatively evaluate and visually verify the color reproduction effect, providing technical support for the scientific calibration of color reproduction models and the transformation of technological achievements. Attached Figure Description

[0007] Figure 1 This is a flowchart of the present invention; Figure 2a This is a rendering of the color chart. Figure 2b A rendering of the target image captured on the central control screen; Figure 3 The effect of moiré pattern removal using the ESDNet model; Figure 4 This is a diagram showing the correction effect after color chart mapping using 3D LUT Creator Pro software; Figure 5a This is the original image; Figure 5b To generate a before-and-after comparison of the effects of image correction using 3D LUT Creator Pro software in conjunction with Adobe Photoshop's Camera Raw module and a custom .xmp configuration file; Figure 6 This image shows the color difference evaluation effect of color chart correction using the Colorcheck function of Imatest software. Ideal: Ideal (color, used for comparison); Camera: Camera (actual color in the shot); Mean camera chroma (saturation) = 103.8%: Average camera chroma (saturation) = 103.8%; Color errors: Color errors. chroma corr. mean=1.84, max=8.23: corrected The average chromaticity value is 1.84, and the maximum value is 8.23. uncorr. mean = 1.9, max = 8.7: Uncorrected The average chromaticity is 1.9, and the maximum chromaticity is 8.7. Mean = 2.62, Max = 12.7: Mean = 2.62, Maximum = 12.7; aRGB D65: aRGB color space based on D65 light source; ColorChecker: Color chart (standard color chart used for color calibration); Indicates red and green components; The yellow and blue components are represented. Figure 7 shows a color difference heatmap for evaluating the captured and corrected images using an improved feature registration algorithm based on pre-alignment and region filtering. Heatmap: Color difference heatmap. The color bars on the right are a visual mapping of color difference values: Blue → Cyan → Green → Yellow → Orange → Red (0 → 20 → 40 → 60 → 80 → 100): The color difference gradually increases, and the red area indicates the most serious color deviation. Detailed Implementation

[0008] Specific Implementation Method 1: The specific process of this implementation method for color restoration and evaluation of images captured by a car central control screen is as follows: Step 1: Use a mobile phone or camera to capture an image dataset containing a standard color chart and a car central control screen; Step 2: Perform moiré suppression on the captured images containing the standard color chart and the car central control screen to obtain an image after suppressing moiré interference; Step 3: Crop the standard color chart image from the image obtained in Step 2 after suppressing moiré interference; Generate a filter file based on the standard color chart image; Correct the image obtained in Step 2 after suppressing moiré interference based on the filter file to obtain a corrected image; Step 4: Evaluate the effect of the corrected image.

[0009] Specific Implementation Method Two: This implementation method differs from Specific Implementation Method One in that: in step one, a mobile phone or camera is used to collect an image dataset containing the standard color chart and the central control screen; the specific process is as follows:

[0010] Place the standard color chart image on the left side of the car's central control screen; use a camera equipped with a polarizing filter (such as a digital camera or smartphone) to photograph the standard color chart image and the car's central control screen with different interfaces, obtaining an image dataset; each image in the dataset contains both the standard color chart and the car's central control screen. Where the equipment allows, ambient illuminance and color temperature information can be recorded simultaneously for reference in subsequent error analysis. Other steps and parameters are the same as in Specific Implementation Method 1.

[0011] Specific Implementation Method 3: This implementation method differs from Specific Implementation Method 1 or 2 in that: in step 2, moiré pattern suppression is performed on the acquired image containing the standard color chart and the car's central control screen to obtain an image after suppressing moiré pattern interference. The specific process is as follows: the acquired image containing the standard color chart and the car's central control screen is input into the pre-trained neural network model ESDNet, and the pre-trained neural network model ESDNet outputs the image after suppressing moiré pattern interference (the image containing the standard color chart and the car's central control screen). The pre-trained neural network model ESDNet is an open-source pre-trained neural network model that performs moiré pattern interference suppression processing on the acquired image to improve the detail restoration of the image. ESDNet is a publicly available image super-resolution demoiré pattern model with semantic alignment and scale awareness modules, suitable for processing moiré pattern interference in 4K resolution images (model address: GitHub - CVMI-Lab / UHDM: (ECCV2022) This is the official PyTorch implementation of ECCV2022 paper: Towards Efficient and Scale-RobustUltra-High-Definition Image Demoireing). In this invention, the model is incorporated as an image preprocessing module into the overall color restoration process, serving subsequent color card matching and color difference evaluation tasks, thereby improving the overall color restoration accuracy. Other steps and parameters are the same as in specific implementation methods one or two.

[0012] Specific Implementation Method Four: This implementation method differs from Specific Implementation Methods One to Three in that: in step three, a standard color chart image is cropped from the image obtained in step two after suppressing moiré interference; a filter file is generated based on the standard color chart image; the image obtained in step two after suppressing moiré interference is corrected based on the filter file to obtain a corrected image; the specific process is as follows:

[0013] Step 31: Cropped standard color chart image from the image obtained in Step 2 after suppressing moiré interference; Step 32: Imported the cropped standard color chart image into the 3D LUT Creator Pro software platform, and enabled the "Color Chart Network" module in the 3D LUT Creator Pro software platform to generate a standard color chart grid; Step 33: The user manually aligned the standard color chart grid to the position of the cropped standard color chart, ensuring that the coordinates of the cropped standard color chart are accurately matched; Step 34: In the 3D LUT Creator Pro software platform, selected "Curves + 3D LUT", the software will automatically calculate the color transformation relationship based on the aligned cropped standard color chart, completing the color correction of the cropped standard color chart image; Step 35: Exported a .cube file (matrix) from the 3D LUT Creator Pro software platform; Step 36: Imported the .cube file into the Camera Raw module of Adobe Photoshop, the Camera Raw module generates a .xmp custom configuration file, and exported the .xmp configuration file from Adobe Photoshop; Step 37: Imported the .xmp configuration file into Adobe Lightroom, Adobe... Lightroom generates a filter file; Step 38: Import the image obtained in Step 2 after suppressing moiré interference into Adobe Lightroom. The filter file corrects the image obtained in Step 2 after suppressing moiré interference (including the images of the standard color chart and the car's central control screen) to obtain the corrected image (including the images of the standard color chart and the car's central control screen).

[0014] The other steps and parameters are the same as those in one of the specific implementation methods one to three.

[0015] Specific Implementation Method Five: This implementation method differs from Specific Implementation Methods One to Four in that: in step four, the corrected image is evaluated for its effectiveness; the specific process is as follows:

[0016] The evaluation methods include two categories: color card correction effect evaluation and image correction effect evaluation;

[0017] 41. Evaluation of Color Card Correction Effect: Input the image after correction in step three into Imatest software. Use the Colorcheck function of Imatest software to manually match the standard color card image area in the image after correction in step three (including the standard color card and the image of the car's central control screen) to generate a color difference evaluation report and color distribution analysis results.

[0018] The color difference evaluation report includes:

[0019] 1) Color dots ( Figure 6 The colors of the circles and squares in (a) and the color space ( Figure 6 (The background color in (a));

[0020] 2) Each color patch (the standard color chart includes 24 color patches) corresponds to a square and a circle mark. The square represents the reference color of the standard color chart (the true value of the color chart color), and the circle represents the color of the standard color chart in the image after correction in step three of the Imatest software (the color value of the standard color chart in the image input to the Imatest software).

[0021] 3) Average chroma percentage; Average chroma percentage is defined as the ratio of the average chroma of the standard color chart in the corrected image input into Imatest software to the average chroma of the reference color of the standard color chart, and is used to evaluate the saturation reproduction level.

[0022] The evaluation criteria are as follows:

[0023] Average chroma percentage < 100%: The color is grayish and undersaturated; Average chroma percentage = 100%: The reference color with saturation closest to the standard color chart; 100% < Average chroma percentage ≤ 120%: The color is vibrant and appears brighter to the human eye.

[0024] Average chroma percentage > 120%: indicates oversaturation, which may result in loss of detail, unnatural colors, and visual discomfort;

[0025] 42. Image correction effect evaluation. Other steps and parameters are the same as those in specific implementation methods one to four.

[0026] Specific Implementation Method Six: This implementation method differs from Specific Implementation Methods One through Five in that the image correction effect evaluation in section four-two is as follows:

[0027] To accurately assess the color difference between the image corrected in step three and the real image (without photographic processing) on ​​the car's central control screen, an improved feature registration algorithm based on pre-alignment and region filtering is constructed. This algorithm uses feature points for coarse registration and combines a quality threshold for evaluation to ensure the reliability of the registration results. Subsequently, enhanced correlation coefficient (ECC) optimization is introduced to achieve sub-pixel-level global fine alignment, thereby improving the accuracy of subsequent color difference calculations. Simply put, after importing the real image (without photographic processing) from the car's central control screen and the corresponding image corrected in step three, geometric registration is automatically completed, and the following analysis results are output: effective area overlay display (used to verify the effective registration range), and statistical analysis results (such as average ΔE, etc.).

[0028] 421. Assume the actual image on the car's central control screen (without any image processing) is... , Let the image after correction in step three be... , ; A real image of the car's center console screen Spatial coordinates of the middle pixel The set of real numbers is 3, where 3 represents the three color channels RGB. , The height of the actual image on the car's central control screen. The width of the actual image on the car's central control screen; Image after correction in step three Spatial coordinates of the middle pixel , The height of the image after correction in step three. The width of the image after correction in step three; 422. Based on the real image of the car's central control screen. and the image after correction in step three Obtain the pre-aligned corrected image 423. Display the real image on the car's central control screen. and the pre-aligned corrected image The region is obtained by meshing. For the region A comprehensive quality score is obtained through quality assessment. Based on comprehensive quality score Perform effective region filtering to obtain effective regions that meet the requirements. 424. Within the effective area Perform fine-grained local registration to obtain a locally captured image after fine-grained local registration. 425. Realistic images based on the central control screen The local images obtained after fine registration with 424 Calculate the color difference. Other steps and parameters are the same as in any of the specific implementation methods one to five.

[0029] Specific Implementation Method Seven: This implementation method differs from Specific Implementation Methods One through Six in that: the image in 422 is based on the real image of the car's central control screen. and the image after correction in step three Obtain the pre-aligned corrected image The specific process is as follows:

[0030] 4221. Realistic images based on car center console screens Candidate feature points are obtained; based on the image after correction in step three. Candidate feature points are obtained; the specific process is as follows:

[0031] SIFT (Scale Invariant Feature Transform) feature points are key locations in an image that are salient and reproducible. Feature point extraction is based on scale-space theory and detects local extrema by constructing a Gaussian difference pyramid.

[0032] 1) Based on real images from the car's central control screen Construct scale space functions;

[0033] The scale space function is expressed as: ;in, For position ,scale The scale space representation below; It is a two-dimensional Gaussian kernel function. ; For scaling parameters (Gaussian kernel standard deviation); This is the convolution operator;

[0034] 2) Calculate the Difference of Gaussians (DOG) extrema based on scale space functions;

[0035] The extreme values ​​of the Gaussian difference are expressed as: In the formula, These are the extreme values ​​of the Gaussian difference. The factor is the multiple between adjacent scales, taken as... Each scale layer typically has 3 layers; the DOG function is used to approximate the scale-normalized Laplacian operator, searching for extreme points in this space as candidate feature points.

[0036] 3) Based on Gaussian difference extrema Determine candidate feature points; the specific process is as follows: If If the following conditions are met, then Considered an extreme point As candidate feature points; if If the following conditions are not met, then It is not an extreme point. Not a candidate feature point;

[0037]

[0038] in, A real image of the car's center console screen any position in the space of a medium pixel and arbitrary scale ; express The 26 neighboring points (8 neighboring points of the current scale layer + 9 points of the previous scale layer + 9 points of the next scale layer); express Gaussian difference extrema; based on the image after correction in step three The specific process for obtaining candidate feature points is the same as 1), 2), and 3); change the superscript 1 to 2.

[0039] 4222. Realistic images based on car center console screens and the image after correction in step three The Scale Invariant Feature Transform (SIFT) algorithm was used to extract the real images of the car's central control screen. feature point set and the image after correction in step three feature point set The specific process is as follows:

[0040] ;

[0041] in, This represents a real image of a car's central control screen extracted using the Scale Invariant Feature Transform (SIFT) algorithm. The set of feature points in; This represents the image after correction in step three, extracted using the Scale Invariant Feature Transform (SIFT) algorithm. The set of feature points in; Represents the actual image on the car's central control screen. The total number of feature points in the data is a non-negative integer. This indicates the image after correction in step three. The total number of feature points in the data is a non-negative integer. Represents the actual image on the car's central control screen. The position parameters of the feature points in the data. ; A real image of the car's center console screen Extracted from the first The position coordinates of each feature point in the image coordinate system; A real image of the car's center console screen Extracted from the first The feature scale of each feature point represents Scale size; A real image of the car's center console screen The The principal direction angle of each feature point is determined by the peak value of the gradient direction histogram; the coordinate system is Cartesian coordinates.

[0042]

[0043] In the formula, Indicates the gradient magnitude; Indicates position Scale-space representation; Indicates position Scale-space representation; Indicates position Scale-space representation; Indicates position Scale-space representation; Represents the neighborhood window of feature points; Indicates the gradient direction; Represents the actual image on the car's central control screen. Extracted from the first Descriptor vectors of feature points It is a 128-dimensional real vector, obtained by calculating the 8-directional gradient histogram of 4×4 sub-regions within a 16×16 neighborhood around the feature point; here and It is the index number of the feature point. Represents the actual image on the car's central control screen. The Middle One feature point, This indicates the image after correction in step three. The Middle One feature point; The acquisition process is as follows:

[0044] Selecting a neighborhood: Select a 16×16 pixel neighborhood window around the feature point;

[0045] Divide the neighborhood into sub-regions: Divide the 16×16 = 256 pixel neighborhood into 4×4 = 16 sub-regions, each containing 4×4 pixels;

[0046] Calculate the gradient histogram: Quantize the gradient direction into 8 directional intervals: [0°, 45°), [45°, 90°), ..., [315°, 360°). Calculate the cumulative sum of the gradient magnitudes within each directional interval to form an 8-dimensional histogram.

[0047] Combined into a descriptor: 16 sub-regions × 8 directions = 128-dimensional vector, i.e. ;

[0048] Represents the actual image on the car's central control screen. The Middle The dimension of the first direction of the descriptor vector of each feature point in the first sub-region. Represents the actual image on the car's central control screen. The Middle The dimension of the second direction of the descriptor vector of each feature point in the first sub-region. Represents the actual image on the car's central control screen. The Middle The dimension of the 8th direction of the descriptor vector of each feature point in the first sub-region; Represents the actual image on the car's central control screen. The Middle The dimension of the first direction of the second sub-region in the descriptor vector of each feature point; Represents the actual image on the car's central control screen. The Middle The dimension of the 8th direction of the 16th sub-region of the descriptor vector of each feature point; superscript This indicates a transpose; a real image on a car's central control screen. The Middle The descriptor vector for each feature point has 128 dimensions; This indicates the image after correction in step three. The position parameters of the feature points in the data. ; Image after correction in step three Extracted from the first The position coordinates of each feature point in the image coordinate system; Image after correction in step three Extracted from the first The feature scale of each feature point represents Scale size; Image after correction in step three The The principal direction angle of each feature point is determined by the peak value of the gradient direction histogram; the coordinate system is Cartesian coordinates.

[0049]

[0050] In the formula, Indicates the gradient magnitude; Indicates position Scale-space representation; Indicates position Scale-space representation; Indicates position Scale-space representation; Indicates position Scale-space representation; Indicates the gradient direction; Represents the neighborhood window of feature points; This indicates the image after correction in step three. Extracted from the first Descriptor vectors of feature points It is a 128-dimensional real vector, obtained by calculating the 8-directional gradient histogram of 4×4 sub-regions in a 16×16 neighborhood around the feature point; The acquisition process is as follows:

[0051] Selecting a neighborhood: Select a 16×16 pixel neighborhood window around the feature point;

[0052] Divide the neighborhood into sub-regions: Divide the 16×16 = 256 pixel neighborhood into 4×4 = 16 sub-regions, each containing 4×4 pixels;

[0053] Calculate the gradient histogram: Quantize the gradient direction into 8 directional intervals: [0°, 45°), [45°, 90°), ..., [315°, 360°). Calculate the cumulative sum of the gradient magnitudes within each directional interval to form an 8-dimensional histogram.

[0054] Combined into a descriptor: 16 sub-regions × 8 directions = 128-dimensional vector, i.e. ;

[0055] This indicates the image after correction in step three. The Middle The dimension of the first direction of the descriptor vector of each feature point in the first sub-region. This indicates the image after correction in step three. The Middle The dimension of the second direction of the descriptor vector of each feature point in the first sub-region. This indicates the image after correction in step three. The Middle The dimension of the 8th direction of the descriptor vector of each feature point in the first sub-region; This indicates the image after correction in step three. The Middle The dimension of the first direction of the second sub-region in the descriptor vector of each feature point; This indicates the image after correction in step three. The Middle The dimension of the 8th direction of the 16th sub-region of the descriptor vector of each feature point; superscript This indicates the transpose;

[0056] Step 3 Corrected image The Middle The descriptor vector for each feature point has 128 dimensions;

[0057] 4223. Realistic images based on car center console screens Extracted from the first Descriptor vector of each feature point and the image after correction in step three Extracted from the first Descriptor vector of each feature point Feature matching was performed using the FLANN (Fast Nearest Neighbor) matcher to obtain the real images of the car's central control screen that met the matching conditions. Feature points in and the image after correction in step three Feature points in FLANN stands for Fast Nearest Neighbor Approximation; the specific process is as follows:

[0058] 1) Define the Euclidean distance function:

[0059] In the formula, This represents the L2 norm (Euclidean norm). Represents the actual image on the car's central control screen. Extracted from the first The k-th dimension of the descriptor for each feature point; This indicates the image after correction in step three. Extracted from the first The k-th dimension of the descriptor for each feature point; express and The Euclidean distance between them;

[0060] 2) Obtain matching values ​​based on Euclidean distance Select real images from the car's central control screen that meet the matching criteria. Feature points in and the image after correction in step three Feature points in ; indicates as:

[0061]

[0062] In the formula, This represents a real image of the car's central control screen that meets the matching criteria. Feature points in and the image after correction in step three Feature points in Feature point matching pairs; Represents the actual image on the car's central control screen. All dimensions of the descriptor of the extracted i-th feature point are compared with the image after correction in step three. Extracted from the first The Euclidean distance between all dimensions of the descriptors of a feature point (the feature point with the nearest neighbor distance); Represents the actual image on the car's central control screen. All dimensions of the descriptor of the extracted i-th feature point are compared with the image after correction in step three. Extracted from the first Feature points (excluding) The Euclidean distance between all dimensions of the descriptor of the feature points (the second nearest neighbor distance outside the feature points); This represents the ratio of the nearest neighbor distance to the second nearest neighbor distance. When the ratio is small, it indicates that the nearest neighbor matching point is significantly better than the second nearest neighbor matching point, and the matching result is unique and can be considered a reliable and correct match. When the ratio is large, it indicates that the matching quality of the nearest neighbor and the second nearest neighbor is comparable, and it is impossible to determine which is the true corresponding point. There is matching ambiguity, and it should be excluded. Indicates the ratio threshold. ; The index of the nearest matching point, i.e. argmin returns the value of the argument that makes the function minimum, i.e., it represents the value that makes the function minimum. Take the minimum value The value; The index of the second nearest matching point, i.e. argmin returns the value of the argument that makes the function minimum, i.e., it represents the value that makes the function minimum. Take the next smallest value The value;

[0063] 4224, based on Feature points in and feature points To obtain the optimal homography matrix The specific process is as follows:

[0064] 1) The homography matrix describes the mapping relationship between two planes;

[0065] Estimating the homography matrix using the RANSAC (Random Sample Consistency) algorithm The specific process is as follows:

[0066] Assume a real image on the car's central control screen. The Middle The coordinates of the feature points are Step 3: Corrected image The Middle The coordinates of the feature points are Matching point pairs The correspondence is determined by the homography matrix. describe( unknown);

[0067]

[0068] After unfolding, we get: ;

[0069] 2) Based on homography matrix Construct the objective function for optimizing the homography matrix to obtain the optimal homography matrix. :

[0070]

[0071] in, Huber robust loss function

[0072] In the formula, This is a threshold parameter, typically set to 1.35; express ; Represents the L2 norm; , , , , , , , , Representing the homography matrix Elements in;

[0073] 4225. Determining the Optimal Homography Matrix Does the image simultaneously satisfy determinant constraint, condition number constraint, and perspective component constraint? If yes, proceed to step 4226; otherwise, proceed to step 3 after correction. Unqualified; the specific process is as follows:

[0074] Determinant constraints: Condition number constraints: Perspective component constraints: ;in, Indicates determinant calculation; Represents the optimal homography matrix Condition number constraints; Represents the optimal homography matrix Determinant constraints; and They are matrices The maximum and minimum singular values; the larger the condition number, the closer the matrix is ​​to ill-conditioned, and the less stable the numerical computation. For homography matrices... Its singular value decomposition (SVD) form is: ;in It is an orthogonal matrix. diagonal elements For matrix The singular values ​​are sorted in descending order; , This represents the perspective component; an excessively large value indicates severe perspective distortion.

[0075] 4226. Based on the optimal homography matrix Spatial coordinates of pixels in the image after step three correction Perform the transformation to obtain the pre-aligned corrected image. ; indicates as:

[0076]

[0077] in, express That is, homography matrix Inverse matrix aligned with secondary coordinate vector Perform matrix multiplication to obtain a homogeneous coordinate matrix ,right After performing a normalization operation (dividing the first two components by the third component), the corrected image is obtained. Corresponding source coordinates in ; This represents matrix multiplication, with the superscript T indicating transpose. , Represents the scaled coordinate components after transformation; This represents the scaling factor used to eliminate scale ambiguity; Indicates the correction image The pre-aligned image is obtained by pixel sampling of the coordinate positions after inverse homography transformation. In coordinates The pixel value at that location.

[0078] The other steps and parameters are the same as those in one of the specific implementation methods one to six.

[0079] Specific Implementation Method Eight: This implementation method differs from Specific Implementation Methods One through Seven in that: in section 423, the actual image of the car's central control screen is used. and the pre-aligned corrected image The region is obtained by meshing. For the region A comprehensive quality score is obtained through quality assessment. Based on comprehensive quality score Perform effective region filtering to obtain effective regions that meet the requirements. The specific process is as follows:

[0080] 4231. Display the real image on the car's central control screen. Mesh generation is performed; the specific process is as follows:

[0081] Real image from the car's center console screen Divided into The grid represents the actual image on the car's central control screen. The Line 1 Column area Represented as:

[0082]

[0083] in, For grid height, ; For grid width, ; The height of the actual image on the car's central control screen. The width of the actual image on the car's central control screen; For grid indexing; and These represent the number of grid divisions for width and height, respectively.

[0084] 4232. The pre-aligned corrected image... Mesh generation is performed; the specific process is as follows:

[0085] Pre-aligned image Divided into The grid, then the pre-aligned image The Okay, number Column area Represented as:

[0086]

[0087] in, For grid height, ; For grid width, ; The height of the pre-aligned corrected image. The width of the pre-aligned corrected image; and These represent the number of grid divisions for width and height, respectively. For grid indexing;

[0088] 4233, Regarding the region A quality assessment is conducted to obtain a comprehensive quality score. The specific process is as follows:

[0089] For the region Calculate the overall quality score:

[0090]

[0091] The weights satisfy the normalization condition , ;

[0092] The The acquisition process is as follows:

[0093] 1) Calculate structural similarity index The specific process is as follows:

[0094] The structural similarity index has a value range of [-1, 1], and the closer the value is to 1, the more similar the similarity is.

[0095]

[0096] In the formula, , This represents the mean; , Indicates variance; Represents covariance; It is the stability constant. 255 represents the grayscale range; Indicates the region Structural similarity index (SSIM);

[0097] mean and The calculation formula is:

[0098] ;

[0099] variance and The calculation formula is:

[0100] ;

[0101] covariance The calculation formula is:

[0102]

[0103] in, express and The total number of pixels in the m-th row and n-th column region; Representing coordinates The corresponding real image of the car's central control screen; Representing coordinates The corresponding pre-aligned corrected image; Reference Interior coordinates , express and The shared m-th row and n-th column region reflects the spatial alignment relationship;

[0104] 2) Calculate normalized cross-correlation (NCC) The specific process is as follows:

[0105] Normalized cross-correlation The similarity used to evaluate grayscale patterns is represented as:

[0106]

[0107] in, For the real image of the car's central control screen area The grayscale value at that location; For the pre-aligned corrected image in area The grayscale value at that location; For the real image of the car's central control screen area The grayscale value at that location; For the pre-aligned corrected image in area The grayscale value at that location; For the real image of the car's central control screen The average gray value of the region; For the pre-aligned corrected image in The average gray value of the region;

[0108] 3) Calculate feature matching quality The specific process is as follows:

[0109]

[0110] in, ,express The set of feature points within the region of row m and column n; express The set of feature points within the region of row m and column n; For the region The set of matching feature points within; The cardinality (number of elements) of a set;

[0111] 4) Calculate marginal correlation; the specific process is as follows:

[0112] Edge correlation is used to evaluate the consistency of edge structure between two images; for real images of car center console screens and the pre-aligned corrected image Canny edge detection was performed separately to obtain binary edge maps. and Where edge points have a value of 1, and non-edge points have a value of 0; region ( The marginal relevance index (representing the common region in row m and column n) is defined as:

[0113]

[0114] Covariance calculation:

[0115]

[0116] in, and For the real image and the pre-aligned corrected image area The Canny edge detection result at the coordinates, with a value of 0 or 1; express and covariance; Represents edge density, i.e. In the region The mean, ; Represents edge density, i.e. In the region The mean, ;

[0117] 4234. Based on the overall quality score Perform effective region filtering to obtain effective regions that meet the requirements. The specific process is as follows:

[0118]

[0119] in, For quality threshold, Output area ( , representing the overall quality score of the common region (m-th row and n-th column). and based on threshold (Experience points) are used to filter valid regions.

[0120] The other steps and parameters are the same as those in any of the specific implementation methods one to seven.

[0121] Specific Implementation Method Nine: This implementation method differs from Specific Implementation Methods One through Eight in that: in the effective area described in section 424... Perform fine-grained local registration to obtain a locally captured image after fine-grained local registration. The specific process is as follows:

[0122] 4241. Construct a local affine transformation model; the specific process is as follows:

[0123] Estimate the local affine transformation matrix for the effective region:

[0124] in, Represents the local affine transformation matrix; By scaling factor Rotation angle Shear parameters Together, they determine the scaling, rotation, and shearing transformations in the Cartesian coordinate system. Comprehensive coefficient of direction; Due to scaling factor Rotation angle Shear parameters Together, they determine that the "scaling + rotation + shearing" transformations in the Cartesian coordinate system... Directional component pairs Influence coefficient of directional component; Due to scaling factor Rotation angle Shear parameters Together, they determine the scaling, rotation, and shearing transformations in the Cartesian coordinate system. Directional component pairs Influence coefficient of directional component; By scaling factor Rotation angle Shear parameters Together, they determine that the "scaling + rotation + shearing" transformations in the Cartesian coordinate system... Comprehensive coefficient of direction; Represents the Cartesian coordinate system Translation amount in direction, Represents the Cartesian coordinate system The amount of translation in the direction;

[0125] The geometric meaning of affine transformations includes rotation, scaling, shearing, and translation:

[0126]

[0127] in, This is the scaling factor; The rotation angle; , For shearing parameters; , This is the translation amount;

[0128] All parameters of quantity 4241 have known initial values;

[0129] 4242. Based on Local Affine Transformation Matrices Obtain the optimal transformation parameters The specific process is as follows:

[0130] For the region Finding the optimal transformation parameters Maximize the enhanced correlation coefficient (ECC):

[0131]

[0132] in, Indicates the region Inside, real images and The enhanced correlation coefficient between the two is used to measure their similarity. Represents a real image In the region Local subgraphs and transformed images In the region Local subgraphs within; Indicates the correction image After local affine transformation The corrected image is obtained by pixel sampling at the coordinate position after the coordinates. In coordinates Pixel value at; express That is, the local affine transformation matrix Align secondary coordinate vector Perform matrix multiplication to obtain a homogeneous coordinate matrix ,right After performing a normalization operation (dividing the first two components by the third component), the corrected image is obtained. Corresponding source coordinates in ; This represents matrix multiplication, with the superscript T indicating transpose. , The first two components of the homogeneous coordinate matrix after the local affine transformation are represented, corresponding to the scaled abscissa component and the scaled ordinate component after the transformation. The third component of the homogeneous coordinate matrix is ​​used as a scaling factor to eliminate scale ambiguity. Represents the affine transformation model; the enhanced correlation coefficient (ECC) is defined as:

[0133]

[0134] in, Represents a zero-mean image. ; Represents a real image In the region The mean within, ; Represents a zero-mean image;

[0135] ;

[0136] express In the region The mean within;

[0137] ;

[0138] Affine transformation model Represents the actual image from the car's central control screen. coordinates To the pre-aligned corrected image coordinates The transformation; expressed as:

[0139]

[0140] Among them, the part before the semicolon Input coordinates, ;

[0141] After the semicolon For the affine transformation parameters (to be determined), the superscript... This indicates the transpose;

[0142] 4243. Using the Gauss-Newton method to adjust affine transformation parameters Perform iterative solutions to obtain the optimal affine transformation parameters. The obtained optimal affine transformation parameters Substitute into the affine transformation model To obtain the optimal transformation parameters The specific process is as follows:

[0143] 1) Let the number of iterations be... Setting parameters The initial value;

[0144] 2) Maximizing ECC is equivalent to minimizing the following error function:

[0145]

[0146] in, Represents a zero-mean image; Represents a zero-mean image. Indicates the first The error function of the next iteration;

[0147] 3) Based on The region within Pixels are stacked one by one to construct the first pixel. Error vector of the next iteration , ; indicates as:

[0148]

[0149] in, The zero-mean image representing the true image RGB coordinates of the point Representing an image After affine transformation model In the zero-mean image after RGB coordinates of the point; express That is, the local affine transformation matrix Align secondary coordinate vector Perform matrix multiplication to obtain a homogeneous coordinate matrix ,right After performing a normalization operation (dividing the first two components by the third component), the corrected image is obtained. Corresponding source coordinates in ; This represents matrix multiplication, with the superscript T indicating transpose. , The first two components of the homogeneous coordinate matrix after the local affine transformation are represented, corresponding to the scaled abscissa component and the scaled ordinate component after the transformation. The third component of the homogeneous coordinate matrix is ​​used as a scaling factor to eliminate scale ambiguity. Represents the affine transformation model;

[0150] The zero-mean image representing the true image RGB coordinates of the point Representing an image After affine transformation model In the zero-mean image after RGB coordinates (acquisition process and) same);

[0151] express ; express ;

[0152] The zero-mean image representing the true image RGB coordinates of the point Representing an image After affine transformation model In the zero-mean image after RGB coordinates (acquisition process and) same); Representing the error vector, the region It is formed by stacking the errors of all pixels point by point; For the region Inner pixel coordinates , For the region Total number of inner pixels;

[0153] 4) Obtain the first Error vector of the next iteration For parameters The partial derivatives are expressed as:

[0154]

[0155] in, This represents the partial derivative of the error vector with respect to the parameters; Indicated in pixels At, affine transformation matrix For parameters The partial derivatives of , corresponding to the partial derivatives of in the local affine transformation. Related transform components; Indicated in pixels At, affine transformation matrix For parameters The partial derivatives of , corresponding to the partial derivatives of in the local affine transformation. Related transform components; Indicated in pixels At, affine transformation matrix For parameters The partial derivatives; Indicated in pixels At, affine transformation matrix For parameters The partial derivatives; For image gradient, , Represents the x-coordinate in a Cartesian coordinate system. Represents the ordinate in a Cartesian coordinate system, superscript This indicates the transpose;

[0156] Local affine transformation matrix The partial derivatives are:

[0157]

[0158] 5) Let the number of iterations be... Update parameters Repeat steps 2)-5) until the convergence condition is met to obtain the optimal parameters. ;

[0159] Parameter increment ;in, This is the step size factor, used to control the convergence speed and stability;

[0160] The obtained optimal affine transformation parameters Substitute into the affine transformation model To obtain the optimal transformation parameters ;

[0161] The convergence condition is: the process terminates when any of the following conditions are met:

[0162] (1) The parameter increment is small enough: (like ), For the parameter convergence threshold, This represents the L2 norm. When the norm of the parameter increment is less than this threshold, it indicates that the parameter has essentially stopped changing, and the iteration can be terminated.

[0163] (2) The error variation is small enough: (like ), The error convergence threshold is defined as the threshold at which the absolute value of the error change between two consecutive iterations is less than the error convergence threshold. When the error has basically converged, the iteration can be terminated.

[0164] (3) Reaching the maximum number of iterations: (like ), This represents the maximum number of iterations.

[0165] 4244, Optimal Transformation Parameters Based on 4243 Obtain a finely registered local image ; indicates as:

[0166]

[0167] Output exist The effective area visualization results are displayed, and green is used as the overlay of the effective area.

[0168] Indicates the correction image After local affine transformation The corrected image is obtained by pixel sampling at the coordinate position after the coordinates. In coordinates Pixel value at;

[0169] express That is, the optimal transformation matrix Inverse matrix aligned with secondary coordinate vector Perform matrix multiplication to obtain a homogeneous coordinate matrix ,right After performing a normalization operation (dividing the first two components by the third component), the corrected image is obtained. Corresponding source coordinates in ; This represents matrix multiplication, with the superscript T indicating transpose. , These represent the first two components of the homogeneous coordinate matrix after the optimal transformation, corresponding to the scaled abscissa component and the scaled ordinate component after the transformation. The third component of the homogeneous coordinate matrix is ​​used as a scaling factor to eliminate scale ambiguity. This represents the affine transformation model. Other steps and parameters are the same as in specific implementation methods one through eight.

[0170] Specific Implementation Method Ten: This implementation method differs from Specific Implementation Methods One through Nine in that: in step four, two, and five, the image is based on the real image of the central control screen. The finely registered local images obtained from 4244 Calculate the color difference; the specific process is as follows:

[0171] The real image of the central control screen The finely registered local images obtained from 4244 After converting the RGB values ​​to the CIE Lab color space, based on the image... and images The corresponding pixel in the CIE Lab color space , , Components, for the effective region CIE calculation for each pixel within Color difference ; Represents the luminance component, with a value range of 0 (black) to 100 (white), describing the lightness or darkness of a color; Represents the red-green components, with a value range of −128 (green) to +127 (red), describing the color offset on the red-green axis; Represents the yellow-blue component, with values ​​ranging from -128 (blue) to +127 (yellow), describing the color offset on the yellow-blue axis; it represents the color difference. The numerical values ​​are mapped to pseudo-color (such as Jet color chart), and the heat map is output based on pseudo-color. The color difference range of the heat map is 0-100.

[0172] Based on color difference and analysis mask Calculate the effective region Average color difference ; indicates as:

[0173]

[0174] Analysis mask The acquisition process is as follows:

[0175] Based on the grid region that meets the requirements Constructing the analysis mask:

[0176] symbol This represents the union operation of sets, i.e., merging all valid regions. Mask The location indicates the area that needs to be optimized or removed (removed areas will be displayed in black on the heatmap and will not be included in the calculation of average color difference).

[0177] in, For effective area Inner pixel set; This represents the total number of effective pixels. Other steps and parameters are the same as in any of the specific implementation methods one through nine.

[0178] The beneficial effects of the present invention are verified using the following embodiments:

[0179] Example 1: This example uses a Redmi K70 Ultra smartphone as the shooting device to capture images of a 23.8-inch, 1920×1080 resolution, center console-like display, verifying the effectiveness of the proposed "a method for color reproduction and evaluation of images captured from a car center console screen." During the verification process, a smartphone equipped with an adjustable polarizing filter was first used to photograph the display screen. The displayed content included a standard color chart image and the target image of the center console screen to be calibrated. The shooting results are as follows... Figure 2a , 2b As shown in the image. Due to the obvious moiré pattern interference in the image, a pre-trained ESDNet model is used to suppress the moiré pattern, and the effect is as follows. Figure 3 As shown. Subsequently, the image with suppressed moiré patterns was imported into 3D LUTCreator Pro software. Color mapping and color correction were achieved by manually matching the color chart grid. The correction process is as follows: Figure 4 As shown. After completing the color mapping relationship construction, the generated LUT configuration file is exported and loaded into Adobe Photoshop and Lightroom to perform batch color restoration processing on the entire image set. A comparison of the original image effect and the correction effect is shown below. Figure 5a , 5b As shown, the color correction process is now complete. To evaluate the color reproduction effect, the Colorcheck function of Imatest software was used to perform color difference analysis on the corrected color chart area. The results are as follows: Figure 6 As shown. Specific indicators include: chroma percentage of 103.8%, close to the reference value of 100%, indicating good overall saturation recovery; average... The color difference is approximately 1.9, which is relatively small and meets the standard of no obvious color cast. Furthermore, to verify the overall correction effect of the image set, an improved feature registration algorithm based on pre-alignment and region filtering was used to compare and analyze the captured images and the corrected images. The corresponding color difference heatmap (including effective region registration) is shown below. Figure 7a , 7b As shown. The results indicate that there are areas of registration failure in the captured images (i.e., Figure 7a The black shadow area in the image is considered an ineffective area, with an average color difference of approximately 9.16. In contrast, the corrected image shows effective color reproduction across all areas, with high feature matching accuracy and a color difference of only 6.09, representing an optimization of nearly 33% compared to the captured image. This further validates the effectiveness of this method in overall image restoration. In summary, this embodiment demonstrates that the method proposed in this invention exhibits excellent color restoration performance in actual central control screen image acquisition, effectively suppressing moiré interference and accurately restoring the true color appearance of the image, demonstrating good practicality and scalability.

[0180] This invention may have other embodiments. Without departing from the spirit and essence of this invention, those skilled in the art can make various corresponding changes and modifications according to this invention, but these corresponding changes and modifications should all fall within the protection scope of the appended claims.

Claims

1. A color restoration and evaluation method for images captured by a center screen in an automobile, characterized by: The method specifically comprises the following steps: Step one, collecting an image dataset containing a standard color card and a car center screen; Step two, performing Moiré interference suppression on the collected image containing the standard color card and the car center screen to obtain an image after Moiré interference suppression; Step three, cropping the standard color card image from the image after Moiré interference suppression obtained in step two, and generating a filter file based on the standard color card image; Based on the filter file, correcting the image after Moiré interference suppression obtained in step two to obtain a corrected image; Step four, performing effect evaluation on the corrected image.

2. The color restoration and evaluation method for the image captured by the center screen of the vehicle according to claim 1, characterized in that: In step one, the image dataset containing the standard color card and the car center screen is collected, and the specific process is as follows: The standard color card image is placed on the left side of the car center screen; A shooting device equipped with a polarizer is used to shoot the standard color card image and the car center screen with different interfaces to obtain an image dataset; Each image in the image dataset contains a standard color card and a car center screen.

3. The color restoration and evaluation method for the image captured by the center screen of the vehicle according to claim 2, characterized in that: In step two, the collected image containing the standard color card and the car center screen is subjected to Moiré interference suppression to obtain an image after Moiré interference suppression, and the specific process is as follows: The collected image containing the standard color card and the car center screen is input into a pre-trained neural network model ESDNet, and the pre-trained neural network model ESDNet outputs an image after Moiré interference suppression.

4. The color restoration and evaluation method for the image captured by the center screen of the vehicle according to claim 3, characterized in that: In step three, the standard color card image is cropped from the image after Moiré interference suppression obtained in step two, and a filter file is generated based on the standard color card image; Based on the filter file, the image after Moiré interference suppression obtained in step two is corrected to obtain a corrected image; The specific process is as follows: Step three one, cropping the standard color card image from the image after Moiré interference suppression obtained in step two; Step three two, importing the cropped standard color card image into a 3D LUT Creator Pro software platform, enabling a "color card network" module in the 3D LUT Creator Pro software platform, and generating a standard color card grid; Step three three, manually aligning the standard color card grid to the position of the cropped standard color card by a user; Step three four, selecting "curve + 3D LUT" in the 3D LUT Creator Pro software platform to complete color correction of the cropped standard color card image; Step three five, exporting a.cube file in the 3D LUT Creator Pro software platform; Step three six, importing the.cube file into a Camera Raw module of Adobe Photoshop, and the Camera Raw module generates an.xmp custom configuration file, and the.xmp configuration file is exported in Adobe Photoshop; Step three seven, importing the.xmp configuration file into Adobe Lightroom, and Adobe Lightroom generates a filter file; Step three eight, importing the image after Moiré interference suppression obtained in step two into Adobe Lightroom, and the filter file corrects the image after Moiré interference suppression obtained in step two to obtain a corrected image.

5. The color restoration and evaluation method for the image captured by the center screen of the vehicle according to claim 4, characterized in that: The step four is effect evaluation on the corrected image; the specific process is: The evaluation method includes two types: color card correction effect evaluation and image correction effect evaluation; Four one, color card correction effect evaluation: input the corrected image in step three into Imatest software, manually match the standard color card image area in the corrected image in step three through the Colorcheck function of Imatest software, and generate a color difference evaluation report and color distribution analysis results; The color difference evaluation report includes: 1), color points and color space; 2), each color block corresponds to a square and a circular mark, the square represents the reference color of the standard color card, and the circle represents the color of the standard color card in the corrected image input into the Imatest software in step three; 3), average chroma percentage; The average chroma percentage is defined as the ratio of the average chroma of the color of the standard color card in the corrected image input into the Imatest software to the average chroma of the reference color of the standard color card; Four two, image correction effect evaluation.

6. The color restoration and evaluation method for the image captured by the center screen of the vehicle according to claim 5, characterized in that: The image correction effect evaluation in the four two; the specific process is: Four two one, the real image of the car central control screen is , ; Let the corrected image after step three be , ; real image for a car center screen spatial coordinates of the middle pixel, is a real set, 3 indicates the RGB three color channels; , height of the real image for a car center screen, width of the real image for a car center screen; corrected image for step three spatial coordinates of the center pixel, , height of the corrected image for step three, width of the corrected image for step three; Forty-two, real image based on car central screen and step three corrects the image , obtain pre-aligned corrected image ; Four two three, the real image of the car central screen And the pre-aligned correction image Carry out meshing to obtain a region , the region Quality assessment to obtain a comprehensive quality score ; Based on the comprehensive quality score Effective area screening is carried out, and the effective area meeting the requirements is obtained ; Four two four, in the effective area Local fine registration is performed in the effective area ; Four twenty-five, real image based on a control screen And four twenty-four obtained after local fine registration of the local captured image , calculate the color difference.

7. The color restoration and evaluation method for the image captured by the center screen of the vehicle according to claim 6, characterized in that: The four two two based on the real image of car center screen And step three corrects the image , Obtain the pre-aligned corrected image ; The specific process is: Forty-two one, based on the real image of car central screen , obtain candidate feature points; image based on step three obtaining candidate feature points; The specific process is: 1) Real image based on car central screen , construct scale space function; The scale space function is represented as: ; wherein is a position , scale under scale space representation; is a two-dimensional Gaussian kernel function, ; is a scale parameter; is a convolution operator; 2), calculate the Gaussian difference extreme value based on the scale space function; The difference of Gaussians extreme value is expressed as: In the formula, is a Gaussian difference extremum; is a factor of the adjacent scale, taken as ; 3) Based on Gaussian difference extremum determining candidate feature points; the specific process is as follows: If the following condition is met, is considered an extreme point, as a candidate feature point; if the following condition is not met, is not an extreme point, is not a candidate feature point; wherein, real image for car center screen any position in the space of the middle pixel and any scale ; denotes the 26-neighborhood points of denotes the Gauss-difference extrema of image based on step three , the specific process of obtaining candidate feature points is the same as 1), 2), and 3); 4222, real image based on car central screen and step three corrected image , using scale invariant feature transform (SIFT) algorithm to extract feature point set of real image of car central screen and step three corrected image and step three corrected image of feature point set ; the specific process is: Wherein, represents a set of feature points extracted from a real image of a car center screen using a scale-invariant feature transform (SIFT) algorithm algorithm denotes the set of feature points of the third rectified image extracted using the scale invariant feature transform (SIFT) algorithm denotes the set of feature points of the third rectified image extracted using the scale invariant feature transform (SIFT) algorithm representing a real image of a car center screen total number of feature points in the total number of feature points in the step three corrected image the total number of feature points in the step three corrected image representing a real image of a car center screen position parameters of feature points in the image, ; the real image of the car center control screen the position coordinates of the first feature point extracted in the real image of the car center control screen the feature scale of the first feature point extracted in the real image of the car center control screen the real image of the car center control screen the feature scale of the first feature point extracted in the real image of the car center control screen the real image of the car center control screen the main direction angle of the first feature point extracted in the real image of the car center control screen the real image of the car center control screen the main direction angle of the first feature point extracted in the real image of the car center control screen In the formula, denotes the gradient magnitude; representation of a position scale space representation; representation of a position scale space representation; representation of a position scale space representation; representation of a position scale space representation; denotes the gradient direction; denotes a neighborhood window of the feature point; a descriptor vector representing a first feature point extracted from a real image of a car center screen a descriptor vector representing a first feature point extracted from a real image of a car center screen a descriptor vector representing a first feature point extracted from a real image of a car center screen The acquisition process is: Select neighborhood: select a 16x16 pixel neighborhood window around the feature point; Divide the sub-region: divide the 16x16 = 256 pixel neighborhood into 4x4 = 16 sub-regions, each containing 4x4 pixels; Calculate the gradient histogram: quantize the gradient direction into 8 direction intervals: [0°, 45°), [45°, 90°),..., [315°, 360°) and count the gradient amplitude accumulation sum in each direction interval to form an 8-dimensional histogram; Combinatorial descriptors: 16 sub-regions x 8 directions = 128-dimensional vector, i.e. ; real image representing a car center screen the first feature point dimension of the descriptor vector of the first feature point in the first direction of the first sub-region, real image representing a car center screen the first feature point dimension of the descriptor vector of the first feature point in the second direction of the first sub-region, real image representing a car center screen the first feature point dimension of the descriptor vector of the first feature point in the eighth direction of the first sub-region; real image representing a car center screen the first feature point dimension of the descriptor vector of the first feature point in the first direction of the second sub-region; real image representing a car center screen the first feature point dimension of the descriptor vector of the first feature point in the eighth direction of the sixteenth sub-region; upper index denotes transposition; Real image of car center screen In the first Descriptor vector of a feature point has 128 dimensions; representing the position parameters of the feature points in the third step corrected image ;​ position coordinates of the first feature point extracted in the step three corrected image feature scale of the first feature point extracted in the step three corrected image principal direction angle of the first feature point of the step three corrected image; the coordinate system is Cartesian coordinate;​​​​​​ In the formula, denotes the gradient magnitude; representation of a position scale space representation; representation of a position scale space representation; representation of a position scale space representation; representation of a position scale space representation; represents the gradient direction; representing a neighborhood window of the feature point; representing the step three corrected image descriptor vector of the first feature point extracted in the step The acquisition process is as follows: Select neighborhood: select a 16x16 pixel neighborhood window around the feature point; Divide the sub-region: divide the 16x16 = 256 pixel neighborhood into 4x4 = 16 sub-regions, each containing 4x4 pixels; Calculate the gradient histogram: quantize the gradient direction into 8 direction intervals: [0°, 45°), [45°, 90°),..., [315°, 360°) and count the gradient amplitude accumulation sum in each direction interval to form an 8-dimensional histogram; Combine into a descriptor: 16 sub-regions x 8 directions = 128-dimensional vector, i.e. ; denotes the image after step three correction the first feature point in the image dimension of the descriptor vector of the first feature point in the image in the first sub-region in the first direction, denotes the image after step three correction the first feature point in the image dimension of the descriptor vector of the first feature point in the image in the first sub-region in the second direction, denotes the image after step three correction the first feature point in the image dimension of the descriptor vector of the first feature point in the image in the first sub-region in the eighth direction; denotes the image after step three correction the first feature point in the image dimension of the descriptor vector of the first feature point in the image in the second sub-region in the first direction; denotes the image after step three correction the first feature point in the image dimension of the descriptor vector of the first feature point in the image in the sixteenth sub-region in the eighth direction; upper index denotes the transpose; Step three corrected image In the middle The descriptor vector of a feature point has 128 dimensions; 4223. Realistic images based on car center console screens Extracted from the first Descriptor vector of each feature point and the image after correction in step three Extracted from the first Descriptor vector of each feature point The FLANN matcher is used for feature matching to obtain the real images of the car's central control screen that meet the matching conditions. Feature points in and the image after correction in step three Feature points in FLANN stands for Fast Nearest Neighbor Approximation. The specific process is: 1) Define the Euclidean distance function: wherein denotes the L2 norm; a real image representing a car center screen the kth dimension of the descriptor of the mth feature point extracted from the image the kth dimension of the descriptor of the mth feature point extracted from the image representing the third step corrected image the kth dimension of the descriptor of the mth feature point extracted in the step of extracting the mth feature point denotes and the Euclidean distance between 2) Obtain matching values ​​based on Euclidean distance Select real images from the car's central control screen that meet the matching criteria. Feature points in and the image after correction in step three Feature points in ; indicates as: In the formula, representing feature points in the real image of the car center screen satisfying the matching condition representing feature points in the real image of the car center screen satisfying the matching condition and the feature point matching pair composed of the feature points in the corrected image of step three and the feature point matching pair composed of the feature points in the corrected image of step three and the feature point matching pair composed of the feature points in the corrected image of step three representing a real image of a car center screen all dimensions of the descriptor of the i-th feature point extracted in step two corrected image euclidean distance between all dimensions of the descriptor of the i-th feature point extracted in step two representing a real image of a car center screen all dimensions of the descriptor of the i-th feature point extracted in step two all dimensions of the descriptor of the i-th feature point extracted in step three Euclidean distance between all dimensions of the descriptor of the i-th feature point extracted in step two a ratio value representing the ratio of the nearest neighbor distance to the next nearest neighbor distance; representing a ratio threshold, ; is the index of the closest match, i.e. argmin returns the value of the argument of a function that makes the function take its minimum value, i.e. the value of that makes take its minimum value; For the second closest matching point index, i.e. argmin returns the value of the argument of a function that makes the function take its minimum value, i.e. the value of x that makes take its second smallest value. ​ Forty-two four, based on feature points in and feature points , obtain the optimal homography matrix ; the specific process is: 1) Estimate homography matrix using RANSAC algorithm The specific process is: Set the real image of the car control screen In the first Feature point coordinates are ; Step three corrected image In the middle The coordinates of the feature points are ; Matching point pairs The correspondence is derived from a homography matrix Description: 2) based on homography matrix An optimization objective function for constructing a homography matrix is obtained to obtain an optimal homography matrix ; represented as: wherein, is the Huber robust loss function, In the formula, is a threshold parameter; denotes ; denotes the L2 norm; , , , , , , , , denotes an element in the homography matrix ; 4225, judging the optimal homography matrix whether the determinant constraint, the condition number constraint and the perspective component constraint are satisfied simultaneously, if yes, executing 4226; otherwise, correcting the image in step three unqualified; the specific process is: Determinant constraint: , represents a determinant calculation; Condition number constraint: wherein, denotes a condition number constraint on the optimal homography matrix denotes a determinant constraint on the optimal homography matrix ​​ and are the largest and smallest singular values of the matrix respectively; Perspective component constraints: ; 4226. based on the optimal homography matrix spatial coordinates of the pixels in the corrected image after step three performing a transformation to obtain a pre-aligned corrected image ; is represented as: Wherein, express That is, homography matrix Inverse matrix aligned with secondary coordinate vector Perform matrix multiplication to obtain a homogeneous coordinate matrix ,right After normalization, the corrected image is obtained. Corresponding source coordinates in ; This represents matrix multiplication, with the superscript T indicating transpose. , denotes the transformed band-scaled coordinate components; denotes a scaling factor for disambiguating the scale; denotes a pixel sampling of the coordinate positions of the rectified image after inverse homography transformation at the coordinates pixel values.

8. The color restoration and evaluation method for the image captured by the center screen of the vehicle according to claim 7, characterized in that: The real image of the car central control screen in the four two three And the correction image after pre-alignment Carry out meshing to obtain a region , the region Quality assessment to obtain a comprehensive quality score ; Based on the comprehensive quality score Effective area screening is carried out, and the effective area meeting the requirements is obtained ; The specific process is: Four two three one, the real image of the car central screen Carrying out meshing; the specific process is that: a real image of a car center screen is divided into a grid, a real image of a car center screen is represented as: the first row and the first column of the grid the first row and the first column of the grid the first row and the first column of the grid the first row and the first column of the grid wherein, is the grid height, ; is the grid width, ; is the height of the real image of the car center screen, is the width of the real image of the car center screen; is the grid index; and are the number of divisions of the grid width and height, respectively; Four two three two, the pre-alignment of the corrected image Grid division is performed; the specific process is as follows: the pre-aligned image is divided into a grid of N rows and M columns, then the pre-aligned image is represented as: ​​​​​​ wherein, is the grid height, ; is the grid width, ; is the height of the pre-aligned corrected image, is the width of the pre-aligned corrected image; is the grid index; and are the number of divisions of the grid width and height, respectively; Four thousand three hundred thirty-three, to the region Quality assessment is carried out to obtain a comprehensive quality score The specific process is: To the region Calculate the overall quality score: The weights satisfy a normalization condition , ; The acquisition process is: 1) Computing a structural similarity index ; the specific process is: The structural similarity index value domain is [-1, 1]; wherein , denotes the mean value; , denotes the variance; denotes the covariance; is a constant of stability, , 255 is the grey level range; denotes the structural similarity index of a region . mean value and The formula for calculating the mean value is: variance and The calculation formula is: Covariance The formula for the calculation of the covariance is: refers to inner coordinates, , denotes and the mth row and nth column region common to wherein, represents and the number of pixels in the mth row and nth column region common to representing coordinates corresponding real image of the car center screen; representing coordinates corresponding pre-aligned corrected image; 2) Calculate normalized cross-correlation ; the specific process is: Normalized cross-correlation For assessing the similarity of the gray scale patterns, the following is expressed: wherein, the real image of the car center screen is in region the gray value at For the pre-aligned corrected image at Region the gray value at The real image of the car center screen is in The gray value of the region at the place For the pre-aligned corrected image in Region of the gray value at The average gray value of the real image of the car central screen in the region; the average gray value of the region; and the average gray value of the region; and 3) calculating the quality of the feature match The specific process is: wherein, , represents a set of feature points in the mth row and nth column region in the image. representing a set of feature points in the mth row and nth column region of the matrix for the region set of matching feature points within the region; cardinality of the set 4), calculate the edge correlation; the specific process is: Real image of car center screen and pre-aligned corrected image Canny edge detection is performed respectively to obtain binary edge images and ; Wherein, the edge point value is 1, and the non-edge point value is 0; Region The edge correlation index of a region is defined as: Covariance calculation: wherein, with the Canny edge detection result at the coordinates of the region in the real image and the pre-aligned corrected image is 0 or 1. is 0 or 1. denotes and covariance; represents the edge density, i.e. In the region of the mean, ; represents the edge density, i.e. the mean value of the region of the region, ; Four two three four, based on the comprehensive quality score An effective area screening is performed to obtain an effective area satisfying the requirements The specific process is: wherein is a mass threshold value.

9. The color restoration and evaluation method for the image captured by the center screen of the vehicle according to claim 8, characterized in that: The four 424 in the effective area Local fine registration is performed in the effective area of the four 424, and a local fine registered local captured image is obtained The specific process is as follows: Four two four one, construct a local affine transformation model; the specific process is: Estimating a local affine transformation matrix for the valid region: wherein denotes a local affine transformation matrix; by scaling factor , rotation angle , shear parameter together determine the "scaling + rotation + shear" in the corresponding transformation in the Cartesian coordinate system directions; by scaling factor , rotation angle , shear parameter together determine the "scaling + rotation + shear" in the corresponding transformation in the Cartesian coordinate system direction components influence coefficients of direction components; by scaling factor , rotation angle , shear parameter together determine the "scaling + rotation + shear" in the corresponding transformation in the Cartesian coordinate system direction components influence coefficients of direction components; by scaling factor , rotation angle , shear parameter together determine the "scaling + rotation + shear" in the corresponding transformation in the Cartesian coordinate system directions; translation of Cartesian coordinate system directions, translation of Cartesian coordinate system directions; The geometric meaning of affine transformation includes rotation, scaling, shearing and translation: wherein, is a scaling factor; is a rotation angle; , is a shear parameter; , is a translation amount; Four two two two, based on local affine transformation matrix Obtaining optimal transform parameters The specific process is: For the region , find the optimal transform parameters maximize the enhancement correlation coefficient ECC: wherein, representing a region within which the real image is enhanced relative to ; representing a real image within a region ; representing a local sub-image within a region ; representing a corrected image after a local affine transformation of the coordinate positions in the corrected image ; denotes a local affine transformation matrix a homogeneous coordinate vector a matrix multiplication transformation to obtain a homogeneous coordinate matrix , after a normalization operation on , to obtain the corresponding source coordinates in the rectified image ; denotes a matrix multiplication, the upper index T denotes the transpose; , denotes the first two components of the homogeneous coordinate matrix after a local affine transformation, corresponding to the scaled horizontal coordinate component after the transformation and the scaled vertical coordinate component after the transformation; denotes the third component of the homogeneous coordinate matrix; denotes an affine transformation model; The enhancement correlation coefficient ECC is defined as: wherein denotes a zero-mean image, ; representing a real image the mean value in the region ;​ denotes a zero-mean image; ； representing the mean value within the region ​ ； Affine transformation model representing a real image from a car center screen coordinates of to a pre-aligned corrected image coordinates of transformations; represented as: wherein the semicolon before is an input coordinate, ; after the semicolon for the affine transformation parameters, the upper index denotes the transpose; Fourthly, the Gauss-Newton method is used to solve the affine transformation parameters iteratively to obtain optimal affine transformation parameters ; the obtained optimal affine transformation parameters are substituted into the affine transformation model to obtain optimal transformation parameters ; the specific process is as follows: 1) setting the initial value of the iteration number ; setting the initial value of the parameters ; 2), maximizing ECC is equivalent to minimizing the following error function: wherein, denotes a zero-mean image; denotes a zero-mean image, denotes the error function of the ith iteration; 3) based on The region Within the Pixels are stacked point by point, and the first Error vector of the second iteration , ; represented as: wherein, the RGB of a coordinate point in a zero-mean image representing the real image the image after an affine transformation model the RGB of a coordinate point in a zero-mean image representing the image after an affine transformation model​ denotes i.e. a local affine transformation matrix homogeneous coordinate vector a matrix multiplication transformation, resulting in a homogeneous coordinate matrix , after a normalization operation on , resulting in the corresponding source coordinates in the rectified image ; denotes a matrix multiplication, the upper index T denotes the transpose; , denotes the first two components of the homogeneous coordinate matrix of the local affine transformation, corresponding to the scaled x-coordinate component after the transformation and the scaled y-coordinate component after the transformation; denotes the third component of the homogeneous coordinate matrix; represents an affine transformation model; RGB of coordinate points in the zero-mean image representing the real image RGB of coordinate points in the zero-mean image representing the real image RGB of coordinate points in the zero-mean image representing the real image RGB of coordinate points in the zero-mean image representing the real image RGB of coordinate points in the zero-mean image representing the real image RGB of coordinate points in the zero-mean image representing the real image indicates ; RGB of the coordinate point in the zero-mean image representing the real image RGB of the coordinate point in the zero-mean image representing the real image RGB of the coordinate point in the zero-mean image representing the real image RGB of the coordinate point in the zero-mean image representing the real image RGB of the coordinate point in the zero-mean image representing the real image RGB of the coordinate point in the zero-mean image representing the real image indicates ; represents an error vector; is the number of pixels in the region is the number of pixels in the region is the number of pixels in the region , is the number of pixels in the region is the number of pixels in the region 4) obtain the first error vector of the second iteration partial derivative of the parameter is expressed as: wherein denotes the partial derivative of the error vector with respect to the parameters; denotes the affine transformation matrix at pixel denotes the partial derivative of the affine transformation matrix with respect to the parameter ; denotes the affine transformation matrix at pixel denotes the partial derivative of the affine transformation matrix with respect to the parameter ; denotes the affine transformation matrix at pixel denotes the partial derivative of the affine transformation matrix with respect to the parameter ; denotes the affine transformation matrix at pixel denotes the partial derivative of the affine transformation matrix with respect to the parameter ; for the image gradient, , denotes the Cartesian coordinate system abscissa, denotes the Cartesian coordinate system ordinate, the upper index denotes the transposition; Local affine transformation matrix The partial derivative of the function f with respect to the variable x is: 5) the number of iterations ; update parameters ; repeat 2) - 5) until the convergence condition is met, and obtain the optimal parameters ; Parameter increment ; wherein is a step factor; The obtained optimal affine transformation parameters Substitute the affine transformation model , and obtain optimal transformation parameters ; The convergence condition is: terminate when any of the following conditions is met: (1) the parameter increment is small enough: , is a parameter convergence threshold, denotes the L2 norm; (2) the error change is small enough: , is an error convergence threshold; (3) the maximum number of iterations is reached: , is the maximum number of iterations; Four two four four, optimal transform parameters based on four two four three obtaining the fine registered partial captured image ; is represented as: representing a corrected image after a local affine transformation in the corrected image at coordinates ; denotes i.e. the optimal transformation matrix the inverse matrix aligns the homogeneous coordinate vector a matrix multiplication transformation is performed to obtain a homogeneous coordinate matrix , which is normalized to obtain the corrected image with the corresponding source coordinates ; ; denotes a matrix multiplication, the upper index T denotes the transpose; , denotes the first two components of the homogeneous coordinate matrix after the optimal transformation, corresponding to the scaled x-coordinate component after the transformation and the scaled y-coordinate component after the transformation; denotes the third component of the homogeneous coordinate matrix; represents an affine transformation model.

10. The color restoration and evaluation method for the image captured by the center screen of the vehicle according to claim 9, characterized in that: The step four two five real image based on the central control screen And the fine registration of the obtained local photographed image Calculate the color difference; the specific process is: the real image of the central control screen and the fine registered partial captured image obtained from the four 2x2 RGB components are converted to CIE Lab color space, based on the image and the image corresponding pixels in CIE Lab color space, , , the CIE color difference is calculated for each pixel within the valid area ; represents the luminance component; represents the red-green component; represents the yellow-blue component; Map the numerical values of the color difference to pseudo-colors, output a heat map based on the pseudo-colors, and the color difference range of the heat map is 0-100; based on a color difference and an analysis mask , calculating an average color difference of the effective area ; expressed as: Analysis mask The acquisition process is: Grid regions based on requirements being met Constructing an analysis mask: wherein, is the total number of valid pixels. is the total number of valid pixels. is the total number of valid pixels.

Images