A color image color temperature compensation method applied to offline collection

Abstract

The application discloses a color image color temperature compensation method applied to offline collection, which comprises the following steps: obtaining a to-be-corrected image and a standard template image, extracting pixel points of a low-texture non-edge area as feature points and forming a feature point pair; realizing sub-pixel level alignment through geometric transformation of a homography matrix; calculating an illumination correlation parameter of the feature point pair; extracting a local illumination regularity parameter of the feature points, including a local texture orientation feature, a local contrast feature and a local texture complexity feature; constructing a mapping model from a pixel position to a color compensation amount through an interpolation algorithm; calculating and superimposing the color compensation amount pixel by pixel according to the mapping model, and outputting a corrected image; the application can be applied to offline collection images with large illumination differences and large environment differences, effectively solves the problem of inconsistent color temperatures of different batches of images in rail train detection, and significantly improves the precision and robustness of color correction.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and more specifically to a color temperature compensation method for color images acquired offline. Background Technology

[0002] In industrial vision inspection systems, camera imaging quality is highly susceptible to environmental factors such as temperature, humidity, dust, light source attenuation, and vibration. This can cause sensor response drift, resulting in color temperature shifts, overall color casts, local color differences, or uneven brightness in the acquired images. These issues severely affect the accuracy and stability of subsequent detection algorithms.

[0003] To address the aforementioned issues, various color or color temperature correction methods have been proposed in the existing technology, mainly including the following categories: Global white balance methods estimate color temperature shifts by statistically analyzing the global grayscale or color distribution of an image and then perform overall correction. However, global white balance can only correct overall color temperature deviations and cannot handle local color differences and uneven lighting caused by uneven illumination, local shadows, or material differences. It is not effective for images with complex textures or multiple color regions.

[0004] Color correction methods based on style transfer or deep learning train neural networks with a large number of paired samples to learn the mapping relationship from color-biased images to standard images. These methods have poor generalization ability, insufficient adaptability to different scenarios or devices, and are prone to losing image details. They also have a large computational load and are difficult to meet the needs of real-time industrial detection.

[0005] Template difference compensation method: This type of method calculates the difference between the image to be corrected and the template image by pre-acquiring a standard template image. This method is extremely sensitive to changes in camera position, angle and distance, and requires strictly fixed shooting conditions. Once there is a change in translation, rotation, scaling or viewing angle, the compensation effect will be significantly reduced. It is difficult to deploy on site and has low robustness.

[0006] Color constancy-based correction methods assume the presence of highlight or neutral color regions in an image, estimate ambient lighting by detecting these regions, and then perform corrections. However, existing color constancy algorithms do not incorporate local image features and local lighting residuals, resulting in insufficient correction accuracy and making it difficult to meet the needs of high-precision industrial inspection.

[0007] In summary, the existing methods mentioned above are mostly applicable to image sequences captured continuously or under relatively consistent lighting conditions. Specifically, in the field of track detection, target detection is required at different time points. That is, for offline acquired images, there are large differences in illumination and environment between different images, making it difficult for existing compensation methods to be applied and unable to achieve effective color correction. Summary of the Invention

[0008] To address the shortcomings of existing technologies, the present invention aims to provide a color temperature compensation method for offline color image acquisition. The purpose is to solve the technical problems that existing color correction methods are not applicable to offline acquired images with large differences in illuminance and environment, and are difficult to handle local color differences and uneven illumination.

[0009] To achieve the above objectives, the present invention provides the following technical solution: A method for color temperature compensation of color images acquired offline includes the following steps: The image feature extraction step involves acquiring the image to be corrected and the standard template image, extracting pixels with a local window variance lower than a preset threshold and located in a non-edge region from the two images as feature points, and combining them to form feature point pairs. The geometric transformation step involves transforming the image to be corrected using the estimated homography matrix through feature points, so that the transformed image to be corrected is in the same image coordinate system as the standard template image. The illuminance difference calculation steps involve calculating the pixel value difference between the two feature points in each channel of the color space for each feature point pair as the illuminance correlation parameter for that feature point pair. The local gradient analysis step involves taking each feature point in the transformed image to be corrected as the center, calculating the gradient direction and gradient magnitude of the feature point and its surrounding pixels in each channel of the color space, which are used as the local illumination regularity parameters of the feature point. The mapping function construction steps are as follows: the illumination correlation parameters of all feature point pairs are used as sparse control points, the local illumination regularity parameters of each feature point are used as local constraints, and an interpolation algorithm is used to construct a mapping model from the pixel position of the transformed image to be corrected to the color compensation amount. The color compensation step involves calculating the color compensation amount for each pixel in the transformed image to be corrected based on the mapping model, superimposing the color compensation amount onto the original pixel value of that pixel, and outputting the corrected image.

[0010] Furthermore, the local gradient analysis step includes an illumination regularity extraction strategy. This strategy involves constructing multiple concentric windows centered on the feature point. In the inner window, the principal gradient direction and gradient magnitude of the feature point in each color channel are calculated. In the outer window, the statistical histogram of the gradient direction and the cumulative distribution function of the magnitude are calculated. The principal gradient direction is used as the local texture orientation feature of the feature point. The ratio of the gradient magnitude to the mean magnitude in the outer window is used as the local contrast feature of the feature point. The peak distribution width of the statistical histogram of the gradient direction is used as the local texture complexity feature of the feature point. The combination of the local texture orientation feature, local contrast feature, and local texture complexity feature is used as the local illumination regularity parameter of the feature point.

[0011] Furthermore, the local gradient analysis step also includes a feature point propagation strategy. The feature point propagation strategy includes inserting virtual feature points at equal intervals on the line connecting adjacent feature points when the Euclidean distance between adjacent feature points in the image to be corrected is greater than a preset distance threshold. The local illumination regularity parameters of the virtual feature points are obtained by linear interpolation or cubic spline interpolation based on the local illumination regularity parameters of adjacent feature points. When the number of feature points in a certain local area of ​​the image to be corrected is lower than a preset density threshold, new feature points are generated by spreading outward along the gradient direction or the direction of iso-illuminance lines, using the original feature points in that area as seed points. The local illumination regularity parameters of the new feature points are obtained by weighting the local illumination regularity parameters of the corresponding seed points with the attenuation weight coefficient of the diffusion distance.

[0012] Furthermore, the mapping function construction step includes an anisotropic interpolation strategy. The anisotropic interpolation strategy includes using the illumination correlation parameter of each feature point pair as the function value of the sparse control point, using the local texture orientation feature in the local illumination regularity parameter of each feature point as the anisotropic direction of the interpolation basis function, using the local texture complexity feature in the local illumination regularity parameter of each feature point as the anisotropic scale of the interpolation basis function, setting a first smoothing weight along the direction parallel to the local texture orientation, setting a second smoothing weight along the direction perpendicular to the local texture orientation, and the value of the first smoothing weight is greater than the value of the second smoothing weight. When the local texture complexity feature increases, the ratio of the first smoothing weight and the second smoothing weight is reduced according to a preset proportional relationship.

[0013] Furthermore, the mapping function construction step also includes a region constraint interpolation strategy. The region constraint interpolation strategy includes dividing the standard template image into multiple regions with uniform color by using a superpixel segmentation algorithm, establishing a region label for each region, and using the region label as the boundary identifier of the interpolation process. Pixels corresponding to the same region label participate in the interpolation calculation by feature point pairs falling within that region. Pixels corresponding to different region labels are assigned infinitely large smoothing weights in the interpolation calculation.

[0014] Furthermore, the mapping function construction step also includes a local color residual fusion strategy. The local color residual fusion strategy includes calculating the color residual between the standard template image and the image to be corrected at each feature point location. The color residual includes red channel residual, green channel residual, and blue channel residual. The color residual is added as an additional constraint term to the objective function of the interpolation algorithm. The objective function of the interpolation algorithm includes fitting terms for illumination correlation parameters and fitting terms for color residuals. Based on the color residual at each feature point location, the color residual at non-feature point locations is obtained by diffusion through the interpolation algorithm. The diffused color residual is then weighted and summed pixel-by-pixel with the color compensation amount output by the mapping model, and the summation result is used as the final color compensation amount at that pixel location.

[0015] Furthermore, the mapping function construction step also includes a multi-basis function construction strategy. The multi-basis function construction strategy includes using the position coordinates of each feature point pair in the image as input independent variables, using the illumination correlation parameter of each feature point pair as input dependent variables, constructing a first mapping model through a radial basis function interpolation algorithm, and constructing a second mapping model through a thin plate spline interpolation algorithm.

[0016] Furthermore, based on the prediction errors of the output values ​​of the first and second mapping models at each feature point location relative to the illumination correlation parameter, adaptive weight coefficients are calculated at each feature point location. Based on the adaptive weight coefficients at each feature point location, fusion weight coefficients at each pixel location in the entire image are obtained through interpolation algorithm. Then, the outputs of the first and second mapping models are weighted and summed pixel by pixel based on the fusion weight coefficients, and the summation result is used as the final color compensation amount for that pixel location.

[0017] Furthermore, the color compensation step includes a confidence evaluation strategy, which includes calculating the confidence coefficient of the color compensation amount at each pixel position based on the spatial distance between each pixel position and the nearest feature point pair, as well as the local texture complexity features of the region where the pixel position is located.

[0018] Furthermore, the confidence evaluation strategy also includes correcting the color compensation amount of pixels with a confidence coefficient higher than the threshold in the neighborhood of the pixel location by weighted averaging when the confidence coefficient is lower than the preset confidence threshold, and then replacing the original color compensation amount with the corrected color compensation amount and adding it to the original pixel value of the pixel.

[0019] The beneficial effects of this invention are as follows: 1. This invention extracts pixels with low variance in pixel values ​​within a local window and located in non-edge regions as feature points. Combined with geometric transformation, it achieves sub-pixel alignment between the image to be corrected and the standard template image. It obtains the illuminance correlation parameters of feature point pairs and the local illuminance regularity parameters of feature points through illuminance difference calculation and local gradient analysis, respectively. Then, it constructs a mapping model from pixel position to color compensation amount through interpolation algorithm, realizing the diffusion of color compensation amount from sparse feature points to all pixels in the image. Compared with the prior art, this invention does not require strictly fixed shooting conditions and can be applied to offline acquired images with sparse or unevenly distributed feature points, large illuminance differences, and large environmental differences. It solves the problems of sensitivity to shooting conditions and low robustness of traditional template difference compensation methods.

[0020] 2. This invention employs an anisotropic interpolation strategy to dynamically adjust interpolation weights based on local texture orientation and complexity features, prioritizing the propagation of color compensation along texture directions and suppressing cross-texture propagation. A region-constrained interpolation strategy uses the region segmentation results of a standard template image as interpolation boundary constraints, prohibiting the propagation of compensation between different color regions. A local color residual fusion strategy incorporates the color residuals at feature point locations as additional constraints into the interpolation objective function, and obtains the full-image color residual distribution through diffusion, fusing it with the color compensation output by the mapping model. Compared to existing technologies, this invention effectively avoids the problems of compensation overflow across texture boundaries and cross-regional color contamination, achieving multi-source information complementary correction and significantly improving the accuracy of local color difference correction and the integrity of image texture details.

[0021] 3. This invention constructs multiple mapping models by simultaneously employing radial basis function interpolation and thin-plate spline interpolation, and adaptively determines the fusion weight based on the prediction error of each model at the feature point location, comprehensively utilizing the advantages of different interpolation algorithms. Through a confidence evaluation strategy, the confidence coefficient of the compensation amount is calculated based on the spatial distance between the pixel and the nearest feature point and the local texture complexity features, and the compensation amount in low-confidence areas is corrected by neighborhood weighting. Compared with the prior art, this invention further improves the prediction accuracy of the mapping model across the entire image range, ensures the natural continuity of color transitions, avoids the generation of compensation faults or artifacts, and can be widely applied in the field of rail transit inspection. Attached Figure Description

[0022] Figure 1This is a flowchart of the overall method in this invention; Figure 2 This is a flowchart of local gradient analysis and feature point propagation in this invention; Figure 3 This is a flowchart of the mapping model construction and compensation fusion process in this invention; Figure 4 This is a schematic diagram of color temperature compensation for the oil level image of the train undercarriage gearbox in this invention; Figure 5 This is a schematic diagram of color temperature compensation for train bellows images in this invention. Detailed Implementation

[0023] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Identical components are denoted by the same reference numerals. It should be noted that the terms "front," "rear," "left," "right," "upper," and "lower" used in the following description refer to directions in the accompanying drawings, and the terms "bottom surface," "top surface," "inner," and "outer" refer to directions toward or away from the geometric center of a specific component, respectively.

[0024] The core of this invention lies in constructing an image color temperature compensation method based on the analysis of local feature illuminance patterns. By extracting feature point pairs between the image to be corrected and the standard template image, illuminance correlation parameters and local illuminance regularity parameters are calculated. A mapping model is constructed through an interpolation algorithm to achieve color compensation for the entire image. It is particularly suitable for offline acquisition scenarios and can effectively handle the technical problems of large differences in illuminance and environmental differences between different images.

[0025] like Figure 1-3As shown, a color temperature compensation method for offline color images includes an image feature extraction step, acquiring the image to be corrected and a standard template image. The image to be corrected refers to an offline image acquired under non-standard lighting conditions, affected by factors such as ambient temperature, humidity, light source attenuation, dust, and train vibration, causing sensor response drift and resulting in color temperature shift or color distortion. In the field of track inspection, train inspection operations are usually carried out at night or in the early morning, with complex and variable lighting conditions. Furthermore, the natural and artificial lighting conditions differ significantly across seasons and time periods, leading to significant color temperature differences between different batches of images of the same train component. A template image refers to an ideal image under standard lighting conditions without environmental interference. Feature points are extracted from two images. The selection criteria for feature points require that the pixels simultaneously meet the conditions of low-texture areas and non-edge areas, such as the uniformly painted areas on the surface of a train body, the flat surface of a gearbox shell, and the middle of the cylindrical surface of an air tank in a wind source device. These feature points have the advantages of being located in areas with uniform color and flat texture, far from structurally sensitive positions such as edges and corners, and their color values ​​are only affected by changes in ambient illumination under different shooting conditions, without being affected by local texture structures or edge positioning errors. They can stably reflect the illumination difference between the image to be corrected and the standard template image.

[0026] The specific selection of the low-texture region condition is as follows: within a local window centered on the candidate pixel, the variance of pixel values ​​for each color channel, including the red, green, and blue channels, is calculated. When the variance of pixel values ​​for all color channels is lower than a preset first threshold, the candidate pixel is determined to be located in a flat texture region. This condition ensures that the extracted feature points are located in regions with gentle grayscale changes, avoiding the extraction of feature points in complex texture regions, such as the casting and forging texture on the surface of a gearbox or the liquid surface fluctuation region in an oil observation window. This is because the pixel values ​​in complex texture regions are easily affected by local texture structures, and their color values ​​cannot stably reflect changes in ambient illumination. At the same time, the pixel values ​​in low-texture regions have better repeatability in the image and are easier to be stably detected and matched under different shooting conditions.

[0027] The specific selection of the non-edge region condition is as follows: the gradient magnitude of the candidate pixel in the horizontal direction and the gradient magnitude in the vertical direction are calculated by the Sobel operator. When the gradient magnitude in the horizontal direction and the gradient magnitude in the vertical direction are both lower than the preset second threshold, the candidate pixel is determined to be located in the non-edge region. This condition ensures that the extracted feature points are far away from the edge positions in the image, such as the edge of the gearbox outline, the border of the oil observation window, and the bolt outline, because there are significant gray-level jumps at the edge positions, and their pixel values ​​are easily affected by factors such as edge direction and sub-pixel positioning errors, and are not suitable as the benchmark points for illumination analysis.

[0028] After extracting feature points, the feature points in the image to be corrected are matched with the feature points in the standard template image. The matched feature points are combined to form feature point pairs. Specifically, by calculating the Euclidean distance between the descriptor vectors of each feature point in the image to be corrected and the descriptor vectors of each feature point in the standard template image, the pair of feature points with the smallest distance and below a preset third threshold is taken as the initial matching pair, ensuring that the two feature points in each pair correspond to the same physical location on the train component. In addition, to improve the matching accuracy, a step to eliminate mismatched point pairs can be designed. The RANSAC algorithm is used to iteratively filter the initial matching pairs, retaining the matching pairs that satisfy the homography constraint as the final feature point pairs.

[0029] Since this invention uses the color difference of feature point pairs as sparse control points to derive the color compensation amount of the surrounding area, the two feature points in each feature point pair must correspond precisely to the same physical location. If there is a deviation in geometric alignment, even a one-pixel offset will cause the feature points in the image to be corrected to not correspond to the same physical location as the feature points in the standard template image. In this case, the calculated pixel value difference will be mixed with the texture difference caused by the position offset, rather than a simple difference in ambient illumination.

[0030] Therefore, the geometric transformation steps, including subpixel-level alignment, can eliminate this positional offset error to the greatest extent and ensure that the pixel value difference between feature point pairs truly reflects the changes in ambient illumination. Specifically, by estimating the homography matrix based on the feature point pairs, the spatial transformation relationship between two images can be accurately described. That is, in the scenario of railway train detection, the camera may not be able to be installed at the exact same position and angle every time due to the space constraints at the bottom of the train. The camera carried by the inspection robot may also have slight pose changes between different inspection cycles. High-precision geometric alignment can also be achieved. Then, the coordinate transformation of the image to be corrected is performed through the estimated homography matrix. Specifically, for each pixel in the image to be corrected, its original pixel coordinates are projected through the homography matrix, and the transformed target coordinates are calculated. Since the transformed target coordinates are usually not integers, the transformed image needs to be resampled to obtain the pixel values ​​at each pixel location. The resampling process uses bilinear interpolation or bicubic interpolation algorithms. Taking bilinear interpolation as an example: for a pixel location in the target image, find its four nearest neighbor original pixels, calculate the weights based on the distances from the location to the four neighboring points, and perform a weighted average of the pixel values ​​of the four neighboring points to obtain the pixel value at that location. During the coordinate transformation and resampling process, the original pixel values ​​of the image to be corrected are mapped to a new coordinate system, generating the transformed image to be corrected. This image and the standard template image are in the same image coordinate system, and pixels corresponding to the same physical location in the two images have the same image coordinates.

[0031] The illuminance difference calculation steps involve calculating the pixel value difference between the two feature points in each feature point pair across all channels of the color space as the illuminance correlation parameter for that feature point pair. Specifically, first, the color space used for calculating the difference is determined. Image acquisition and subsequent image correction for railway trains are typically performed in the RGB color space, which includes the red, green, and blue channels. Then, for each feature point pair, the pixel values ​​of the feature points in the image to be corrected and the corresponding pixel values ​​of the feature points in the standard template image are obtained. The pixel values ​​of the feature points in the image to be corrected are... The pixel value of the feature points in the standard template image is The algorithm calculates the pixel value differences between two feature points in the red, green, and blue channels. These differences can be signed (positive values ​​indicate the channel value of the image to be corrected is greater than the standard template image, negative values ​​indicate it is less) or absolute (absolute values). Signed differences retain the direction information of the color shift, which is helpful for distinguishing shifts in different directions during subsequent interpolation. Absolute differences only reflect the magnitude of the shift and are suitable for applications where the shift direction is not critical. The calculated channel differences are then combined to form the illuminance correlation parameter for the feature point pair. This parameter is a multi-dimensional vector with the same dimension as the number of channels in the selected color space. It quantitatively describes the degree and direction of color shift at the same physical location under different environmental illuminance conditions.

[0032] In actual acquired images, some feature point pairs may be affected by factors such as local reflections (e.g., oil film reflections on the surface of a gearbox), stains (e.g., dust or oil on the observation window glass), shadows, or obstructions, resulting in abnormally high or low illumination correlation parameters. If these outliers are directly used for subsequent interpolation, they will affect the accuracy of the mapping model. Therefore, during the difference calculation process, the absolute value of the difference between each set of feature point pairs can be judged. When the absolute value of the difference between a certain color channel or the average of all color channels exceeds the preset difference threshold, the set of feature point pairs is marked as a high difference pair. For high difference pairs, a low weight coefficient can be assigned to reduce the influence of subsequent differences, or the feature point pair can be directly removed.

[0033] like Figure 2As shown, the local gradient analysis step, centered on each feature point in the transformed image to be corrected, calculates the gradient direction and magnitude of the gradient between the feature point and its surrounding pixels in each channel of the color space. This serves as a local illumination regularity parameter for the feature point. In the train image, the casting surface of the gearbox housing has randomly distributed rough textures, the surface of the oil observation window glass has smooth and flat textures, the painted surface of the vehicle body has a consistent brushed texture, and the bolt connections have radial textures. Different texture regions should have different response characteristics to the propagation of color compensation. In areas with simple textures and clear directionality, color... The compensation amount should be propagated along the texture direction first to maintain the consistency of the structure. In areas with complex textures and varied directions, the color compensation amount should adopt a more isotropic propagation method to retain rich detail variations. To achieve this differentiated propagation control, it is necessary to first quantify the texture structure of the region where each feature point is located. That is, to construct multi-layer concentric windows through the illumination regularity extraction strategy in the local gradient analysis step, extract the gradient features of feature points from different scales, and finally form a local illumination regularity parameter composed of three components: local texture orientation features, local contrast features, and local texture complexity features.

[0034] Specifically, a multi-layer concentric window is constructed with the feature point as the center. The multi-layer concentric window refers to multiple rectangular or circular windows of different sizes that expand outward from the feature point as a common center point. In this invention, there are inner and outer windows. The inner window is a square window with an odd number of pixels of size 3×3. The inner window is used to capture the fine texture information of the feature point location, including local detail features such as the main gradient direction and gradient magnitude. The outer window is 7×7 pixels. The outer window is used to capture the texture distribution characteristics of a larger area around the feature point, including the statistical distribution of gradient direction and the overall level of gradient magnitude. By constructing a multi-layer concentric window, the gradient information of the feature point can be extracted at different spatial scales, forming a multi-level description of the local texture structure.

[0035] In the inner window, the principal gradient direction and gradient magnitude of the feature points in each color channel are calculated. The gradient can be calculated separately in the horizontal direction using the Sobel operator. and vertical gradient Then, the gradient magnitude and gradient direction angle of each pixel are calculated. For all pixels in the inner window, the gradient distribution of each color channel is statistically analyzed. By combining the gradient information of each channel, the direction with the largest gradient energy in the window is determined as the main gradient direction. The main gradient direction reflects the local texture orientation at the location of the feature point. The main gradient direction is perpendicular to the direction of the texture. In flat areas without obvious directionality, the main gradient direction may be randomly distributed. At the same time, the average gradient magnitude in the inner window is calculated. The gradient magnitude reflects the degree of grayscale change in the region. A large gradient magnitude indicates that there is a significant grayscale change in the region (such as texture edges), while a small gradient magnitude indicates that the grayscale change in the region is gradual (such as a uniform surface).

[0036] In the outer window, a statistical histogram of gradient directions is calculated. The gradient direction angular space from 0° to 360° is divided into several intervals. The frequency of the gradient direction of each pixel in the outer window falling within each interval is counted to form a histogram. This histogram is then normalized to obtain the probability distribution of gradient directions. The peak distribution width of the gradient direction statistical histogram reflects the concentration of texture directions in the local area. If the histogram shows a sharp single peak, it indicates that the texture directions in the area are highly consistent and the texture structure is simple. If the histogram shows a wide and flat distribution or a multi-peak distribution, it indicates that the texture directions in the area are diverse or that there are multiple textures with different orientations, and the texture structure is complex. The peak distribution width is used as a feature of local texture complexity. On the painted surface of the vehicle body, the peak distribution width is small; on the gearbox casting surface, the peak distribution width is large.

[0037] The cumulative distribution function of gradient magnitude is calculated. The gradient magnitudes of all pixels in the outer window are collected, sorted by magnitude, and the cumulative distribution function is calculated. The cumulative distribution function reflects the cumulative proportion of different magnitude levels in the window. The cumulative distribution function rises rapidly at small magnitudes, indicating that the window is dominated by low gradient regions (flat regions). The cumulative distribution function still rises significantly at larger magnitudes, indicating that there are more high gradient regions (edge ​​or texture regions) in the window.

[0038] The calculated gradient information is then combined into local texture orientation features, local contrast features, and local texture complexity features. The local texture orientation feature is the principal direction of the gradient calculated within the inner window. This feature is a direction angle that describes the direction of the texture in the local region where the feature point is located. In subsequent anisotropic interpolation, this feature is used to determine the principal direction of the interpolation basis function, ensuring that the color compensation propagates preferentially along the texture direction. The local contrast feature is the ratio of the gradient magnitude in the inner window to the average magnitude in the outer window. This feature reflects the salience of the feature point relative to its surrounding area: a ratio greater than 1 indicates that the gradient magnitude at the feature point is higher than the average level of the surrounding area, making the point more prominent; a ratio smaller than 1 indicates a less prominent feature point. A value of 1 indicates that the gradient magnitude at the feature point is lower than the surrounding average, indicating a relatively gentle gradient at that point. In subsequent interpolation, this feature can be used to adjust the strength or confidence of the compensation amount. The local texture complexity feature is the peak distribution width of the gradient direction statistical histogram in the outer window. This feature describes the richness of texture directions in the local region. The smaller the peak distribution width, the more uniform the texture direction and the simpler the texture structure. The larger the peak distribution width, the more diverse the texture direction and the more complex the texture structure. In subsequent anisotropic interpolation, this feature is used to dynamically adjust the degree of anisotropy. When the texture complexity is low, anisotropy is enhanced (strengthening directional propagation), and when the texture complexity is high, anisotropy is weakened (moving towards isotropy to preserve details).

[0039] Since the image to be corrected may have sparse or uneven distribution of feature points, such as a large area of ​​uniformly painted surface on the side of a train or the surface of various smooth metal parts on the bottom of a train, the number of pixels that can meet the feature point selection criteria is very small. To address this problem, a feature point propagation strategy can also be included. By generating virtual feature points and spreading new feature points, the number and density of feature points can be increased. Even in flat or edge areas where feature points are sparse, a sufficiently dense distribution of feature points can be obtained, providing sufficient local constraints for the subsequent interpolation process. When the Euclidean distance between adjacent feature points is greater than the preset distance threshold, it indicates that the feature points in the region are too sparse. At this time, virtual feature points are inserted at equal intervals on the line connecting adjacent feature points. The local illumination regularity parameters of the virtual feature points are obtained by linear interpolation or cubic spline interpolation based on the local illumination regularity parameters of the adjacent feature points, so that the texture orientation, contrast and complexity features of the virtual feature points naturally transition with those of the adjacent feature points. When the number of feature points in a certain local area is lower than the preset density threshold, the existing feature points in that area are used as seed points to generate new feature points by spreading outward along the gradient direction or the iso-illuminance line direction. The local illuminance regularity parameters of the new feature points are based on the local illuminance regularity parameters of the corresponding seed points and are weighted and corrected according to the weight coefficients calculated by the diffusion distance and the preset attenuation function. The closer the diffusion distance, the larger the weight coefficient, and the closer the parameters of the new points are to the seed points; the farther the diffusion distance, the smaller the weight coefficient, and the parameters of the new points gradually tend to a neutral value.

[0040] like Figure 3 As shown, the mapping function construction steps take the illumination correlation parameters of all feature point pairs as sparse control points and the local illumination regularity parameters of each feature point as local constraints. Through interpolation algorithms, a mapping model from the pixel position of the transformed image to the color compensation amount is constructed. This model can output the corresponding color compensation amount for any pixel position in the image, realizing a complete mapping from sparse control points to the pixel compensation amount of the whole image.

[0041] Specifically, the input to the interpolation algorithm includes independent and dependent variables. The independent variable is the position coordinates of each feature point pair in the image. For each feature point pair, its position coordinates can be the coordinates of the feature points in the image to be corrected, or the coordinates of the feature points in the standard template image. Since the geometric transformation step has aligned the two images to the same coordinate system, the position coordinates are two-dimensional coordinates. The dependent variable is the illuminance correlation parameter for each feature point pair. The illuminance correlation parameter is a multi-dimensional vector, with its dimension matching the number of channels in the selected color space. Therefore, the illuminance correlation parameter is a three-dimensional vector. The output of the interpolation algorithm is a continuous function. The function uses pixel coordinates As input, the color compensation amount at that location. For output, the color compensation amount at any pixel location in the image can be obtained by substituting the coordinates.

[0042] This invention achieves the mapping from sparse control points to continuous functions through two interpolation algorithms: radial basis function interpolation, which uses the position of each feature point as the center of the basis function and represents the mapping model as a weighted sum of basis functions. Radial basis function interpolation can fit local changes well and performs well when processing anisotropic data. It is suitable for image regions with complex textures and drastic local changes. Commonly used radial basis functions include Gaussian functions, multiple quadratic functions, and inverse multiple quadratic functions. The second algorithm is thin plate spline interpolation, which simulates the natural curvature of an infinitely large thin plate at the control points to form a surface at other positions. Thin plate spline interpolation has global smoothness and can ensure the continuity of the mapping model across the entire image plane. It is suitable for flat or gently changing regions.

[0043] The local texture orientation feature in the local illumination regularity parameter is used to guide the anisotropic direction of interpolation. During the interpolation process, a larger smoothing weight is set along the direction parallel to the texture orientation so that the color compensation amount propagates preferentially along the texture direction; a smaller smoothing weight is set along the direction perpendicular to the texture orientation to suppress the propagation of the compensation amount across the texture boundary. The local texture complexity feature in the local illumination regularity parameter is used to guide the anisotropic scale of interpolation. In regions with simple texture and single direction, the anisotropy is enhanced so that the compensation amount is strictly propagated along the texture direction to maintain the integrity of the structure. In regions with complex texture and multiple directions, the anisotropy is weakened so that the interpolation is closer to isotropy and retains rich detail variations. This scale adaptation enables the mapping model to dynamically adjust the smoothing characteristics according to the texture complexity. The local contrast feature in the local illuminance regularity parameter can be used to adjust the weight of feature point pairs in interpolation. Feature points with high contrast are usually located at texture edges or prominent positions, and their illuminance correlation parameters are more reliable and should be given a larger weight. Feature points with low contrast are located in flat areas, and their illuminance correlation parameters may be affected by noise, so they should be given a smaller weight.

[0044] To further improve the prediction accuracy of the mapping model, a multi-basis function construction strategy is also included. This strategy simultaneously employs multiple interpolation algorithms to construct multiple mapping models. Specifically, the position coordinates of each feature point pair in the image are used as input independent variables, and the illumination correlation parameter of each feature point pair is used as input dependent variables. A first mapping model is constructed using a radial basis function interpolation algorithm, and a second mapping model is constructed using a thin-plate spline interpolation algorithm. The first mapping model is sensitive to local changes and is suitable for regions with complex textures; the second mapping model has global smoothness and is suitable for flat regions. Then, based on the prediction errors of the output values ​​of the first and second mapping models at each feature point location relative to the illumination correlation parameter, the adaptive weight coefficients at each feature point location are calculated. The mapping model with a smaller prediction error receives a larger weight coefficient at that location, while the mapping model with a larger prediction error receives a smaller weight coefficient. Based on the adaptive weight coefficients at each feature point location, the fusion weight coefficients at each pixel location in the entire image are obtained through an interpolation algorithm. Then, the outputs of the first and second mapping models are weighted and summed pixel by pixel based on the fusion weight coefficients, and the summation result is used as the final color compensation amount at that pixel location.

[0045] To further improve the local accuracy of color compensation, a region-constrained interpolation strategy can also be included. This strategy applies region-level connectivity constraints to the interpolation process using the region segmentation results of the standard template image. Specifically, firstly, the standard template image is segmented into regions using a superpixel segmentation algorithm to obtain multiple regions with uniform color. The superpixel segmentation algorithm divides the image into several regions with consistent internal color and clear boundaries. Each region corresponds to an image region with similar color attributes, and a unique region label is established for each region. Then, using region labels as boundary markers for the interpolation process, constraints are applied during interpolation calculations. Pixels corresponding to the same region label are interpolated only by feature point pairs falling within that region. Pixels corresponding to different region labels are assigned infinitely large smoothing weights during interpolation calculations, thus prohibiting the propagation of compensation amounts between different regions. Different color regions may be affected by varying degrees of ambient illumination in the image to be corrected, and their color compensation amounts should be calculated independently. For example, red and blue regions in an image may have different response characteristics to changes in light source. Mixing the compensation amounts of the two regions will lead to color distortion. Through region constraints, the compensation amount of each color region is calculated independently without interference, effectively avoiding cross-regional color contamination.

[0046] To further improve the accuracy of color compensation, a local color residual fusion strategy is also included. This strategy incorporates the color residual at feature point locations as an additional constraint term into the interpolation objective function, achieving complementary correction of multi-source information. Specifically, the color residual refers to the color difference between the standard template image and the image to be corrected at various feature point locations, including red channel residual, green channel residual, and blue channel residual. Unlike the illuminance-related parameter, the color residual reflects the color deviation that still exists after preliminary correction and is the target that the mapping model needs to further fit. By incorporating the color residual as an additional constraint term into the objective function of the interpolation algorithm, the objective function of the interpolation algorithm simultaneously includes the fitting term of the illuminance-related parameter and the fitting term of the color residual. The fitting term of the illuminance-related parameter ensures that the mapping model can compensate for the color shift caused by ambient illuminance, while the fitting term of the color residual ensures that the mapping model can further eliminate residual color differences.

[0047] For the color residuals at non-feature point locations, the color residuals at each feature point location are diffused using an interpolation algorithm. The diffusion process can use the same interpolation algorithm as the mapping model to ensure that the distribution of the color residuals has consistent spatial characteristics with the distribution of the mapping model. The diffused color residuals and the color compensation amount output by the mapping model are then weighted and summed pixel by pixel, and the summation result is used as the final color compensation amount for that pixel location. Through this fusion method, the calculation of the color compensation amount utilizes both the illuminance correlation parameter and the color residual information source, which significantly improves the compensation accuracy.

[0048] The color compensation step involves calculating the color compensation amount for each pixel in the transformed image to be corrected based on the mapping model, then adding the color compensation amount to the original pixel value of that pixel, and outputting the corrected image. Specifically, for each pixel position in the transformed image to be corrected, the coordinates of that position are first obtained. Input the coordinates into the mapping model obtained in the mapping function construction step. In the model, the color compensation amount at that location is output; mapping model It is a continuous function whose output is a three-dimensional vector. , respectively representing the position The values ​​that need to be compensated for in the red, green, and blue channels are smoothly transitioned between adjacent pixel positions due to the continuity of the mapping model, without any abrupt jumps.

[0049] After obtaining the color compensation amount, it is superimposed on the original pixel value of that pixel. The superposition operation is performed independently on each channel of the selected color space, resulting in the corrected pixel value. , , .

[0050] In sparse feature point regions or image edge regions, the prediction accuracy of the mapping model may decrease, and the reliability of the color compensation amount may be insufficient. Directly applying unreliable compensation amounts may lead to compensation discontinuities, artifacts, or unnatural color transitions in the corrected image. Therefore, a confidence evaluation strategy is also included, which calculates a confidence coefficient for the color compensation amount at each pixel location. This coefficient reflects the reliability of the compensation amount. Specifically, the Euclidean distance between the pixel location and the nearest feature point pair is calculated. The closer the distance, the more local constraints the pixel is subject to, and the higher the reliability of the interpolation result. The farther the distance, the more the pixel depends on the extrapolation ability of the interpolation algorithm, and the lower the reliability. The spatial distance factor can be converted into a confidence contribution value through the reciprocal of the distance or a Gaussian kernel function.

[0051] Next, obtain the local texture complexity features of the region where the pixel is located. In regions with high texture complexity, the local gradient information is rich, the interpolation algorithm has more constraint information, and the reliability of the color compensation is relatively high. In flat regions with low texture complexity, the local gradient information is scarce, the interpolation algorithm mainly relies on the long-distance propagation of sparse control points, and the reliability is low. The local texture complexity features can be obtained from the feature map that has been calculated in the local gradient analysis step.

[0052] The confidence contributions of the two factors mentioned above are combined by weighted summation or multiplication to obtain the comprehensive confidence coefficient for the pixel location. The confidence coefficient is usually normalized to the range of 0 to 1, with a larger value indicating more reliable compensation. When the confidence coefficient is lower than a preset confidence threshold, it indicates that the reliability of the color compensation at that location is insufficient. In this case, the compensation at that location needs to be corrected. The correction method is to use a weighted average of the color compensation values ​​of pixels with confidence coefficients higher than the threshold in the neighborhood of the pixel location. The weights of the weighted average can be determined based on the spatial distance and the confidence coefficient, with pixels that are closer and have higher confidence coefficients being given greater weights. The corrected color compensation value replaces the original color compensation value and is then superimposed on the original pixel value of the pixel. For pixel locations with confidence coefficients higher than the threshold, the color compensation value output by the mapping model is used directly without correction.

[0053] Taking the scenario of detecting the oil level in the gearbox under a train as an example, such as Figure 4 As shown, the camera needs to capture images of the oil level observation window. Due to the complex environment under the train, poor lighting conditions, and significant differences in shooting time and environment between different trains and maintenance cycles, the acquired observation window images often suffer from severe color temperature shifts and color distortion. This directly affects the accuracy of subsequent extraction of the oil contact line and refraction feature line, thus impacting the accuracy of oil level detection.

[0054] When performing color temperature compensation using the method of this invention, the standard template image is selected as the first clear and color-normal observation window image acquired under standard supplementary lighting conditions, and the image to be corrected is the color-distorted observation window image acquired in subsequent maintenance cycles.

[0055] In the image feature extraction step, feature points are mainly selected from the uniformly painted area of ​​the observation window frame and the flat surface of the gearbox housing. These areas are not affected by oil fluctuations and have stable colors.

[0056] In the geometric transformation step, the image to be corrected and the template image are aligned using a homography matrix to eliminate deviations in the position and angle of the viewing window caused by changes in camera pose. Sub-pixel level alignment accuracy ensures that every pixel within the viewing window corresponds accurately.

[0057] In the local gradient analysis step, the observation window glass surface is a flat texture region with relatively small local texture complexity eigenvalues; the observation window border is an edge region with a clear directionality, and the local texture orientation characteristics are obvious. These characteristics will guide the anisotropic smoothing in the subsequent interpolation process.

[0058] In the mapping function construction step, the anisotropic interpolation strategy ensures that the color compensation amount propagates preferentially along the direction of the viewing window border, without crossing the boundary between the oil region and the air region, thus avoiding color aliasing at the boundary.

[0059] In the color compensation step, the corrected observation window image is output, with a color temperature consistent with the template image. The contrast between the oil area and the air area is clear, providing a high-quality input image for subsequent oil contact line extraction and oil level calculation.

[0060] like Figure 4 As shown, in the scenario of detecting the oil level in the gearbox under the train, due to the small space under the train, complex ambient lighting, and significant differences in shooting conditions at different maintenance cycles, the collected oil observation window images have a serious overall blue or yellow color temperature shift. Before correction, the overall tone of the observation window image deviates from the true color, and some surfaces have highlights. The color contrast between the oil area and the air area is insufficient, and the edge of the oil contact line is blurred, making it difficult for the subsequent oil level detection algorithm to accurately extract the contact line position, resulting in a large error in the liquid level calculation. After color temperature compensation and correction, the color temperature of the observation window image is highly consistent with that of the standard template image. The visual contrast between the oil area and the air area is significantly enhanced, the edge of the oil contact line is clear and sharp, and the liquid level detection algorithm can stably and accurately extract the contact line position. The liquid level calculation error is greatly reduced, effectively solving the problem of misjudgment of liquid level caused by color temperature shift.

[0061] like Figure 5As shown, in the train bellows inspection scenario, the surface to be inspected is a metallic paint surface with directional brushed texture. The surface has local structures such as screws, rivets, and gaps. Before correction, due to the difference in ambient illumination, the bellows surface image has an overall color temperature deviation. At the same time, the color uniformity of different areas is poor. The outlines of key structures such as screws and rivets are not clear enough under the interference of color deviation. Minor defects such as surface scratches and rust are easily covered by color deviation, affecting the accuracy of subsequent defect detection.

[0062] After color temperature compensation and correction, the color of the bellows surface image is accurately restored, the overall color temperature deviation is effectively eliminated, and the color uniformity of each area is significantly improved. At the same time, due to the anisotropic interpolation strategy of this invention, the color compensation amount is strictly propagated along the texture direction, which not only eliminates color deviation but also preserves structural details such as screw edges and gap boundaries. The contrast of surface scratches, rust and other defects is enhanced, providing high-quality image input for subsequent surface defect detection.

[0063] The comparison of the effects before and after correction in the two scenarios above fully demonstrates that the method of the present invention can effectively solve the color temperature shift problem of offline acquired images of rail trains with large differences in illumination and environment. It can achieve accurate color compensation in scenarios with different texture features and structural complexity, while maintaining good detail preservation capabilities.

[0064] The above are merely preferred embodiments of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principle of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. A color temperature compensation method for offline acquired color images, characterized in that: Includes the following steps: The image feature extraction step involves acquiring the image to be corrected and the standard template image, extracting pixels with a local window variance lower than a preset threshold and located in a non-edge region from the two images as feature points, and combining them to form feature point pairs. The geometric transformation step involves transforming the image to be corrected using the estimated homography matrix through feature points, so that the transformed image to be corrected is in the same image coordinate system as the standard template image. The illuminance difference calculation steps involve calculating the pixel value difference between the two feature points in each channel of the color space for each feature point pair as the illuminance correlation parameter for that feature point pair. The local gradient analysis step involves taking each feature point in the transformed image to be corrected as the center, calculating the gradient direction and gradient magnitude of the feature point and its surrounding pixels in each channel of the color space, which are used as the local illumination regularity parameters of the feature point. The mapping function construction steps are as follows: the illumination correlation parameters of all feature point pairs are used as sparse control points, the local illumination regularity parameters of each feature point are used as local constraints, and an interpolation algorithm is used to construct a mapping model from the pixel position of the transformed image to be corrected to the color compensation amount. The color compensation step involves calculating the color compensation amount for each pixel in the transformed image to be corrected based on the mapping model, superimposing the color compensation amount onto the original pixel value of that pixel, and outputting the corrected image. The local gradient analysis step includes an illumination regularity extraction strategy. This strategy involves constructing multiple concentric windows centered on the feature point. In the inner window, the principal gradient direction and gradient magnitude of the feature point in each color channel are calculated. In the outer window, the statistical histogram of the gradient direction and the cumulative distribution function of the magnitude are calculated. The principal gradient direction is used as the local texture orientation feature of the feature point, and the peak distribution width of the statistical histogram of the gradient direction is used as the local texture complexity feature of the feature point. The combination of the local texture orientation feature and the local texture complexity feature is used as the local illumination regularity parameter of the feature point. The mapping function construction step includes an anisotropic interpolation strategy. The anisotropic interpolation strategy includes using the illumination correlation parameter of each feature point pair as the function value of the sparse control point, using the local texture orientation feature in the local illumination regularity parameter of each feature point as the anisotropic direction of the interpolation basis function, using the local texture complexity feature in the local illumination regularity parameter of each feature point as the anisotropic scale of the interpolation basis function, setting a first smoothing weight along the direction parallel to the local texture orientation, setting a second smoothing weight along the direction perpendicular to the local texture orientation, and the value of the first smoothing weight is greater than the value of the second smoothing weight. When the local texture complexity feature increases, the ratio of the first smoothing weight and the second smoothing weight is reduced according to a preset ratio.

2. The color temperature compensation method for offline acquired color images according to claim 1, characterized in that: The illumination regularity extraction strategy also includes using the ratio of the gradient magnitude to the mean magnitude within the outer window as the local contrast feature of the feature point, and using the combination of the local texture orientation feature, the local contrast feature, and the local texture complexity feature as the local illumination regularity parameter of the feature point.

3. The color temperature compensation method for offline acquired color images according to claim 2, characterized in that: The local gradient analysis step also includes a feature point propagation strategy. The feature point propagation strategy includes inserting virtual feature points at equal intervals on the line connecting adjacent feature points when the Euclidean distance between adjacent feature points in the image to be corrected is greater than a preset distance threshold. The local illuminance regularity parameters of the virtual feature points are obtained by linear interpolation or cubic spline interpolation based on the local illuminance regularity parameters of adjacent feature points. When the number of feature points in a certain local area of ​​the image to be corrected is lower than a preset density threshold, new feature points are generated by spreading outward along the gradient direction or iso-illuminance line direction using the original feature points in that area as seed points. The local illuminance regularity parameters of the new feature points are obtained by weighting the local illuminance regularity parameters of the corresponding seed points with the attenuation weight coefficient of the diffusion distance.

4. A color temperature compensation method for offline acquired color images according to claim 2 or 3, characterized in that: The mapping function construction step also includes a region constraint interpolation strategy. The region constraint interpolation strategy includes dividing the standard template image into multiple regions with uniform color by superpixel segmentation algorithm, establishing a region label for each region, and using the region label as the boundary identifier of the interpolation process. Pixels corresponding to the same region label participate in the interpolation calculation by feature point pairs falling within that region. Pixels corresponding to different region labels are assigned infinitely large smoothing weights in the interpolation calculation.

5. The color temperature compensation method for offline acquired color images according to claim 4, characterized in that: The mapping function construction step also includes a local color residual fusion strategy. This strategy involves calculating the color residuals between the standard template image and the image to be corrected at each feature point location. The color residuals include red channel residuals, green channel residuals, and blue channel residuals. The color residuals are added as additional constraint terms to the objective function of the interpolation algorithm. The objective function of the interpolation algorithm includes fitting terms for illumination correlation parameters and fitting terms for color residuals. Based on the color residuals at each feature point location, the color residuals at non-feature point locations are diffused using the interpolation algorithm. The diffused color residuals are then weighted and summed pixel-by-pixel with the color compensation amount output by the mapping model, and the summation result is used as the final color compensation amount for that pixel location.

6. The color temperature compensation method for offline acquired color images according to claim 5, characterized in that: The mapping function construction step also includes a multi-basis function construction strategy. The multi-basis function construction strategy includes using the position coordinates of each feature point pair in the image as the input independent variable, using the illumination correlation parameter of each feature point pair as the input dependent variable, constructing a first mapping model through a radial basis function interpolation algorithm, and constructing a second mapping model through a thin plate spline interpolation algorithm.

7. The color temperature compensation method for offline acquired color images according to claim 6, characterized in that: Based on the prediction error of the output values ​​of the first mapping model and the second mapping model at each feature point location relative to the illumination correlation parameter, the adaptive weight coefficient at each feature point location is calculated. Based on the adaptive weight coefficient at each feature point location, the fusion weight coefficient at each pixel location in the entire image is obtained by diffusion through an interpolation algorithm. Then, the outputs of the first mapping model and the second mapping model are weighted and summed pixel by pixel based on the fusion weight coefficient, and the summation result is used as the final color compensation amount at that pixel location.

8. A color temperature compensation method for offline acquired color images according to claim 2 or 3, characterized in that: The color compensation step includes a confidence evaluation strategy, which includes calculating the confidence coefficient of the color compensation amount at each pixel position based on the spatial distance between each pixel position and the nearest feature point pair, as well as the local texture complexity features of the region where the pixel position is located.

9. The color temperature compensation method for offline acquired color images according to claim 8, characterized in that: The confidence evaluation strategy also includes correcting the color compensation of pixels with confidence coefficients higher than the threshold in the neighborhood of the pixel location by weighted averaging when the confidence coefficient is lower than the preset confidence threshold. The corrected color compensation is then added to the original pixel value of the pixel instead of the original color compensation.

Images