Color Compensation System and Method Based on Adaptive Color Palette

Abstract

This application relates to the field of image processing technology, and provides a color compensation system and method based on adaptive color adjustment. Through the close combination of four steps, from the acquisition of multi-source data, the determination of key parameters, the output of the compensation parameter matrix to the final compensation optimization, each step progresses layer by layer and supports each other, which can effectively improve the color restoration degree of images under different ambient lights and target device conditions, ensure that the output RGB images have accurate and natural colors, conform to human visual perception, and significantly improve the quality and efficiency of image color processing, providing strong technical support for various application scenarios involving image display and processing.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and particularly to a color compensation system and method based on adaptive color adjustment. Background Art

[0002] In the current era where digital images are widely used, the accurate restoration and presentation of image colors are crucial. Whether in photography, film and television production, image display or other related fields, users expect to obtain high-quality and color-realistic image effects. However, most of them do not solve the problem of eliminating residual color differences and color compensation under different ambient lights and device scenarios through image data processing. Summary of the Invention

[0003] In view of the deficiencies of the prior art, this application provides a color compensation system and method based on adaptive color adjustment.

[0004] In a first aspect, this application provides a color compensation method based on adaptive color adjustment. The method includes: obtaining original image data, ambient light data, and the color configuration of the target device through the sampling frequency, aligning the time stamps of the original image data and the ambient light data, where the ambient light data includes color temperature and light intensity, and the color configuration of the target device includes the target device color gamut;

[0005] Identifying color clusters in the original image data with a proportion greater than 15% through a clustering algorithm as the main image color tones, calculating the offset of the color temperature under the light intensity according to the ambient light data to determine the color difference data, determining the reference color based on the main image color tones and the color difference data, and mapping the ambient light data into the original image data to match the image resolution in the original image data, and outputting the light influence factor of each pixel point in the original image data;

[0006] Dividing the sensitive areas in the original image data through a semantic segmentation model, outputting the regional sensitivity of each pixel point in each sensitive area, and at the same time comparing the reference color with the target reference color to obtain the reference color deviation of each pixel point. According to the reference color deviation, light influence factor, and regional sensitivity of each pixel point, combining with the compensation intensity of each pixel point to output a compensation parameter matrix, and compensating the reference color according to the compensation parameter matrix;

[0007] Mapping the compensated reference color to the target device color gamut, outputting an RGB image adapted to the target device, and predicting the color difference data of the sensitive areas in the RGB image through a deep residual network, and performing hierarchical re-compensation according to the change of the color difference data to update the compensation parameter matrix and the sampling frequency.

[0008] As an optional implementation manner, the step of determining the color difference data includes:

[0009] Training historical ambient light data through a recurrent neural network to predict the target color temperature at a light intensity;

[0010] Calculating the deviation between the actual color temperature and the target color temperature to obtain the offset of the color temperature at the light intensity;

[0011] Setting a light intensity influence coefficient according to the light intensity, and determining color difference data based on the light intensity influence coefficient and the offset of the color temperature at the light intensity.

[0012] As an optional implementation manner, the step of determining the reference color includes:

[0013] Calculating the hue weight of each image dominant hue according to the proportion of each image dominant hue in the original image data and the visual saliency of each image dominant hue, and determining the clustering center color of each image dominant hue;

[0014] Calculating the standard deviation between the ambient light data and the historical ambient light data to determine the ambient light weight, and setting a calibration color;

[0015] Determining the reference color by weighted summation of the hue weight and the clustering center color, as well as the ambient light weight and the calibration color.

[0016] As an optional implementation manner, the output step of the light influence factor of each pixel point includes:

[0017] Mapping the ambient light data to the pixel points in the original image data through a ray tracing algorithm, and for each pixel point, outputting the ray propagation path from the ambient light source to the pixel point;

[0018] Judging whether there is an obstacle on each ray propagation path to determine the occlusion coefficient of each pixel point;

[0019] Determining the influence of the light intensity and color temperature in the ambient light data on each pixel point to obtain the influence coefficient of each pixel point;

[0020] Combining the occlusion coefficient and the influence coefficient of each pixel point, and outputting the light influence factor of each pixel point.

[0021] As an optional implementation manner, the output step of the regional sensitivity of each pixel point includes:

[0022] Dividing the sensitive area in the original image data through a semantic segmentation model, and determining the semantic category of each pixel point in each sensitive area through an attention mechanism;

[0023] Extracting a modal feature vector combining a convolutional neural network and a long short-term memory network for each pixel point, and setting an initial sensitivity for each semantic category in combination with the modal feature vector to obtain the initial regional sensitivity of each pixel point;

[0024] Determine the spatial distance and feature similarity between pixel points based on bilateral filtering, and output the regional sensitivity of each pixel point according to the spatial distance and feature similarity between pixel points.

[0025] As an optional implementation manner, the output step of the compensation parameter matrix includes:

[0026] Convert the reference color and the target reference color to the perceptual hash space, and obtain the reference color deviation of each pixel point by calculating the Hamming distance between the perceptual hash values of the reference color and the target reference color.

[0027] Construct reinforcement learning, determine that the state space is the reference color deviation, light influence factor and regional sensitivity of each pixel point, the action space is the weight combination of the reference color deviation, light influence factor and regional sensitivity of each pixel point, and continuously iterate and train the reinforcement learning.

[0028] Calculate the compensation intensity of each pixel point according to the trained weight combination, and output the compensation parameter matrix in combination with the color correction parameter.

[0029] As an optional implementation manner, the determination step of the sampling frequency includes:

[0030] Determine the load factor of the target device according to the CPU computing power and memory bandwidth of the target device, set the reference sampling rate based on the type of the target device, and determine the sampling rate in combination with the load factor of the target device.

[0031] Calculate the change rate of the ambient light based on the change gradient of the color temperature and the change gradient of the light intensity in the historical ambient light data, and correct the sampling rate according to the change rate of the ambient light.

[0032] Determine the image complexity according to the clustering dispersion of the main color of the image and the edge density of the original image data, and optimize the sampling rate according to the image complexity to obtain the finally determined sampling frequency.

[0033] As an optional implementation manner, the execution step of the hierarchical re-compensation includes:

[0034] Layer the RGB image based on the visual saliency and image complexity of the RGB image, and set the compensation priority for each layer.

[0035] Predict the color difference data of the sensitive area in the RGB image through a deep residual network, calculate the average color difference of each layer according to the predicted color difference data, and set the trigger threshold for each layer, and compare the average color difference of each layer with the trigger threshold of each layer to determine whether to trigger the re-compensation of each layer.

[0036] For the layer that triggers re-compensation, adjust the weight combination to re-output the compensation parameter matrix, and update the sampling frequency according to the execution effect of the re-compensation.

[0037] In a second aspect, the present application provides a color compensation system based on adaptive color adjustment. The system includes: obtaining original image data, ambient light data, and the color gamut of the target device through the sampling frequency, and aligning the timestamps of the original image data and the ambient light data;

[0038] Identifying the color clusters in the original image data with a proportion greater than 15% through a clustering algorithm as the main colors of the image, calculating the offset of the color temperature under the illumination intensity according to the ambient light data, determining the color difference data, determining the reference color based on the main colors of the image and the color difference data, and mapping the ambient light data into the original image data to output the light influence factor of each pixel point in the original image data;

[0039] Dividing the sensitive areas in the original image data through a semantic segmentation model to output the area sensitivity of each pixel point in each sensitive area. At the same time, comparing the reference color with the target reference color to obtain the reference color deviation of each pixel point. According to the reference color deviation, light influence factor, and area sensitivity of each pixel point, combining with the compensation intensity of each pixel point to output the compensation parameter matrix, and compensating the reference color according to the compensation parameter matrix;

[0040] Mapping the compensated reference color to the color gamut of the target device to output an RGB image adapted to the target device, and predicting the color difference data of the sensitive areas in the RGB image through a deep residual network. Perform hierarchical re-compensation according to the change of the color difference data to update the compensation parameter matrix and the sampling frequency.

[0041] Compared with the prior art, the beneficial effects of the present application are: through the close combination of four steps, from the acquisition of multi-source data, the determination of key parameters, the output of the compensation parameter matrix to the final compensation optimization, each step progresses layer by layer and supports each other, which can effectively improve the color restoration degree of the image under different ambient light and target device conditions, ensure that the output RGB image has accurate and natural colors, conforms to the human eye visual perception, significantly improves the quality and efficiency of image color processing, and provides strong technical support for various application scenarios involving image display and processing.

[0042] Obtaining the original image data, ambient light data, and the color configuration of the target device through the sampling frequency and performing timestamp alignment provides a comprehensive and synchronous data basis for subsequent color compensation. The collaboration of data from different sources enables color compensation to comprehensively consider the characteristics of the image itself, the influence of ambient light, and the display capabilities of the target device, avoiding color compensation deviations caused by data loss or asynchronization.

[0043] Identify the main color of the image, determine the color difference data and the reference color, and calculate the light influence factor. Extract key parameters from both the image content and the ambient light factors. These parameters are closely related to the essential characteristics of the image color and the environmental impact, providing the core basis for the calculation of subsequent compensation parameters, enabling color compensation to be carried out based on the actual situation of the image and the environmental conditions, and significantly improving the pertinence and accuracy of the compensation.

[0044] Divide the sensitive areas and determine the area sensitivity. Combine the reference color deviation and the light influence factor to calculate the compensation parameter matrix, providing an accurate operation direction for color compensation. Generate personalized compensation parameters according to different area sensitivities and color deviation situations, achieve precise compensation for each part of the image, effectively improve the overall color quality of the image, and highlight the color performance of important areas.

[0045] Map the compensated reference color to the target device color gamut to ensure accurate display of the output image on the target device. Predict the color difference data through a deep residual network and perform hierarchical re-compensation, dynamically monitor and optimize the color compensation effect, further improve the image color quality, and at the same time update the sampling frequency according to the re-compensation effect, achieve adaptive matching of data acquisition and compensation requirements, and improve the overall performance and adaptability of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings. Among them:

[0047] Figure 1 is the method flow chart of the color compensation method based on adaptive color adjustment provided by the embodiment of the present application;

[0048] Figure 2 is the output step diagram of the light influence factor of each pixel point of the color compensation method based on adaptive color adjustment provided by the embodiment of the present application;

[0049] Figure 3 is the execution step diagram of the hierarchical re-compensation of the color compensation method based on adaptive color adjustment provided by the embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0050] In order to make the objectives, technical solutions, and advantages of the embodiments of the present application more obvious and understandable, the following clearly and completely describes the technical solutions in the embodiments of the present application with reference to the accompanying drawings of the specification. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments.

[0051] Embodiment 1

[0052] As shown in Figure 1 the figure, the method flow chart of the color compensation method based on adaptive color adjustment provided by the embodiment of the present application is as follows. The method includes:

[0053] S1. Obtain the original image data, ambient light data, and the color configuration of the target device through the sampling frequency, align the original image data and the ambient light data in terms of time stamps. The ambient light data includes the color temperature and the light intensity, and the color configuration of the target device includes the target device color gamut.

[0054] Among them, the original image data is the unprocessed image data directly obtained from the camera, usually represented in the form of a pixel matrix, containing the color and brightness information of the image; the ambient light data refers to the light information in the surrounding environment, including the color temperature and the light intensity. The color temperature reflects the color characteristics of the light, and the light intensity reflects the brightness of the light. An ambient light sensor is used to obtain the ambient light data; while the color configuration of the target device refers to the color range and characteristics that the target device can display, mainly including the target device color gamut. The target device color gamut defines the set of colors that the target device can display, and the color configuration information of the target device can be obtained by querying the technical specification document of the target device; time stamp alignment means synchronizing the original image data and the ambient light data in terms of time to ensure that the original image data and the ambient light data obtained at the same time point are corresponding. When obtaining the original image data and the ambient light data, the time stamp of each data is recorded at the same time. In subsequent processing, the corresponding image data and ambient light data are matched through the time stamp, such as using methods like linear interpolation or nearest neighbor interpolation to align the time stamps.

[0055] The steps for determining the sampling frequency include:

[0056] Determine the load factor of the target device according to the CPU computing power and memory bandwidth of the target device, set the reference sampling rate based on the type of the target device, and determine the sampling rate in combination with the load factor of the target device;

[0057] Calculate the change rate of the ambient light based on the change gradients of the color temperature and the light intensity in the historical ambient light data, and correct the sampling rate according to the change rate of the ambient light;

[0058] Determine the image complexity according to the clustering dispersion of the main color of the image and the edge density of the original image data, and optimize the sampling rate according to the image complexity to obtain the finally determined sampling frequency.

[0059] Different target devices have different computing capabilities and resource limitations. The CPU computing power and memory bandwidth directly affect the speed and ability of the device to process data. If the sampling frequency is too high, the device cannot process all the data in time, resulting in system lag or even crashes. If the sampling frequency is too low, it cannot capture the changes in ambient light and images in time. Therefore, it is necessary to reasonably adjust the sampling rate according to the device's load conditions. At the same time, different types of devices have differences in usage scenarios and performance. Setting a baseline sampling rate can provide a basis for subsequent adjustments.

[0060] Obtain the CPU usage rate and memory usage rate of the target device through system monitoring tools. The load factor of the target device is obtained by weighted summing the CPU usage rate and memory usage rate. Classify according to the type of the target device, such as mobile phones, tablets, and computers, etc. Through experiments and tests, set a suitable baseline sampling rate for each type of device. For example, the baseline sampling rate of a mobile phone is set to 10Hz, that of a tablet is 8Hz, and that of a computer is 12Hz. Then the sampling rate is obtained by multiplying the difference between 1 and the load factor by the baseline sampling rate. Thus, it can dynamically adjust the sampling rate according to the actual load conditions of the device, avoid performance problems caused by insufficient processing capacity of the device, and at the same time ensure that the sampling rate can be appropriately increased when the device load is low to obtain data more timely.

[0061] The change situation of ambient light is an important factor affecting image color. If the ambient light changes rapidly, a higher sampling frequency is required to capture these changes in time for accurate color compensation. If the ambient light changes slowly, the sampling frequency is appropriately reduced to reduce the amount of data processing. Record the ambient light data for a period of time (such as the past 10 seconds), including color temperature and light intensity. Calculate the change gradient of color temperature and the change gradient of light intensity. The change gradient of color temperature is obtained by dividing the color temperature difference value in this time period by the time interval, and the change gradient of light intensity is obtained by dividing the light intensity difference value in this time period by the time interval. And obtain the change rate of ambient light by weighted summing the change gradient of light intensity and the change gradient of color temperature. Set a change rate threshold. When the change rate of ambient light is greater than the change rate threshold, correct the sampling rate. The corrected sampling rate = sampling rate × (1 + (change rate of ambient light - change rate threshold) / change rate threshold). Thus, it can dynamically adjust the sampling rate according to the actual change situation of ambient light, ensure that data can be obtained in time when the ambient light changes violently, and improve the accuracy of color compensation.

[0062] Image complexity is also an important factor affecting the sampling frequency. Complex images contain more details and variations, requiring a higher sampling frequency to accurately capture this information. Simple images, on the other hand, require an appropriately reduced sampling frequency. The K-means algorithm is used to cluster the original image data to obtain the main color tone of the image. The average distance between each cluster center and other cluster centers is calculated as the cluster dispersion. The larger the cluster dispersion, the more dispersed the distribution of the main color tone of the image, and the more complex the image. The Canny edge detection algorithm is used to perform edge detection on the original image data to obtain an edge image. The ratio of the number of edge pixels in the edge image to the total number of pixels is calculated as the edge density. The larger the edge density, the richer the edge information of the image, and the more complex the image. The image complexity is obtained by weighted summation of the cluster dispersion and the edge density. A complexity threshold is set. When the image complexity is greater than the complexity threshold, it is determined as a complex scene, and the optimized sampling rate = the corrected sampling rate × (1 + (image complexity - complexity threshold) / (1 - complexity threshold)). Otherwise, it is a simple scene, and the corrected sampling rate does not need to be optimized. Thus, the sampling rate can be dynamically adjusted according to the actual complexity of the image, ensuring that more detailed information can be obtained when processing complex images, improving the effect of color compensation. The finally determined sampling frequency will determine the frequency of subsequent acquisition of the original image data, thereby affecting the recognition of the main color tone of the image and the calculation of color difference data in subsequent steps.

[0063] S2. Identify the color clusters in the original image data with a proportion greater than 15% through a clustering algorithm as the main color tone of the image. Calculate the offset of the color temperature under the illumination intensity according to the ambient light data to determine the color difference data. Determine the reference color based on the main color tone of the image and the color difference data, and map the ambient light data to the original image data to match the image resolution in the original image data, and output the light influence factor of each pixel point in the original image data.

[0064] Using the K-means clustering algorithm, first determine the number of clusters K based on experience or preliminary analysis, and determine a suitable value through multiple experiments. For example, for general scene images, K is set to 5-10. Then, according to the initialization rule of the K-means algorithm, select pixel colors that are far apart as the initial cluster centers. For each pixel point in the original image data, calculate its distance from each cluster center. In the RGB color space, the Euclidean distance can be used. Assign the pixel point to the cluster where the nearest cluster center is located, recalculate the average value of the pixel colors within each cluster, and update the cluster centers. Repeat the above assignment and update process until the cluster centers no longer change significantly. For example, the moving distance of the cluster centers in consecutive multiple iterations is less than a certain threshold. Finally, count the proportion of the number of pixel points in each cluster to the total number of pixel points, and take the color corresponding to the cluster with a proportion greater than 15% as the main color tone of the image. For example, after clustering, it is found that the number of pixel points in a certain cluster accounts for 20% of the total number of pixels, then the average color of this cluster is a main color tone of the image.

[0065] The steps for determining the color difference data include:

[0066] Train the historical ambient light data through a recurrent neural network to predict the target color temperature under the illumination intensity;

[0067] Calculate the deviation between the actual color temperature and the target color temperature to obtain the offset of the color temperature under the illumination intensity;

[0068] Set the light intensity influence coefficient according to the illumination intensity, and determine the color difference data based on the light intensity influence coefficient and the offset of the color temperature under the illumination intensity.

[0069] There is a complex correlation between the color temperature and the illumination intensity in the ambient light data, and this relationship changes with time and the environment. Using a recurrent neural network can effectively capture the time series characteristics in the historical ambient light data, so as to accurately predict the target color temperature under the current illumination intensity, which can provide a reference color temperature value that is more in line with the actual environmental changes for subsequent color difference calculation; continuously collect ambient light data for a period of time (such as the past week), including color temperature and illumination intensity, and record a data point every 5 minutes. Normalize the collected data so that the values of color temperature and illumination intensity are both within the range of 0-1, which is convenient for the RNN model to process. Select the long short-term memory network, which is a variant of the RNN and can better handle long-term dependence problems. The model structure of the long short-term memory network includes an input layer, several LSTM hidden layers, and an output layer. The input layer receives the normalized historical illumination intensity sequence, the hidden layer learns the relationship between the illumination intensity and the color temperature through LSTM units, and the output layer outputs the predicted target color temperature. For example, the dimension of the input layer is 1 because only the illumination intensity is input, two hidden layers are set, each hidden layer has 64 LSTM units, and the dimension of the output layer is 1, which outputs the predicted color temperature.

[0070] The preprocessed data is divided into a training set and a test set in chronological order. For example, 80% of the data is used for training and 20% for testing. The mean squared error is used as the loss function. During the training process, the historical light intensity sequence is used as the input, and the corresponding actual color temperature is used as the label. The model parameters of the long short-term memory network are continuously updated through the backpropagation algorithm to minimize the error between the predicted value and the actual value of the long short-term memory network model. The training process is set to iterate through the training set 50 times. Through the powerful time series processing ability of the RNN, it is able to accurately predict the target color temperature under the light intensity, taking into account the dynamic changes of the ambient light data over time, providing a more reliable reference for subsequent color difference calculation.

[0071] By calculating the deviation between the actual color temperature and the predicted target color temperature, it is possible to quantify the difference between the ambient light color temperature and the expected value under the current light intensity. This difference is one of the key factors in determining the color difference data and can reflect the direction and degree of the impact of ambient light on the image color. The introduction of the light intensity influence coefficient takes into account the non-linear relationship between the light intensity and the color temperature deviation, making the calculation result more in line with the perceptual characteristics of the human eye for colors under different light intensities. Subtracting the target color temperature predicted by the RNN from the actual color temperature gives the deviation between the actual color temperature and the target color temperature. The setting of the light intensity influence coefficient is obtained through calculation. represents the light intensity, and are constants obtained through experimental fitting; thus, the offset of the color temperature under the light intensity is clearly quantified, and the influence of the light intensity on the perception of color temperature deviation is considered through the light intensity influence coefficient, providing the necessary data for accurate color difference calculation.

[0072] The light intensity has a greater impact on the human eye's perception of color. Under different light intensities, the same change in color temperature will result in different degrees of color perception differences. By setting the light intensity influence coefficient and combining the offset of the color temperature under the light intensity to determine the color difference data, it is possible to more comprehensively consider the comprehensive impact of ambient light on the image color and improve the accuracy of color compensation. Multiplying the light intensity influence coefficient obtained in the previous step by the offset of the color temperature under the light intensity gives the color difference data; thus, the impacts of both the light intensity and the color temperature offset on the color are comprehensively considered, making the calculated color difference data better reflect the changes in the image color under the actual ambient light, which helps to perform more accurate color compensation subsequently. Accurate color difference data is an important basis for determining the reference color, and it directly affects the accuracy of determining the reference color based on the main color tone and color difference data of the image and performing color compensation based on the reference color.

[0073] The steps for determining the reference color include:

[0074] Calculate the hue weight of each dominant hue of the image according to the proportion of the dominant hue of each image in the original image data and the visual saliency of each dominant hue of the image, and determine the clustering center color of each dominant hue of the image;

[0075] Calculate the standard deviation between the ambient light data and the historical ambient light data to determine the ambient light weight, and set the calibration color;

[0076] Perform weighted summation on the hue weight and the clustering center color, as well as the ambient light weight and the calibration color to determine the reference color.

[0077] The proportion of the dominant hue of the image in the image reflects its importance in the overall color composition of the image, while the visual saliency reflects the prominence of the color in the visual perception of the human eye. Calculating the hue weight by combining these two factors can more reasonably determine the contribution of each dominant hue when determining the reference color. Determine the clustering center color as the representative color of the dominant hue for subsequent weighted calculations, making the determination of the reference color more representative; First, cluster the original image data through the K-means algorithm, and identify the color clusters with a proportion greater than 15% as the dominant hues of the image. Suppose there are dominant hues of the image. For the th dominant hue of the image, its proportion in the original image data is , and the visual saliency is obtained by calculating factors such as the contrast between the dominant hue of the image and the surrounding colors and the spatial distribution in the original image data, denoted as . The hue weight is calculated through the formula , where is a balance coefficient that needs to be dynamically adjusted according to the types of different original image data. During the clustering process, the K-means algorithm will automatically determine the center of each cluster. For the cluster corresponding to each dominant hue of the image, use the color value of the center of this cluster as the clustering center color; thus, comprehensively consider the proportion and visual saliency of the dominant hue of the image to calculate the weight, making the determination of the reference color more in line with the human eye's perception of the image color and the color composition of the image itself, and improving the accuracy and representativeness of the reference color.

[0078] The stability of ambient light has an important impact on image color. By calculating the standard deviation of the ambient light data and the historical ambient light data, the degree of change of the current ambient light can be measured. The smaller the change in ambient light, the greater the influence weight on the determination of the reference color; conversely, the smaller it is. The calibration color is set to eliminate the errors of the sensor itself and some systematic errors in ambient light measurement, so that the ambient light data can more accurately reflect the influence of the actual ambient light on the image color; calculate the standard deviation of the current ambient light data and the historical ambient light data. First, calculate the standard deviation of color difference and the standard deviation of light intensity, and perform weighted summation calculation on the standard deviation of color difference and the standard deviation of light intensity to obtain the standard deviation of the current ambient light data and the historical ambient light data. Set the standard deviation threshold, then the ambient light weight = 1 / (1 + e ^ (standard deviation - standard deviation threshold)). At the same time, during the calibration process, use a standard light source to measure the ambient light sensor multiple times, record the output data of the sensor under the standard light source, and establish a calibration table based on these data. When the current ambient light data is obtained, through querying the calibration table and performing interpolation calculation, the calibrated ambient light data is obtained, and the corresponding color is used as the calibration color; thus, the ambient light weight is determined through the standard deviation, which can dynamically adjust the role of ambient light in the determination of the reference color, improve the adaptability to changes in ambient light, and the setting of the calibration color improves the accuracy of the ambient light data, thereby enhancing the accuracy of the determination of the reference color.

[0079] Combining the hue weight and clustering center color of the main hue of the image, as well as the ambient light weight and calibration color, the reference color is determined by weighted summation, which can comprehensively consider the color characteristics of the image itself and the influence of ambient light on color, making the determined reference color more in line with the color that the actual image should present under the current ambient light, providing a more accurate basis for subsequent color compensation. Determining the reference color by integrating both the original image data and ambient light factors improves the accuracy and adaptability of the reference color, helps to enhance the effect of subsequent color compensation, enables the original image data to more accurately restore colors under different ambient lights, and an accurate reference color is the basis for subsequent color compensation, which has a decisive impact on determining the compensation parameter matrix for each pixel point and the color accuracy of the final output RGB image adapted to the target device.

[0080] Ambient light data is the measured value of ambient light, and the original image data has a specific resolution, that is, the number of pixels in the horizontal and vertical directions of the image. Mapping the ambient light data into the original image data means making the ambient light data correspond to each pixel point in the original image data, so as to analyze the influence of ambient light on the color of each pixel. Matching the image resolution means reasonably distributing the ambient light data to each pixel position according to the pixel distribution of the image, ensuring that the ambient light data is spatially aligned with the image data.

[0081] Ambient light data is acquired through one or more ambient light sensors, which are located near the image acquisition device or at specific positions in the scene. First, the relative position relationship between the ambient light sensor and the image acquisition device is determined. If there are multiple sensors, bilinear interpolation is used to estimate the ambient light data at the position of each pixel in the original image data. For a certain pixel in the original image data, find its position relationship in the grid of sensor measurement values. For example, let the spacing of sensor measurement values in the horizontal direction be and the spacing in the vertical direction be . The distance of this pixel from the nearest measurement point on the left in the horizontal direction is , and the distance from the nearest measurement point on the lower side in the vertical direction is . Then, the ambient light data of this pixel is calculated by bilinear interpolation using the ambient light data of the four surrounding measurement points. In this way, the ambient light data is mapped to each pixel of the original image data, achieving the matching of the ambient light data and the original image data in terms of resolution.

[0082] As Figure 2 shown, the output steps of the light influence factor for each pixel include:

[0083] Map the ambient light data to the pixels in the original image data through the ray tracing algorithm. For each pixel, output the light propagation path from the ambient light source to the pixel.

[0084] Determine whether there are occluders on each light propagation path to determine the occlusion coefficient of each pixel.

[0085] Determine the influence of the light intensity and color temperature in the ambient light data on each pixel to obtain the influence coefficient of each pixel.

[0086] Combine the occlusion coefficient and influence coefficient of each pixel to output the light influence factor of each pixel.

[0087] Before the ambient light reaches the image pixel, its propagation path will be affected by various objects in the scene, including reflection, refraction, and occlusion, etc. Only by clarifying the specific path of the light from the light source to the pixel can the actual effect of the ambient light on each pixel be accurately analyzed, and thus provide a key basis for calculating the light influence factor. Through the ray tracing algorithm, such as the path tracing algorithm, first build a scene model. Model the ambient light source as a luminous body with a specific position, intensity, and spectral distribution. The objects in the scene where the original image data is located are defined according to their geometric shapes and material properties. The geometric shape refers to the polygonal object represented by the triangular mesh, and the material properties include characteristics such as diffuse reflection, specular reflection, and refraction. For each pixel of the original image data, emit a ray in the direction of the ambient light source.

[0088] In the process of light propagation, spatial data structures (such as octrees) are used to accelerate the intersection detection between light and objects. When light intersects with an object, the reflection and refraction directions of the light are calculated according to the material properties of the object and the corresponding optical model. The optical model can be used to calculate the reflection and refraction ratios through the Fresnel equation, and continue to track the newly generated light. At each intersection, the intersection position, surface normal, and change in the direction of light propagation are recorded until the light leaves the scene or reaches the maximum tracking depth. For example, in an indoor scene, the light starts from the light source located on the ceiling and reaches a pixel in the original image data after a diffuse reflection of the wall. The optical model will record detailed information such as the position of the reflection point of the light on the wall and the direction of the light before and after the reflection. This accurately simulates the propagation process of light in complex scenes, takes into account a variety of optical phenomena, and provides real and reliable light propagation path information for the subsequent accurate calculation of the occlusion coefficient and the influence coefficient, significantly improving the accuracy of the calculation of the light influence factor, thereby improving the accuracy of color compensation, which is especially suitable for scenes with complex lighting environments.

[0089] The presence of obstructions will change the intensity and color of ambient light reaching the pixel point, which is an important factor affecting the light impact factor. Accurately judging whether there are obstructions on the light propagation path and quantifying the degree of obstruction, that is, determining the obstruction coefficient, can more realistically reflect the actual effect of ambient light on the pixel point, thereby improving the accuracy of light impact factor calculation and making the color compensation result more consistent with the actual scene; in the ray tracing process, when the light intersects with an object in the scene, it is determined whether the object is an obstruction. For opaque objects, it is directly determined that the light is completely blocked. For translucent objects, the light transmission ratio is calculated based on the transmittance of its material and the angle between the light and the surface of the object. The transmittance can be obtained through the optical property parameters of the object material. For example, the transmittance of glass material is usually between 0.8-0.95. Assuming the transmittance of the object is , the cosine of the angle between the light and the surface of the object is , is the angle between the light and the normal line of the object surface, then the calculation formula of the occlusion coefficient is For example, for a piece of glass with a transmittance of 0.9 and light incident at an angle of 30° to the surface normal, cos30°≈0.866, then the occlusion coefficient z=1−0.9×0.866≈0.22; thus, the effect of the obstruction on the propagation of light is taken into account, and the degree of attenuation of the ambient light reaching the pixel point can be accurately quantified according to different occlusion conditions, so that the light impact factor more truly reflects the actual lighting conditions, and the accuracy and reliability of color compensation in complex scenes are improved.

[0090] Illumination intensity and color temperature are the core elements of ambient light affecting image color. Quantifying their influence on each pixel to obtain the influence coefficient can accurately reflect the effect of ambient light on the pixel color in these two key aspects, thereby precisely calculating the light influence factor and providing the necessary data support for achieving accurate color compensation; determining the influence of illumination intensity and color temperature in the ambient light data on each pixel to obtain the influence coefficient of each pixel, and the influence coefficient of each pixel = illumination intensity × (1 + (color temperature - standard color temperature) / (maximum color temperature - standard color temperature)), where the standard color temperature refers to the color temperature of the standard light source and the maximum color temperature refers to the maximum color temperature on the target device; thus comprehensively considering the effect of the key characteristics of ambient light on image color, enabling the influence coefficient to accurately reflect the degree of influence of ambient light on pixel color, providing reliable data for precisely calculating the light influence factor, and significantly improving the accuracy and scientific nature of color compensation.

[0091] The occlusion coefficient and the influence coefficient are quantified from two aspects: the occlusion situation during the light propagation process and the influence of the characteristics of ambient light itself on pixel color. Combining them can comprehensively and accurately reflect the comprehensive influence of ambient light on each pixel, thereby obtaining the light influence factor and providing the core data for subsequent color compensation based on ambient light factors; multiplying the occlusion coefficient of each pixel by the influence coefficient of each pixel to obtain, and storing the calculated light influence factor in one-to-one correspondence with each pixel in the original image data to form a matrix of light influence factors for subsequent use in the color compensation process; thus comprehensively considering the occlusion factor in light propagation and the ambient light characteristic factor, comprehensively and accurately quantifying the influence of ambient light on each pixel, and the obtained light influence factor can provide accurate ambient light-related parameters for color compensation, effectively improving the accuracy and adaptability of color compensation, enabling the image to more accurately restore color under different ambient light conditions. The light influence factor is one of the important bases for calculating the compensation parameter matrix subsequently, and together with factors such as the deviation from the reference color and the regional sensitivity, it determines the compensation intensity and method for each pixel, playing a crucial role in finally outputting an RGB image with accurate color adapted to the target device.

[0092] S3. Divide the sensitive regions in the original image data through a semantic segmentation model, output the regional sensitivity of each pixel in each sensitive region, and at the same time compare the reference color with the target reference color to obtain the reference color deviation of each pixel. According to the reference color deviation, light influence factor, and regional sensitivity of each pixel, combine with the compensation intensity of each pixel to output the compensation parameter matrix, and compensate the reference color according to the compensation parameter matrix.

[0093] The output steps of the regional sensitivity of each pixel include:

[0094] The semantic segmentation model is used to divide the sensitive areas in the original image data, and the attention mechanism is used to determine the semantic category of each pixel in each sensitive area;

[0095] Extract the modal feature vector of the convolutional neural network combined with the long short-term memory network for each pixel, and set the initial sensitivity for each semantic category based on the modal feature vector to obtain the initial regional sensitivity of each pixel;

[0096] The spatial distance and feature similarity between pixels are determined based on bilateral filtering, and the regional sensitivity of each pixel is output according to the spatial distance and feature similarity between pixels.

[0097] Different regions have different importance and requirements for color accuracy in the original image data. The semantic segmentation model can divide the original image data into regions with different semantic meanings, such as people, sky, and ground. The attention mechanism can further accurately focus on the uniqueness and importance of each pixel in its semantic category, providing a semantic basis for the subsequent setting of regional sensitivity, so that color compensation can give priority to important areas and key pixels. CNN is used as a semantic segmentation model. During the training process, a large-scale image dataset is used. These images not only mark the bounding boxes of objects, but also mark the semantic categories of each pixel in detail. In the feature extraction stage of the semantic segmentation model, the self-attention module is introduced to allow the semantic segmentation model to fully consider the original image when processing each pixel. The information of pixel points at other positions in the image data can be used to better understand the role of pixel points in the overall semantics. When the original image data is semantically segmented, the semantic segmentation model outputs the semantic category to which each pixel point belongs, such as "person face" and "background building". For example, for a person landscape photo, the semantic segmentation model can accurately mark the pixel points on the person's face as the "person face" category, and mark the pixel points of the distant mountains as the "natural landscape mountain range" category; thereby, the sensitive areas in the original image data can be accurately divided and the semantic category of each pixel point can be determined, which provides a reliable basis for the subsequent setting of regional sensitivity according to semantic importance, improves the pertinence and effectiveness of color compensation, and can highlight the color compensation effect of important areas, especially in complex scene images.

[0098] The features of the pixels in the original image data contain not only spatial information, but also time or context-related information. Combining these two features can more comprehensively describe the characteristics of the pixels. According to the different requirements of different semantic categories for color compensation, these features are combined to set the initial sensitivity for each semantic category, which can preliminarily distinguish the importance of different areas and provide a basis for subsequent optimization. A joint model consisting of a convolutional neural network and a long short-term memory network is constructed. For each pixel, CNN is first used to extract features from the local image block (such as a 3×3 or 5×5 pixel block centered on the pixel). After multiple layers of convolution and pooling operations, CNN obtains a vector containing spatial features. These local image blocks are then input into LSTM in a certain order (such as from left to right or from top to bottom). LSTM captures the contextual dependencies between pixels and outputs a vector containing contextual features. The modal feature vector of each pixel is obtained by concatenating the vector and the vector containing the contextual features. According to the semantic category, for example, for the "human face" semantic category, due to its extremely high requirements for color accuracy, the initial sensitivity is set to 0.8-1.0, and for the "background grass" semantic category, the initial sensitivity is set to 0.3-0.5. For each pixel, according to its semantic category, fine-tuning is performed in combination with its modal feature vector. For example, for a pixel belonging to the "human face" semantic category, if its modal feature vector shows that the pixel is located in a key part of the face, such as near the eyes or lips, its initial sensitivity is set to a value close to 1.0, such as 0.95; thereby, the spatial and contextual features of the pixel are comprehensively considered, and the initial sensitivity is set in combination with the semantic category, so that the sensitivity setting is more scientific and reasonable, which provides a good starting point for subsequent optimization, helps to improve the accuracy of regional sensitivity, and thus improves the effect of color compensation.

[0099] Adjacent pixel points usually have similarities in terms of space and features, and this similarity is related to the regional sensitivity. Based on bilateral filtering, considering both the spatial distance and feature similarity between pixel points, it is possible to smooth and optimize the initial regional sensitivity, making it more in line with the actual characteristics of the image, avoiding sudden changes in sensitivity, and improving the accuracy and stability of the regional sensitivity. For each pixel point, the spatial distance between adjacent pixel points is determined according to the image resolution and actual effect, and the feature similarity between pixel points is calculated through the cosine similarity algorithm. The regional sensitivity of each pixel point is obtained by weighted summing the initial sensitivities between pixel points. For example, for a pixel point located at the edge of the original image data, some of the pixel points in its neighborhood belong to different semantic categories. Through bilateral filtering, combining the spatial distance and feature similarity, and integrating the initial sensitivities of the neighboring pixel points, a more reasonable regional sensitivity of this pixel point is obtained. Thus, the initial regional sensitivity is optimized using bilateral filtering, fully considering the spatial and feature relationships between pixel points, making the regional sensitivity more smoothly and accurately reflect the image characteristics, improving the quality of the regional sensitivity, providing a more reliable basis for accurately calculating the compensation parameter matrix and performing color compensation later. Accurate regional sensitivity is one of the important factors for calculating the compensation parameter matrix. Together with the deviation from the reference color and the light influence factor, it determines the compensation intensity, directly affects the accuracy of the compensation parameter matrix, and further affects the accuracy and effect of color compensation.

[0100] The output steps of the compensation parameter matrix include:

[0101] Convert the reference color and the target reference color to the perceptual hash space, and obtain the deviation of the reference color of each pixel point by calculating the Hamming distance between the perceptual hash values of the reference color and the target reference color.

[0102] Construct reinforcement learning, determine the state space as the deviation of the reference color, the light influence factor, and the regional sensitivity of each pixel point, the action space as the weight combination of the deviation of the reference color, the light influence factor, and the regional sensitivity of each pixel point, and continuously iterate and train the reinforcement learning.

[0103] Calculate the compensation intensity of each pixel point according to the obtained weight combination, and output the compensation parameter matrix in combination with the color correction parameters.

[0104] Traditional color space distance metrics cannot accurately reflect the human eye's perception of color differences. The perceptual hash space can convert colors into a perception-based hash value. By calculating the Hamming distance between the hash values, it can more conform to the human visual perception to measure the difference between the reference color and the target reference color, providing more reasonable color deviation data for determining the compensation intensity later. Through the perceptual hash algorithm, the reference color and the target reference color are converted from the RGB color space to the CIELAB color space to better correlate with the human visual perception. For the colors in the CIELAB color space, they are represented as an image block, such as an 8×8 pixel block. Assuming that the representations of the reference color and the target reference color in the image can be approximated by such small blocks, perform a DCT transform on the image block to obtain frequency domain coefficients, and retain the low-frequency coefficients because the low-frequency coefficients contain the main structural information of the image and have a greater impact on color perception. Then, perform quantization and binarization processing on the low-frequency coefficients to generate a perceptual hash value. For example, compare the low-frequency coefficients with a threshold, set those greater than the threshold to 1 and those less than the threshold to 0, thus obtaining a binary hash value sequence. For each pixel point, calculate the Hamming distance between the perceptual hash values of its reference color and the target reference color, that is, the number of different bits in the two hash values. For example, if the perceptual hash value of the reference color is "10101010" and the perceptual hash value of the target reference color is "10001010", then the Hamming distance is 2. Thus, by calculating the reference color deviation through the perceptual hash space and the Hamming distance, it can more accurately reflect the human eye's perception of color differences, providing color deviation data that better meets the visual needs for color compensation and improving the visual effect of color compensation.

[0105] Different pixel points vary in terms of reference color deviation, light influence factor, and regional sensitivity. The traditional fixed-weight method cannot adaptively adjust according to the characteristics of each pixel point. Reinforcement learning can automatically find the optimal weight combination through continuous trial and error and learning based on the information in the state space to achieve the best color compensation effect. The Q-learning algorithm is used as the basis of reinforcement learning. The state space consists of the reference color deviation, light influence factor, and regional sensitivity of each pixel point, and the action space consists of different weight combinations, corresponding to the weights of the reference color deviation, light influence factor, and regional sensitivity respectively. The reward function is defined as the similarity metric between the compensated image and the reference high-quality image, such as the structural similarity index. During the training process, starting from an initial state, select an action (weight combination), and calculate the compensation intensity according to the compensation intensity function where and are preset constants, represents the th pixel point's regional sensitivity, represents the th pixel point's reference color deviation. Indicates the light influence factor of the nth pixel point, compensates the reference color to obtain a compensated image, calculates the structural similarity index between the compensated image and the reference image as a reward, updates the Q-value table according to the Q-learning algorithm, continuously iterates this process, traverses a large number of pixel point states, makes the Q-value table converge, and finds the optimal action (weight combination) in each state; thus, automatically finds the optimal weight combination through reinforcement learning, can perform adaptive adjustment according to the unique characteristics of each pixel point, significantly improves the effect and accuracy of color compensation, and is more flexible and intelligent than the traditional fixed weight method.

[0106] According to the optimal weight combination for each pixel point obtained through reinforcement learning, combines with the compensation intensity function to calculate the compensation intensity, can achieve precise compensation for each pixel point, forms a compensation parameter matrix in combination with the color correction parameters, and provides a complete parameter basis for the subsequent compensation of the reference color; for each pixel point, substitutes the trained weight combination into the compensation intensity function to calculate the compensation intensity, and at the same time determines the correction parameters of each pixel point on the RGB color channels according to the requirements of color compensation, organizes the compensation intensity and color correction parameters into a compensation parameter matrix, and each row of the matrix corresponds to the compensation parameter information of a pixel point. Assuming the image has N pixel points, the compensation parameter matrix is an N×4 matrix; thus, realizes precise compensation according to the characteristics of each pixel point. The compensation parameter matrix provides comprehensive and accurate parameters for color compensation, can effectively improve the accuracy and effect of color compensation, makes the output image more in line with expectations in terms of color. The compensation parameter matrix is the direct basis for compensating the reference color, determines the compensation method and degree, plays a decisive role in the final output of a high-quality RGB image adapted to the target device, and at the same time affects the evaluation of the compensation effect and parameter update in hierarchical re-compensation.

[0107] S4. Maps the compensated reference color to the target device color gamut, outputs an RGB image adapted to the target device, and predicts the color difference data of the sensitive area in the RGB image through a deep residual network, and performs hierarchical re-compensation according to the change of the color difference data to update the compensation parameter matrix and the sampling frequency.

[0108] The compensated reference color is the color reference of the image after color compensation in the previous steps. The target device color gamut is the color range that the target device can display. Mapping the compensated reference color to the target device color gamut is to adjust the color values obtained through compensation to the color range that the target device can display, so as to ensure that the output RGB image can be accurately displayed on the target device and avoid color distortion or exceeding the device's display ability.

[0109] As Figure 3 shown, the execution steps of hierarchical re-compensation include:

[0110] The RGB image is stratified based on the visual saliency and image complexity of the RGB image, and the compensation priority of each layer is set;

[0111] The color difference data of the sensitive areas in the RGB image is predicted by a deep residual network, the average color difference of each layer is calculated according to the predicted color difference data, and the trigger threshold of each layer is set. The average color difference of each layer is compared with the trigger threshold of each layer to determine whether to trigger the re-compensation of each layer;

[0112] For the stratified layers that trigger re-compensation, the weight combination is adjusted to re-output the compensation parameter matrix, and the sampling frequency is updated according to the execution effect of the re-compensation.

[0113] The visual saliency and image complexity of different regions in the RGB image are different. Through stratification, different compensation strategies can be adopted for regions with different characteristics, giving priority to compensating regions with high visual saliency and high image complexity, improving the overall color compensation effect, while improving the calculation efficiency and avoiding resource waste caused by indiscriminate processing of all regions; The RGB image is converted to the Lab color space, the luminance channel is Gaussian blurred to obtain the background image, the visual saliency of the background image is obtained according to the foregoing method, and then the image complexity of the RGB image is obtained according to the foregoing method. The image is divided into four layers according to the visual saliency and image complexity: high visual saliency and high complexity region (high saliency and high complexity region), such as the close-up of a person's face with a complex background, high visual saliency but low complexity region (high saliency and low complexity region), such as a bright object in front of a simple background, low visual saliency but high complexity region (low saliency and high complexity region), such as a background with complex texture but not eye-catching, low visual saliency and low complexity region (low saliency and low complexity region), such as a large area of single-color background. The high saliency and high complexity region layer is set as the highest priority, followed by the high saliency and low complexity region layer, then the low saliency and high complexity region layer, and the low saliency and low complexity region layer has the lowest priority. For example, when performing re-compensation, the high saliency and high complexity region layer is processed first to ensure the color accuracy of important and complex regions; thus, different regions of the RGB image can be processed targeted, improving the effect and efficiency of color compensation, highlighting the color quality of important regions, while reasonably allocating computing resources and avoiding over-compensation of low-importance regions.

[0114] The deep residual network has powerful feature learning ability and can accurately predict the color difference data of sensitive regions. By calculating the average color difference of each layer and comparing it with the trigger threshold, it can determine which regions have color deviations beyond the acceptable range, thus deciding whether re-compensation is needed to achieve dynamic monitoring and optimization of the color compensation effect. A deep residual network with multiple residual blocks is constructed. A large number of RGB images containing different scenes and color deviations are collected as the training set, and at the same time, the corresponding accurate color images are collected as labels. During the training process, the RGB images are input into the deep residual network, and the deep residual network outputs the predicted color difference data. The mean squared error is used as the loss function, and the parameters of the deep residual network are updated through the backpropagation algorithm to minimize the error between the predicted color difference data and the true color difference data. For the RGB image to be processed, the color difference data of the sensitive regions in the RGB image is predicted through the trained deep residual network. For each layer, the average color difference of the pixel points in all sensitive regions within the layer is calculated.

[0115] Then, through experiments and analysis of a large number of RGB images, different trigger thresholds are set for each layer. Since the high-visibility and high-complexity region layer has high importance, the trigger threshold is set to 3, with the unit being the color difference unit in the CIELAB color space. The trigger threshold for the high-visibility and low-complexity region layer is set to 5, the trigger threshold for the low-visibility and high-complexity region layer is set to 7, and the trigger threshold for the low-visibility and low-complexity region layer is set to 10. The average color difference of each layer is compared with the corresponding trigger threshold. If the average color difference is greater than the trigger threshold, re-compensation for that layer is triggered. Thus, it can accurately monitor the color deviation situation of each layer of the RGB image, automatically determine whether re-compensation is needed, improve the accuracy and self-adaptability of color compensation, and ensure that the RGB image can achieve better color effects in different regions.

[0116] When re-compensation is triggered for a certain layer, it indicates that the current compensation effect has not reached the ideal state. Adjusting the weight combination can re-distribute the roles of the reference color deviation, light influence factor, and regional sensitivity in the calculation of the compensation intensity to better adapt to the characteristics of that layer and improve the compensation effect. Updating the sampling frequency according to the execution effect of re-compensation can dynamically adjust the data sampling frequency, increasing the sampling frequency in the case of large color changes to obtain more accurate data, and vice versa, reducing the sampling frequency to save resources. For the stratified layer where re-compensation is triggered, the optimal weight combination is re-searched through the Q-learning reinforcement learning algorithm. Using the reference color deviation, light influence factor, and regional sensitivity of the pixel points within that layer as the state space, different weight combinations as the action space, and the reward function is defined as the improvement value of the structural similarity index between the RGB image of that layer after compensation and the reference high-quality image. Through multiple iterative trainings, the weight combination that maximizes the reward is found.

[0117] According to the new weight combination, combined with the compensation intensity function, recalculate the compensation intensity of each pixel point in this layer, and combine color correction parameters, such as the correction values of the RGB channels, to generate a new compensation parameter matrix. If the quality of the RGB image of this layer is significantly improved after re-compensation, such as the structural similarity index is increased by more than 0.1, and the number of layers triggering re-compensation is relatively large, such as more than half of the total number of layers, then increase the sampling frequency by 20%. If the image quality improvement after re-compensation is not obvious and the number of layers triggering re-compensation is small, then reduce the sampling frequency by 10%. For example, after re-compensation, both the high-display high-recovery area layer and the high-display low-recovery area layer have obvious quality improvement and trigger re-compensation. At this time, increase the sampling frequency from 10 Hz to 12 Hz; thus, it is possible to dynamically adjust the compensation strategy according to the re-compensation requirements, improve the color compensation effect, and at the same time reasonably adjust the sampling frequency to balance the relationship between the resource consumption of data acquisition and processing and the color compensation quality. The updated compensation parameter matrix is used to compensate the reference color again to improve the image color quality. The updated sampling frequency affects subsequent data acquisition, providing a more suitable data basis for continuously optimizing color compensation.

[0118] Embodiment 2

[0119] The embodiment of the present application provides a color compensation system based on adaptive color adjustment, and the system includes:

[0120] Obtain the original image data, ambient light data, and the color gamut of the target device through the sampling frequency, and align the timestamps of the original image data and the ambient light data;

[0121] Identify the color clusters with a proportion greater than 15% in the original image data through a clustering algorithm as the main colors of the image. Calculate the offset of the color temperature under the illumination intensity according to the ambient light data to determine the color difference data. Determine the reference color based on the main colors of the image and the color difference data, and map the ambient light data into the original image data to output the light influence factor of each pixel point in the original image data;

[0122] Divide the sensitive areas in the original image data through a semantic segmentation model, and output the area sensitivity of each pixel point in each sensitive area. At the same time, compare the reference color with the target reference color to obtain the reference color deviation of each pixel point. According to the reference color deviation, light influence factor, and area sensitivity of each pixel point, combine the compensation intensity of each pixel point to output a compensation parameter matrix, and compensate the reference color according to the compensation parameter matrix;

[0123] Map the compensated reference color to the color gamut of the target device, output the RGB image adapted to the target device, and predict the color difference data of the sensitive areas in the RGB image through a deep residual network. Perform hierarchical re-compensation according to the change of the color difference data to update the compensation parameter matrix and the sampling frequency.

[0124] For the solution of the above system, please refer to the content of the embodiments of the method, which will not be elaborated here one by one.

Claims

1. A color compensation method based on adaptive color palette, characterized in that Including: Obtain the original image data, ambient light data, and the color configuration of the target device through the sampling frequency, align the original image data and the ambient light data in terms of time stamps. The ambient light data includes the color temperature and the light intensity, and the color configuration of the target device includes the target device color gamut; Identify the color clusters in the original image data with a proportion greater than 15% through a clustering algorithm as the main image colors. Calculate the offset of the color temperature under the light intensity according to the ambient light data to determine the color difference data. Determine the reference color based on the main image colors and the color difference data, and map the ambient light data into the original image data to match the image resolution in the original image data, and output the light influence factor of each pixel point in the original image data; The steps for determining the reference color include: Calculate the hue weight of each main image color according to the proportion of each main image color in the original image data and the visual saliency of each main image color, and determine the clustering center color of each main image color; Calculate the standard deviation between the ambient light data and the historical ambient light data to determine the ambient light weight, and set the calibration color; Determine the reference color by performing weighted summation on the hue weight and the clustering center color, as well as the ambient light weight and the calibration color; Divide the sensitive areas in the original image data through a semantic segmentation model, output the area sensitivity of each pixel point in each sensitive area. At the same time, compare the reference color with the target reference color to obtain the reference color deviation of each pixel point. According to the reference color deviation, light influence factor, and area sensitivity of each pixel point, combine the compensation intensity of each pixel point to output the compensation parameter matrix, and compensate the reference color according to the compensation parameter matrix; The steps for outputting the compensation parameter matrix include: Convert the reference color and the target reference color to the perceptual hash space, and obtain the reference color deviation of each pixel point by calculating the Hamming distance between the perceptual hash values of the reference color and the target reference color; Construct a reinforcement learning, determine that the state space is the reference color deviation, light influence factor, and area sensitivity of each pixel point, the action space is the weight combination of the reference color deviation, light influence factor, and area sensitivity of each pixel point, and continuously iterate and train the reinforcement learning; Calculate the compensation intensity of each pixel point according to the trained weight combination, and output the compensation parameter matrix in combination with the color correction parameters; Map the compensated reference color to the target device color gamut, output the RGB image adapted to the target device, and predict the color difference data of the sensitive areas in the RGB image through a deep residual network. Perform hierarchical re-compensation according to the change situation of the color difference data to update the compensation parameter matrix and the sampling frequency; The steps for performing the hierarchical re-compensation include: Perform hierarchical division on the RGB image based on the visual saliency and image complexity of the RGB image, and set the compensation priority for each layer; Predict the color difference data of the sensitive areas in the RGB image through a deep residual network, calculate the average color difference of each layer according to the predicted color difference data, and set the trigger threshold for each layer. Compare the average color difference of each layer with the trigger threshold of each layer to determine whether to trigger the re-compensation of each layer; For the layer that triggers re - compensation, adjust the weight combination to re - output the compensation parameter matrix, and update the sampling frequency according to the execution effect of the re - compensation.

2. The color compensation method based on adaptive color palette as claimed in claim 1, wherein The steps for determining the color difference data include: Train the historical ambient light data through a recurrent neural network to predict the target color temperature under the illumination intensity; Calculate the deviation between the actual color temperature and the target color temperature to obtain the offset of the color temperature under the illumination intensity; Set the light intensity influence coefficient according to the illumination intensity, and determine the color difference data based on the light intensity influence coefficient and the offset of the color temperature under the illumination intensity.

3. The color compensation method based on adaptive color adjustment according to claim 2, wherein, The steps for outputting the light influence factor of each pixel point include: Map the ambient light data to the pixel points in the original image data through the ray - tracing algorithm. For each pixel point, output the light propagation path from the ambient light source to the pixel point; Judge whether there is an occluder on each light propagation path to determine the occlusion coefficient of each pixel point; Determine the influence of the illumination intensity and color temperature in the ambient light data on each pixel point to obtain the influence coefficient of each pixel point; Combine the occlusion coefficient and influence coefficient of each pixel point to output the light influence factor of each pixel point.

4. The color compensation method based on adaptive color matching according to claim 3, characterized in that, The steps for outputting the regional sensitivity of each pixel point include: Divide the sensitive regions in the original image data through a semantic segmentation model, and determine the semantic category of each pixel point in each sensitive region through an attention mechanism; Extract the modal feature vector combining the convolutional neural network and the long - short - term memory network for each pixel point, and set the initial sensitivity for each semantic category based on the modal feature vector to obtain the initial regional sensitivity of each pixel point; Based on bilateral filtering, determine the spatial distance and feature similarity between pixel points, and output the regional sensitivity of each pixel point according to the spatial distance and feature similarity between pixel points.

5. The color compensation method based on adaptive color adjustment according to claim 4, characterized in that The steps for determining the sampling frequency include: Determine the load factor of the target device according to the CPU computing power and memory bandwidth of the target device, set the reference sampling rate based on the type of the target device, and determine the sampling rate in combination with the load factor of the target device; Based on the change gradient of the color temperature and the change gradient of the illumination intensity in the historical ambient light data, calculate the change rate of the ambient light, and correct the sampling rate according to the change rate of the ambient light; Determine the image complexity according to the clustering dispersion of the main color tone of the image and the edge density of the original image data, and optimize the sampling rate according to the image complexity to obtain the finally determined sampling frequency.

6. A color compensation system based on adaptive color adjustment, for implementing the color compensation method based on adaptive color adjustment according to any one of claims 1-5, characterized in that, Include: Obtain the original image data, ambient light data, and the color gamut of the target device through the sampling frequency, and align the time stamps of the original image data and the ambient light data; Identify the color cluster with a proportion greater than 15% in the original image data through a clustering algorithm as the main color tone of the image, calculate the offset of the color temperature under the illumination intensity according to the ambient light data, determine the color difference data, determine the reference color based on the main color tone of the image and the color difference data, and map the ambient light data to the original image data to output the light influence factor of each pixel point in the original image data. Divide the sensitive regions in the original image data through a semantic segmentation model, output the regional sensitivity of each pixel in each sensitive region, and at the same time compare the reference color with the target reference color to obtain the reference color deviation of each pixel. According to the reference color deviation, light influence factor and regional sensitivity of each pixel, combine the compensation intensity of each pixel to output a compensation parameter matrix, and compensate the reference color according to the compensation parameter matrix; Map the compensated reference color to the target device color gamut, output an RGB image adapted to the target device, and predict the color difference data of the sensitive regions in the RGB image through a deep residual network. Perform hierarchical re-compensation according to the change of the color difference data to update the compensation parameter matrix and sampling frequency.

Images