Image color correction method, device, color correction robot and storage medium

Abstract

The present application provides a method, device, color correction robot and storage medium for image color correction, the method comprising: obtaining an image to be processed, extracting the color value of each pixel in multiple color blocks in the image to be processed, and obtaining a color value list; setting weight coefficients for each color value in the color value list; fitting a polynomial regression model using the color value list with weight coefficients to obtain a fitted polynomial regression model; generating a color conversion matrix using the fitted polynomial regression model, the color conversion matrix is ​​used to represent the mapping relationship between the color value in the image to be processed and the standard color value in the colorimetric card; correcting the color value of the pixel in the image to be processed using the color conversion matrix to obtain a corrected image. This embodiment improves the color accuracy of the image, ensures the reliability and authenticity of the image information, and thus provides reliable data support for subsequent intelligent analysis and auxiliary diagnosis.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image color correction method, device, color correction robot and storage medium. Background Art

[0002] With the advancement of artificial intelligence technology, the role of medical images in clinical diagnosis has become increasingly prominent. However, the quality of medical images directly affects the accuracy of doctors' diagnoses, especially in areas where color information is crucial to diagnosis, such as dermatology, dentistry, and ophthalmology. For example, high-quality tongue images are the data basis for all subsequent intelligent and auxiliary tongue diagnosis, and the accuracy of tongue image color is directly related to the reliability of the diagnosis results.

[0003] When collecting tongue images, the camera will be affected by various factors such as lighting conditions, focal length changes, shooting environment, etc. during the imaging process. In order to compensate for the above unfavorable factors, the camera usually automatically adjusts the image quality through various built-in image processing algorithms. The existing image processing algorithms are mainly used to make the captured images easier for humans to watch and look more beautiful. However, in actual medical applications, image processing algorithms may cause deviation and distortion of the color of the tongue image. Since the image processing algorithm is built into the camera and uses an automatic adjustment mechanism, the imaging color is not accurate, especially under complex lighting and background conditions. This deviation may be reflected in the color saturation, contrast, and even white balance of the tongue image.

[0004] The above color deviation not only affects the visual authenticity of the tongue image, but also seriously reduces the diagnostic value of the tongue image. Especially in the field of intelligent analysis and auxiliary diagnosis of traditional Chinese medicine tongue diagnosis, which requires accurate color information, the accuracy of the color of the tongue image is crucial. Slight color difference may increase the difficulty of identifying the lesion site and even cause misdiagnosis or missed diagnosis. Accurate color correction of tongue images is an urgent problem to be solved. Summary of the invention

[0005] In order to solve the above technical problems, the present application provides an image color correction method, device, color correction robot and storage medium to improve the color accuracy of the image, ensure the reliability and authenticity of the image information, and thus provide reliable data support for subsequent intelligent analysis and auxiliary diagnosis.

[0006] In a first aspect, the present application provides a method for correcting image color, the method comprising: obtaining an image to be processed, wherein the image to be processed includes a colorimetric card area, and the colorimetric card area includes multiple color blocks; extracting the color value of each pixel in the multiple color blocks to obtain a color value list; setting a weight coefficient for each color value in the color value list; fitting a polynomial regression model using the color value list with the weight coefficient to obtain a fitted polynomial regression model; generating a color conversion matrix using the fitted polynomial regression model, the color conversion matrix being used to represent the mapping relationship between the color values ​​in the image to be processed and the standard color values ​​in the colorimetric card; and correcting the color values ​​of the pixels in the image to be processed using the color conversion matrix to obtain a corrected image.

[0007] In a second aspect, the present application provides an image color correction device, which includes: an image acquisition module, used to acquire an image to be processed, wherein the image to be processed includes a colorimetric card area, and the colorimetric card area includes multiple color blocks; a color value extraction module, used to extract the color value of each pixel in the multiple color blocks to obtain a color value list; a weight setting module, used to set weight coefficients for each color value in the color value list; a fitting module, used to fit a polynomial regression model using a color value list with weight coefficients to obtain a fitted polynomial regression model; a matrix generation module, used to generate a color conversion matrix using the fitted polynomial regression model, the color conversion matrix is ​​used to represent the mapping relationship between the color values ​​in the image to be processed and the standard color values ​​in the colorimetric card; a color correction module, used to correct the color values ​​of the pixels in the image to be processed using the color conversion matrix to obtain a corrected image.

[0008] In a third aspect, the present application provides a color correction robot, which includes: one or more processors; a storage device for storing one or more programs; a camera for capturing images to be processed and transmitting them to the processor; when one or more programs are executed by one or more processors, the one or more processors implement the image color correction method as described in the first aspect above.

[0009] In a fourth aspect, the present application provides a storage medium, which may be a computer-readable storage medium having a computer program stored thereon, and when the program is executed by a processor, the image color correction method as described in the first aspect above is implemented.

[0010] In a fifth aspect, an embodiment of the present application provides a computer program product, which includes a computer program or instructions, and when the computer program or instructions are executed by a processor, the method for correcting the image color as described in any one of the first aspects above is implemented.

[0011] Compared with the prior art, the technical solution provided by the embodiments of the present application has the following advantages:

[0012] The embodiment of the present application provides a method, device, color correction robot and storage medium for image color correction, the method comprising: obtaining an image to be processed, wherein the image to be processed includes a colorimetric card area, and the colorimetric card area includes multiple color blocks; extracting the color values ​​of each pixel in the multiple color blocks to obtain a color value list; setting weight coefficients for each color value in the color value list; fitting a polynomial regression model using the color value list with weight coefficients to obtain a fitted polynomial regression model; generating a color conversion matrix using the fitted polynomial regression model, the color conversion matrix is ​​used to represent the mapping relationship between the color values ​​in the image to be processed and the standard color values ​​in the colorimetric card; using the color conversion matrix to correct the color values ​​of the pixels in the image to be processed to obtain a corrected image. This embodiment improves the color accuracy of the image, ensures the reliability and authenticity of the image information, and thus provides reliable data support for subsequent intelligent analysis and auxiliary diagnosis. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0014] Figure 1 A schematic diagram of a flow chart of a method for correcting image color provided in an embodiment of the present application;

[0015] Figure 2 A schematic diagram of the construction process of the color correction robot provided in an embodiment of the present application;

[0016] Figure 3 A sample schematic diagram of an original image provided in an embodiment of the present application;

[0017] Figure 4 A schematic diagram showing the comparison of images before and after preprocessing provided in an embodiment of the present application;

[0018] Figure 5 A schematic diagram of marking a colorimetric card area in an original image provided in an embodiment of the present application;

[0019] Figure 6 A schematic diagram of annotated color blocks in a colorimetric card area image provided in an embodiment of the present application;

[0020] Figure 7 A schematic diagram of a process of embedding the color correction robot program provided in an embodiment of the present application into a standard robot;

[0021] Figure 8 A schematic diagram of the structure of the colorimetric card detection model provided in the embodiment of the present application;

[0022] Fig. 9A schematic diagram of the structure of the color block segmentation model provided in the embodiment of the present application;

[0023] Fig.10 A complete flow chart of the process from colorimetric card detection to color block segmentation provided in an embodiment of the present application;

[0024] Fig.11 A schematic diagram of a process for generating a color value list provided in an embodiment of the present application;

[0025] Fig.12 A schematic diagram of a weight coefficient adjustment process provided in an embodiment of the present application;

[0026] Fig.13 A schematic diagram of the polynomial regression model fitting process provided in an embodiment of the present application;

[0027] Fig.14 A schematic diagram of a process for generating a color conversion matrix provided in an embodiment of the present application;

[0028] Fig.15 A flowchart of color value conversion provided in an embodiment of the present application;

[0029] Fig.16 A schematic diagram of the structure of an image color correction device provided in an embodiment of the present application;

[0030] Fig.17 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0031] In order to more clearly understand the above-mentioned purposes, features and advantages of the present application, the scheme of the present application will be further described below. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0032] First, the terms that may be involved in this embodiment are briefly introduced.

[0033] Red Green Blue (RGB) color model, the RGB color model uses red, green and blue to represent the three basic components of color. The intensity of each color is usually represented by a numerical value, and the combination of these numerical values ​​forms a variety of different colors.

[0034] Support Vector Machine (SVM) is a supervised learning algorithm used for classification and regression tasks.

[0035] Convolutional Neural Network (CNN) is a deep learning model that is particularly suitable for processing image data and is often used in tasks such as image classification, object detection, and image generation.

[0036] The Canny edge detection algorithm is a widely used image processing technique to detect edges in an image.

[0037] Labelme is an open source image annotation tool for creating and editing annotations in image datasets. It is commonly used in computer vision tasks such as object detection, image segmentation, and image classification.

[0038] Message Digest Algorithm 5 (MD5) is an encryption function that is often used to verify the integrity of data and detect whether the data has been tampered with or damaged.

[0039] Lux is a unit of light intensity. It is a standard unit and terminology widely used in image processing and computer vision, mainly related to the measurement and description of light.

[0040] Kelvin (K) is a unit of color temperature that reflects the hue of light emitted by a light source, that is, the change in the color of the light source from warm to cool. The higher the color temperature, the cooler the color of the light source (bluish), and the lower the color temperature, the warmer the color of the light source (reddish).

[0041] Feature Pyramid Networks (FPN) is a neural network architecture used for computer vision tasks, especially in tasks such as object detection and semantic segmentation.

[0042] The loss function is an indicator used in machine learning and deep learning to evaluate the difference between the model's prediction results and the actual labels. The smaller the value of the loss function, the closer the model's prediction results are to the true value.

[0043] With the advancement of artificial intelligence technology, the role of medical images in clinical diagnosis has become increasingly prominent. However, the quality of medical images directly affects the accuracy of doctors' diagnoses, especially in areas where color information is crucial for diagnosis, such as dermatology, dentistry, and ophthalmology. For example, tongue images are an important basis for tongue diagnosis in traditional Chinese medicine, and their color accuracy is directly related to the reliability of the diagnosis results.

[0044] In the actual process of tongue image data acquisition, factors such as lighting conditions, imaging equipment settings, and shooting environment can cause color deviations in tongue images. This color deviation seriously reduces the diagnostic value of tongue images, especially in the field of intelligent analysis and auxiliary diagnosis of traditional Chinese medicine tongue diagnosis that requires accurate color information. Accurate color correction of tongue images is an urgent problem to be solved. In order to address these problems, in the related art, the methods for color correction of tongue images mainly include the following categories: color correction methods based on polynomial regression and colorimetric cards, color correction methods based on machine learning and colorimetric cards, and color correction methods based on deep learning and colorimetric cards.

[0045] In the above three color correction methods, the steps of extracting color values ​​are mostly based on grayscale conversion of images and edge detection algorithms (such as Canny edge detection) to find the boundaries of color blocks and determine the contour positions of each color block of the colorimetric card. By calculating the average color value of the central area of ​​each color block, the influence of edge effects in the imaging process on the correction results can be effectively reduced.

[0046] Although the color correction method based on polynomial regression, machine learning and deep learning can improve the color accuracy of the image to a certain extent. Since the actual application of TCM tongue diagnosis is highly dependent on the true color restoration of the tongue image, the above color correction method still has many problems. TCM tongue diagnosis is extremely sensitive to the accurate presentation of color features such as tongue color and tongue coating color. Any color deviation may affect the doctor's diagnosis. Therefore, the above color correction method still cannot meet the needs of intelligent analysis and auxiliary diagnosis of TCM tongue diagnosis in actual application. The above color correction method mainly has the following defects:

[0047] (1) Disadvantages of color correction methods based on polynomial regression and colorimetric charts

[0048] The polynomial regression model corrects the image color by fitting the relationship between the standard color value and the color value in the captured image. However, this method has shortcomings in the selection of the polynomial order. When the order is high, the model may overfit, that is, it is too sensitive to the noise in the training data, resulting in poor performance on new images; when the order is too low, it cannot capture complex color changes, and the correction effect is not ideal. In addition, the polynomial regression model usually models each color channel independently, ignoring the correlation between color channels, which may cause the corrected image to appear unnatural.

[0049] (2) Disadvantages of color correction methods based on machine learning and colorimetric charts

[0050] Machine learning methods such as support vector machines (SVMs) are able to handle complex nonlinear problems, but can face challenges when dealing with high-dimensional data and complex color mapping. In particular, the performance of the model is highly dependent on the quality and diversity of the training data, which means that if there is a lack of sufficient labeled data or the data is not diverse enough, the correction effect of the model will be affected. In addition, the training and tuning process of the SVM model is complex and usually requires a lot of computing resources and time.

[0051] (3) Disadvantages of color correction methods based on deep learning and colorimetric cards

[0052] Deep learning models can learn the complex mapping relationship between the color values ​​of the captured image and the standard color values, which has certain advantages in color correction. However, the training of deep learning models often requires a large amount of high-quality annotated data, and the training process is time-consuming and requires high computing resources. In addition, deep learning models face the problem of overfitting and may lack good generalization ability when processing tongue images with diverse backgrounds and lighting.

[0053] The three color correction methods mentioned above all have two common significant problems: (1) the amount of training data is limited; and (2) the assumption that the colorimetric card and the photographed object are located on the same plane.

[0054] (1) Data volume limitation

[0055] The above three color correction methods all use the color values ​​of the colorimetric card for correction, and usually only obtain the average color value of each color block. Taking the commonly used 24-color standard colorimetric card as an example, the data used for correction only has 24 color values. This limitation in the amount of data will cause the model to face the problem of insufficient data during training and cannot fully learn the complex color changes in the image. In addition, the color changes in actual images are often gradual and multi-dimensional. It is difficult to capture these subtle changes with only a small amount of color block data, which affects the detailed performance and overall effect of the correction.

[0056] (2) Assumption that the colorimetric chart and the object being photographed are on the same plane

[0057] Existing methods generally assume that the colorimetric card and the object being photographed are on the same plane, with consistent lighting intensity and angle. However, in actual shooting, lighting and viewing angle conditions often change, which can cause deviations in the color values ​​of color blocks. Since the above three color correction methods usually use the mean of the colorimetric card area as the representative color of the color block, they fail to consider the impact of lighting changes and viewing angle differences, thus limiting the accuracy and applicability of color correction.

[0058] In order to solve the above technical problems, an embodiment of the present application provides a method for correcting image color. The image color correction method provided by the embodiment of the present application is described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0059] Figure 1 This is a flow chart of an image color correction method in an embodiment of the present application. This embodiment is applicable to the case of correcting the color in a captured image. The method can be executed by an image color correction device. The image color correction device can be implemented in software and / or hardware. The image color correction device can be configured in a color correction robot. The color correction robot may include: a standard robot with medical health monitoring or medical auxiliary treatment.

[0060] The image color correction method provided in the embodiment of the present application is mainly implemented by a color correction robot program built into the color correction robot. First, the process of building the color correction robot is introduced.

[0061] The color correction robot in this embodiment is mainly used to solve the defects of insufficient training data and the assumption that the colorimetric card and the photographed object are in the same plane in the traditional color correction method. By introducing the detection and segmentation model, an intelligent, accurate and integrated image color correction method is designed, which can maintain high-precision color restoration in complex imaging environments. The main components of the color correction robot include the following modules: image acquisition module, standard colorimetric card, detection and segmentation module and color correction module.

[0062] The color correction robot is equipped with a high-resolution camera and a camera control unit for capturing high-precision images. The color correction robot can adjust the shooting angle and lighting conditions in different environments to ensure accurate capture of the color information in the scene, and store the captured images in an internal storage unit for subsequent processing.

[0063] In the color correction robot, a standard colorimetric card is used as a benchmark for color correction. The colorimetric card contains multiple color blocks with known standard color values, which provide reliable reference points for image color correction and ensure the accuracy of color adjustment.

[0064] The detection and segmentation module mainly includes the colorimetric card detection model and the color block segmentation model. The colorimetric card detection model is based on a deep learning algorithm and can automatically identify and locate the position of the colorimetric card in the image. This model ensures that the position of the colorimetric card in the image is accurately detected, laying the foundation for subsequent color block segmentation and color value data extraction. The color block segmentation model is also based on a deep learning algorithm and is responsible for pixel-level accurate segmentation of the color blocks in the colorimetric card. This model extracts all pixel data of each color block by analyzing the color information in the color block pixel by pixel, significantly improving the amount and accuracy of data used for color correction.

[0065] The color correction module calculates and generates a color conversion matrix based on the color block pixel data extracted by the colorimetric card detection model and the color block segmentation model. The color conversion matrix is ​​applied to each pixel of the original image to correct its color value, ultimately achieving high-precision image color correction. In this way, the color correction robot can effectively cope with complex lighting and environmental changes and ensure the accuracy of image color restoration.

[0066] The construction process of the color correction robot provided in this embodiment mainly includes five steps: data preprocessing, model training, construction of detection and segmentation modules, construction of color correction modules, and module embedding. The specific process of building the color correction robot is as follows: Figure 2 The specific contents are as follows:

[0067] a. Data preprocessing

[0068] In the data preprocessing stage, three steps will be performed: data collection, data processing, and data labeling to provide data support for subsequent model training and module construction. The specific steps are as follows:

[0069] Step (1): Data collection. A large number of images containing colorimetric cards are collected as original images through the built-in high-definition camera of the standard robot, ensuring that the original images cover a variety of lighting conditions, shooting angles, and different scenes. To ensure data quality, the collected original images are screened, blurry or unevenly illuminated images are removed, and original images with clear and visible colorimetric card areas are retained. After this step, high-quality original images are obtained.

[0070] Accurate and comprehensive original images lay a solid foundation for subsequent model training and color correction. The acquisition of original images mainly revolves around the acquisition of images containing colorimetric cards, aiming to ensure that the color correction robot can work stably under various lighting conditions, angles and scenes. Therefore, a large number of original images containing standard colorimetric cards are collected through the built-in high-resolution camera. And it is necessary to ensure that the colorimetric cards are photographed under different environmental conditions, including various lighting conditions and different shooting angles, to ensure the breadth and representativeness of the collected data. The specific original image acquisition strategy table is shown in Table 1 below.

[0071] Table 1. Collection strategy of original images

[0072]

[0073] Using the above original image acquisition strategy, a total of 12,000 original images with a resolution of 1080×1000 pixels were collected. To ensure image quality, 492 images with blur, uneven lighting or other conditions that may affect colorimetric card recognition were removed during the image screening process. Finally, 11,508 original images with clear and visible colorimetric card areas were retained. The following are the original image acquisition samples: Figure 3 As shown. Figure 3 As shown, taking the application of the image color correction method in the field of tongue diagnosis as an example, the original image 30 includes a tongue area 31 and a colorimetric card area 32 .

[0074] Step (2): Data preprocessing. The original image collected in step (1) is processed. The main operation is to crop the colorimetric card area in the original image to generate a colorimetric card area image for color block segmentation model training. During the cropping process, the integrity of the color blocks in the colorimetric card area image must be ensured to prevent the colorimetric card frame or other objects from affecting the image quality. After this step, a colorimetric card area image suitable for color block segmentation model training is obtained.

[0075] The data preprocessing stage is an important step to ensure data quality and applicability, and provides basic support for subsequent model training and color correction. Data preprocessing mainly involves appropriately cropping the original image collected in step (1), saving it as a colorimetric card area image required for color block segmentation model training, and serving as the data annotation source for the color block segmentation model in the subsequent step (3). During the preprocessing process, it should be noted that while maintaining the integrity of the color blocks in the colorimetric card area image, some colorimetric card frames can be appropriately removed to avoid objects other than the standard colorimetric card being included in the cropped image. The data preprocessing example is shown in the figure below. Figure 4 As shown in , before data preprocessing, Figure 4 In the left figure, the original image 30 includes the tongue area 31 and the colorimetric card area 32. After data preprocessing, Figure 4 In the right figure, the color card area image 40 only includes the color card area, and does not include the tongue area.

[0076] Step (3): Data labeling. This step includes two parts: colorimetric card detection data labeling and color block segmentation data labeling. The colorimetric card detection data labeling uses the original images of the tongue area and the colorimetric card area collected in step (1), and the color block segmentation data labeling uses the colorimetric card area image cropped in step (2). The open source labeling tool Labelme is used as the labeling tool for colorimetric card detection and color block segmentation. During the labeling process, it is necessary to clarify the boundaries of the colorimetric card and the specific position of each color block to provide an accurate reference for subsequent color value extraction and model training. After this step, the colorimetric card detection data set and the color block segmentation data set are obtained.

[0077] In step (3), the colorimetric card detection data needs to be annotated using the original image collected in step (1) to clearly identify the boundaries and position of the colorimetric card. The annotated data set will be used to train the colorimetric card detection model to ensure that the model can accurately identify the position of the colorimetric card in the image. The color block segmentation data annotation needs to use the colorimetric card area image preprocessed in step (2) to annotate the color block segmentation data. Also use the Labelme annotation tool to accurately annotate each color block in the colorimetric card and record the specific position and boundaries of each color block. The annotated data set will be used to train the color block segmentation model to ensure that the model can accurately segment the color block area in the colorimetric card. Colorimetric card detection data annotation sample is as follows: Figure 5 As shown in the figure, the colorimetric card area in the original image is annotated, and the color block segmentation data annotation sample is as follows: Figure 6 As shown, the specific position and boundary of each color block are marked.

[0078] It should be noted that Figure 3-6 It is a grayscale image, which is only used to illustrate the key steps of the image processing process during the image color correction process. In actual use, the images used for model training and image color correction are all color images containing color charts, such as images in RGB format.

[0079] After completing the data annotation of colorimetric card detection and color block segmentation, the annotated data sets are reasonably divided to ensure the scientificity and effectiveness of model training, verification and testing. Specifically, the original images after the colorimetric card detection data are annotated are randomly divided into training set, verification set and test set in a ratio of 8:1:1 to obtain the colorimetric card detection data set; the colorimetric card area images after the color block segmentation data are annotated are randomly divided into training set, verification set and test set in a ratio of 70:15:15 to obtain the color block segmentation data set. The details of the colorimetric card detection data set and the color block segmentation data set are shown in Table 2 below.

[0080] Table 2 Contents of the colorimetric card detection dataset and the color block segmentation dataset

[0081]

[0082] b. Model training phase

[0083] In the model training phase, the main goal is to build an automated detection and segmentation system so that image color correction can be more accurate and efficient in various complex scenarios and reduce human intervention. The following are the specific steps:

[0084] Step (4): Colorimetric card detection model training. Use the colorimetric card detection data set obtained in step (3) to train the pre-trained detection model to build a model that can automatically identify the position of the colorimetric card under different lighting conditions, angles, and scenes. The function of the colorimetric card detection model is to automatically locate the colorimetric card area, providing a basis for subsequent color block segmentation and color correction. During the training process, it is necessary to train and learn information such as the boundaries and positions of the colorimetric card to ensure that the colorimetric card detection model has high accuracy and robustness in practical applications. The process of colorimetric card detection model training is mainly to establish a mapping relationship between the input image and the output result through a large number of annotated original images. After this step, a colorimetric card detection model that can automatically detect the position of the colorimetric card will be obtained.

[0085] Step (5): This step uses the color block segmentation data set obtained in step (3) to train the pre-trained segmentation model and construct a model that can accurately segment the color block area in the colorimetric card area image. The main task of the color block segmentation model is to segment each color block in the colorimetric card area image separately to ensure that the pixel value of each color block can be accurately extracted. The key to this step is to obtain the fine color data in the colorimetric card area image through the color block segmentation model, which far exceeds the 24 color values ​​used in the traditional method. Compared with the use of the regional mean, the color value data can reach 10,000 times or even 100,000 times the original, greatly enriching the data resources required for color correction. After this step, a color block segmentation model that can accurately segment color blocks will be obtained. An accurate and effective color block segmentation model can achieve accurate segmentation of the areas of each color block in the colorimetric card, ensure the accurate extraction of all pixel values ​​in each color block, and provide a large amount of diverse color value data for subsequent color correction.

[0086] c. Construction of detection and segmentation module stage

[0087] In order to realize the interactive processing of the colorimetric card detection model and the color block segmentation model and effectively extract the pixel values ​​in the color block of the colorimetric card, a detection and segmentation module is constructed to integrate the colorimetric card detection model, the color block segmentation model and the color value extraction algorithm, so as to realize the automatic image color correction process in the future. By interactively integrating the colorimetric card detection model, the color block segmentation model and the color value extraction algorithm, a complete automated process from colorimetric card detection, color block segmentation and color value extraction can be realized.

[0088] d. Color correction module construction stage

[0089] The color correction module is the core component of the system. Its purpose is to apply the rich color data obtained in the previous steps to the actual image correction to improve the color accuracy of the tongue image. The color correction module is mainly used to adjust the weights of the extracted color values, and use the adjusted weights and color values ​​as input data to fit the polynomial regression model. Finally, it is applied to the tongue image to be corrected based on the fitted polynomial regression model and color conversion matrix to achieve high-quality tongue image color correction.

[0090] e. Module embedding stage

[0091] In this stage, the detection and segmentation module is integrated with the color correction module and embedded into the standard robot to obtain a color correction robot, so that the color correction robot can realize a fully automated image color correction process. This stage ensures that the various parts of the system can work in coordination, so that the color correction robot can perform color correction processing while taking images.

[0092] The detection and segmentation module and the color correction module are embedded in the standard robot, giving the standard robot the ability of integrated image color correction, so that the standard robot has the ability of color correction. In other words, the detection and segmentation module and the color correction module are embedded in the standard robot to obtain a color correction robot. Among them, the standard robot is equipped with a high-resolution camera and a camera control unit for the acquisition of high-precision tongue images. The standard robot can adjust the shooting angle and lighting conditions in different environments to ensure accurate capture of the color information in the scene, and store the captured images in the internal storage unit for subsequent processing.

[0093] Optionally, the main structure and functions of the standard robot are shown in Table 3, and the main structure and functions of the color correction robot are shown in Table 4. As can be seen from Tables 3 and 4, the hardware structure and accessories of the standard robot and the color correction robot are the same, and the software configurations both include a camera control module; compared with the standard robot, the software configuration of the color correction robot adds a detection segmentation module and a color correction module.

[0094] Table 3 Main structure and function of standard robot

[0095]

[0096] Table 4 Main structure and function of color correction robot

[0097]

[0098] The specific process of embedding the detection and segmentation module and the color correction module into a standard robot is as follows: Figure 7As shown: first, obtain the standard robot program in the standard robot; then, embed the detection segmentation module and the color correction module into the standard robot program in series to form a color correction robot program; finally, overwrite the color correction robot program into the standard robot to form a color correction robot. Among them, the standard robot program includes: the camera shoots based on the shooting signal, and then the imaging module outputs the captured image, and the output module stores the output captured image. The color correction robot program includes: the camera shoots based on the shooting signal, and then the imaging module outputs the captured image, the detection segmentation module processes the captured image and outputs the color block coordinates and color value data, the color correction module performs color correction on the captured image according to the color block coordinates and color value data, and the output module outputs the color-corrected image and stores it.

[0099] The following is an introduction to the method of using a color correction robot to correct image color. Figure 1 As shown, the image color correction method provided in the embodiment of the present application mainly includes steps S101-S106.

[0100] S101. Acquire an image to be processed, wherein the image to be processed includes a colorimetric card area, and the colorimetric card area includes a plurality of color blocks.

[0101] The image to be processed can be understood as an image with color deviation taken by an electronic device. The electronic device refers to an electronic device that deploys a color correction robot program. For example: a color correction robot. Colorimetric cards include standard colorimetric cards, which are usually composed of a series of predefined standard color blocks. These color blocks are printed or displayed in a certain arrangement, and each color has a unique number or name for users to find and reference. Use a standard colorimetric card as a benchmark for color correction. The colorimetric card contains multiple color blocks with known standard color values. The color blocks provide reliable reference points for image color correction to ensure the accuracy of color adjustment.

[0102] Furthermore, the image to be processed includes a target object area, which is used to display the target object. The target object is a target that needs to be color corrected, and the target object area is color corrected to improve the correctness of the color of the target object. The target object is specifically determined according to the purpose of the color correction robot. For example, if the color correction robot is an auxiliary device for tongue diagnosis, then the target object is the tongue; if the color correction robot is an auxiliary device for eye diagnosis, then the target object is the eye. The target object type is not specifically limited in the embodiments of the present application. The embodiments of the present application are described by taking the tongue as an example of the target object.

[0103] In actual application scenarios, the target object and the colorimetric card are placed in the same scene, and then the scene is photographed with a camera, and the image captured by the camera is obtained as the image to be processed. In the tongue diagnosis application scenario, the patient's tongue and the colorimetric card are placed in the same scene, for example: the colorimetric card is placed under the tongue, and then the scene is photographed with a camera, and the image captured by the camera is obtained as the image to be processed. Figure 3 As shown, the image to be processed includes a colorimetric card area and a tongue area. The colorimetric card area contains a plurality of color blocks with known standard color values. The color blocks provide reliable reference points for color correction of the image to be processed, thereby ensuring the accuracy of color adjustment.

[0104] Furthermore, the color correction robot is equipped with a high-resolution camera and a camera control unit, and the camera is used to collect high-precision images as images to be processed. The color correction robot can adjust the shooting angle and lighting conditions in different environments to ensure accurate capture of the color information in the scene, and store the captured images in an internal storage unit for subsequent processing.

[0105] S102: Extract the color value of each pixel in the plurality of color blocks to obtain a color value list.

[0106] The colorimetric card area includes multiple color blocks, each of which is composed of multiple pixels. Each color block in the colorimetric card area has its corresponding standard color value, that is, the color block and the standard color value are in a one-to-one correspondence. For each color block, the pixels in the color block are traversed to extract the color value of each pixel, and the color block code and the color value corresponding to each pixel are stored in the color value list.

[0107] In a possible implementation, the color value of each pixel in a plurality of color blocks is extracted to obtain a color value list, including: using a colorimetric card detection model to detect an image to be processed to obtain position information of a colorimetric card area; cropping the image to be processed based on the position information of the colorimetric card area to obtain a colorimetric card area image; using a color block segmentation model to identify and segment the colorimetric card area image to obtain coordinates of each pixel in each color block; and extracting the color value of each pixel in each color block based on the coordinates of each pixel in each color block to obtain a color value list.

[0108] First, the trained colorimetric card detection model is used to automatically detect the image to be processed and identify the position of the colorimetric card area in the image to be processed. After the detection is completed, the colorimetric card area in the image to be processed is cropped based on the position information of the colorimetric card area, and the cropped colorimetric card area image is passed to the trained color block segmentation model. The color block segmentation model will perform fine segmentation on the cropped colorimetric card area image and accurately identify each color block area. After the color block segmentation is completed, the position information of each color block is output, and the position information of the color block includes: the color block code and its specific area position in the image. This step ensures that all pixel information of each color block in the colorimetric card area can be accurately obtained.

[0109] The colorimetric card detection model is trained based on a deep learning algorithm and can automatically identify and locate the colorimetric card area in the image to be processed. The colorimetric card detection model is used to ensure that the position of the colorimetric card in the image to be processed is accurately detected, laying the foundation for subsequent color block segmentation and color value extraction. Specifically, the colorimetric card detection model is a detection model that can automatically identify and locate the colorimetric card area in the image to be processed, providing a foundation for subsequent color block segmentation and color correction.

[0110] The structure of the colorimetric card detection model is as follows Figure 8 As shown in the figure, the input of the colorimetric card detection model is the image to be processed, and the output of the colorimetric card detection model is the position information (x, y, w, h) and the image type is the colorimetric card, that is, the position information of the output colorimetric card area, which includes the horizontal coordinate, vertical coordinate, width and height (x, y, w, h) in the colorimetric card area, etc. The colorimetric card detection model uses a convolutional neural network (CNN) as a feature extraction network, and the fully connected layer further processes and combines these features, while the output layer generates the final detection result. Figure 8 As shown, the colorimetric card detection model extracts features from the input image to be processed to obtain a first feature map, extracts features from the first feature map to obtain a second feature map, extracts features from the second feature map to obtain a third feature map, extracts features from the third feature map to obtain a fourth feature map, and inputs the fourth feature map into the first fully connected layer; at the same time, the fourth feature map is upsampled to obtain a fifth feature map, the fifth feature map is fused with the third feature map to obtain a sixth feature map, and the sixth feature map is input into the second fully connected layer; the fifth feature map is upsampled to obtain a seventh feature map, the second feature map is fused with the seventh feature map to obtain an eighth feature map, and the eighth feature map is input into the third fully connected layer; the output data of the three fully connected layers are input into the detection head for processing to obtain position information and classification results, wherein the position information is: abscissa, ordinate, width and height (x, y, w, h); the classification result is a colorimetric card.

[0111] The above network structure enables the model to accurately detect the colorimetric card area and its features, thereby improving the detection capability of the color correction robot.

[0112] The color block segmentation model is trained based on a deep learning algorithm and is mainly used to perform pixel-level accurate segmentation of color blocks in the color card area image. The color block segmentation model analyzes the color information in the color block pixel by pixel and extracts the position information of all pixels in each color block, which significantly increases the amount of data and accuracy used for color correction.

[0113] The accurate and effective color block segmentation model can accurately segment the areas of each color block in the colorimetric card area image, ensuring the accurate extraction of all pixel values ​​in each color block, laying the foundation for providing a large number of diverse color values ​​for subsequent color correction.

[0114] like Fig. 9 As shown in the figure, the input of the color block segmentation model is the colorimetric card area image, and the output of the color block segmentation model is the encoding of each color block and the coordinates of each pixel in each color block. Among them, the encoding of the color block is used to indicate the standard color value corresponding to the color block. Furthermore, the color block segmentation model uses a convolutional neural network (CNN) as the main feature extraction network and combines it with a feature pyramid network (FPN) structure. The color block segmentation model first extracts features through a multi-layer CNN to extract different levels of information from the image, including the edges, shapes, and internal details of the color blocks. Then, the upsampling technique is used to restore the scale of the feature map, and with the help of the feature pyramid network, a multi-scale feature representation is constructed.

[0115] Specifically, Fig. 9As shown, the input of the color block segmentation model is the colorimetric card area image, the color block segmentation model performs convolution processing on the colorimetric card area image to obtain a ninth feature map, the ninth feature map is again convolved to obtain a tenth feature map, the tenth feature map is again convolved to obtain an eleventh feature map, the eleventh feature map is again convolved to obtain a twelfth feature map, the twelfth feature map is upsampled to obtain a thirteenth feature map, the eleventh feature map and the thirteenth feature map are fused to obtain a fourteenth feature map, and the fourteenth feature map is upsampled. The 15th feature map is obtained by processing; the 10th feature map and the 15th feature map are fused to obtain the 16th feature map, the 16th feature map is convolved to obtain the 17th feature map, the 14th feature map and the 17th feature map are fused to obtain the 18th feature map, the 18th feature map is convolved to obtain the 19th feature map, the 19th feature map and the 12th feature map are fused to obtain the 20th feature map, the 20th feature map, the 18th feature map and the 16th feature map are input into the segmentation head, and the segmentation head outputs the classification results and the pixel coordinates corresponding to each classification result. Among them, the classification results include the labels of each color block, and the pixel coordinates corresponding to each classification result include the coordinates of each pixel in the labeled color block.

[0116] This feature pyramid structure can better capture the multi-level information of the image, enabling the model to accurately understand and segment color block areas at different scales.

[0117] Through multi-level feature extraction and multi-scale feature fusion, the color block segmentation model can accurately identify and segment 24 color blocks in the color chart under complex lighting and angle conditions, including: Dark skin, Skin, Blue sky, Foliage, Blue flower, Bluishgreen, Orange, Purplish blue, Moderate red, Purple, Yellow, Green, Orange yellow, Blue, Red, Magenta, Cyan, White, Neutral8, Neutral6.5, Neutral5, Neutral3.5, Black. The above color blocks are coded 1-24 from left to right and from top to bottom in the color chart. The color block segmentation model ensures that all pixel values ​​within each color block area can be accurately extracted, providing a solid and reliable data foundation for subsequent color correction.

[0118] In the embodiment of the present application, the colorimetric card detection model obtained by model training is interactively integrated with the color block segmentation model, which can realize a complete automated process from colorimetric card detection to color block segmentation. The workflow of the fusion of the colorimetric card detection model and the color block segmentation model is as follows Fig.10 As shown in the figure: First, the image to be processed is input into the trained colorimetric card detection model. The colorimetric card detection model infers the image to be processed, obtains the colorimetric card detection result, outputs the position and boundary (x, y, w, h) of the colorimetric card area, and automatically crops the colorimetric card area in the image to be processed based on the position and boundary of the colorimetric card area to obtain the colorimetric card area image and pass the colorimetric card area image to the color block segmentation model. In this way, the detection frame generated by the colorimetric card detection model will be used as the input of the color block segmentation model to provide accurate area information for subsequent precise segmentation. The cropped colorimetric card area image is input into the color block segmentation model. The color block segmentation model uses the feature pyramid network (FPN) structure to segment the color blocks at different scales, ensuring that the boundary and internal area of ​​each color block can be accurately identified, obtaining the color block segmentation result, and outputting the coordinates of each color block. In this way, the color correction robot can obtain the precise position of each color block.

[0119] In a possible implementation, based on the coordinates of each pixel in each color block, the color value of each pixel in each color block is extracted to obtain a color value list, including: extracting the color value of each pixel in the image to be processed based on the coordinate value of each pixel; establishing an association relationship between the number of the color block and the color value of each pixel, and storing them in the color value list.

[0120] In an embodiment of the present application, based on the coordinates of each pixel in each color block, combined with a color value extraction algorithm, the color value of the corresponding coordinate position is extracted from the colorimetric card area image. The extracted color values ​​will be summarized into a color value list to provide data support for subsequent color correction. The color value list contains a large number of color values ​​in each color block in the colorimetric card, which far exceeds the limitation of traditional methods that only use the average color value of a single region, and can usually reach 10,000 to 100,000 times the number of regional average color values. In this way, more color details can be captured, providing richer and more accurate data support for subsequent color correction. Finally, the color value list of all color blocks will be output as the basic data input for the subsequent color correction steps. This ensures the automation and high-precision execution of the color correction process.

[0121] Furthermore, after the fusion of the colorimetric card detection model and the color block segmentation model, the next step is to combine the color value extraction algorithm with the output results of the aforementioned model to achieve high-precision color data extraction. The main task of the color value extraction algorithm is to extract the accurate color value (RGB value) of each pixel in each color block from the segmented color blocks. These color values ​​will be used in the subsequent color correction process. The specific process is as follows Fig.11 As shown in the figure, after the color block segmentation is completed, the region information of each color block has been obtained, including the boundary and internal area of ​​each color block. The color value extraction algorithm first reads these region information and identifies the coordinates of all pixels in each color block. Secondly, the pixel coordinates in each color block are traversed to extract the color value (RGB value) of each pixel, and the color block code and color value are stored in the color value list. Finally, the color value list is output to the color correction module. Fig.11 The figure on the right is a schematic diagram of a color value list structure, which includes multiple color value information, including a color block number and the color value of each pixel in the color block corresponding to the number. For example: Fig.11 The 01, 06, and 24 in the code identify the color blocks, and the (R, G, B) after each number identifies the color value of a pixel in the color block corresponding to the number.

[0122] S103: Setting a weight coefficient for each color value in the color value list to obtain a color value list with weight coefficients.

[0123] In order to adapt to different shooting conditions (such as light intensity, angle changes, etc.), the color values ​​are weighted according to the distribution of different color blocks. The purpose of weight adjustment is to assign weight coefficients to the color values ​​in the color block according to the lighting and shooting angle information in the actual scene, and reduce or increase their weight influence when adding them to the polynomial regression model fitting. Specifically, it is to adjust the weight coefficients of different color values ​​according to the distribution of color values ​​in the color value list.

[0124] Specifically, according to the color value list, the weight adjustment is performed by analyzing the distribution of color values. The performance of different color blocks in the image is affected by the light intensity and angle, so it is necessary to adjust the weight coefficient so that the polynomial regression model can better reflect the lighting conditions in the actual scene when fitting. After adding the weight coefficient to each color value, a color value list with weight coefficients is formed. By adjusting the weight, the model is prevented from being overly sensitive to certain abnormal or extreme color values, making the fitting smoother and consistent with the lighting conditions during actual shooting.

[0125] In one possible implementation, a weight coefficient is set for each color value in the color value list, including: counting the color value of each pixel in the color value list to obtain the color distribution of each channel; analyzing based on the color distribution of each channel to obtain the first parameter and the second parameter corresponding to each channel, wherein the first parameter is half of the channel width, and the second parameter is the channel value with the largest proportion in the channel; and calculating the weight coefficient of each channel based on the first parameter and the second parameter corresponding to each channel and the set weight calculation formula.

[0126] In the embodiment of the present application, color distribution statistics are performed on 24 color blocks in the image to be processed and the actual captured image using an ultra-high pixel camera, respectively. It can be found that, whether it is the image to be processed taken by an ultra-high pixel camera or the actual captured image, the pixel distribution of the three channels R, G, and B in the color block pixels in the colorimetric card basically conforms to the normal distribution.

[0127] like Fig.12 As shown in the figure, the specific process of the weight adjustment strategy is as follows: first, the color values ​​of the pixels contained in all color blocks are counted to obtain the pixel value distribution of the R channel, the pixel value distribution of the G channel, and the pixel value distribution of the B channel in the color block, and delete the singular values ​​in each channel; secondly, according to the channel color value distribution, the weight coefficient of the pixel point is adjusted, and the adjustment formula of the weight coefficient k is as follows:

[0128]

[0129] in:

[0130] is the natural logarithm;

[0131] It is the channel value of the current channel of the pixel point, that is, the horizontal coordinate value at this time;

[0132] It is the channel value that accounts for the largest proportion in a single channel, that is, the corresponding horizontal coordinate value of the maximum vertical coordinate value;

[0133] It is half the width of a single channel, that is, half the maximum abscissa value minus the minimum abscissa value.

[0134] After statistically analyzing the color value distribution, we get , And the weight transfer function .go through After weight adjustment, the color value list is converted into a color value list with weight coefficients, and the color value list with weight coefficients is passed as input to the polynomial regression model fitting process to fit the polynomial regression model.

[0135] S104, fitting a polynomial regression model using the color value list with weight coefficients to obtain a fitted polynomial regression model.

[0136] The purpose of polynomial regression model fitting is to construct a color correction model suitable for different shooting scenes based on a list of weighted color values, so as to adjust the color of the image to be processed to a standard state.

[0137] The polynomial regression model was fitted using a list of color values ​​with weight coefficients. A polynomial regression model was established based on the difference between the standard color values ​​of the colorimetric card and the color values ​​in the captured image, and the weight coefficients were considered in the loss function calculation to reduce the fitting error. The polynomial coefficients of the model were iteratively optimized to ensure that the model could accurately reflect the mathematical relationship between the color deviation of each pixel in the image to be processed and the standard color. Ultimately, the optimized polynomial regression model was able to better reflect the overall color shift of the tongue image.

[0138] In one possible implementation, a polynomial regression model is fitted using a target color value list to obtain a fitted polynomial regression model, including: fitting using color values ​​and color block numbers in the color value list to obtain an initial polynomial regression model; iteratively adjusting the initial polynomial regression model using weight coefficients of each color value in the color value list to obtain a fitted polynomial regression model, wherein the fitted polynomial regression model includes a polynomial regression model that minimizes a loss function.

[0139] The specific polynomial regression model fitting process is as follows Fig.13 As shown in the figure, first, the weight values, color values ​​and color block labels in the color value list with weight coefficients are extracted, and the color values ​​and color block labels are used to fit the polynomial regression model; the weight values ​​in the color value list with weight coefficients are used to correct the loss function (Loss) output when the polynomial regression model is fitted; third, the model parameters are iteratively adjusted so that the fitted polynomial curve can reflect the color change law between color blocks as accurately as possible; finally, the fitted polynomial regression model is output.

[0140] in, Fig.13 The loss function Loss in the above formula represents the loss function of the polynomial regression model, where , , Respectively represent the actual color values ​​of the red, green, and blue channel values ​​of the target; x, y, z are the initialization coefficients of the polynomial regression model f; , , are the weighting coefficients for the red, green, and blue channels respectively; It is a polynomial regression model after minimizing the loss function. This model mainly adjusts the coefficients in the polynomial regression model by minimizing the loss function to achieve the best fit for the color data. , , are the coefficients of the fitted polynomial regression model; is the fitting function after the coefficients of the polynomial regression model are split; a, b, c represent the results of the coefficient splitting of the red channel, e, f, g represent the results of the coefficient splitting of the green channel, and i, j, k represent the results of the coefficient splitting of the blue channel.

[0141] S105 , generating a color conversion matrix using the fitted polynomial regression model, where the color conversion matrix is ​​used to represent the mapping relationship between the color values ​​in the image to be processed and the standard color values ​​in the colorimetric card.

[0142] The purpose of generating a color conversion matrix is ​​to convert the fitted polynomial regression model into a conversion matrix suitable for tongue image color correction, which is used to adjust the color values ​​in the image to be processed to the color state of the standard colorimetric card. Use the fitted polynomial regression model to generate a color conversion matrix. The fitted polynomial regression model generates a color conversion matrix for actual color correction. The color conversion matrix contains the coefficients in the polynomial regression model and is passed as a whole to the subsequent tongue image color conversion step. The role of the color conversion matrix is ​​to correct the color value of each pixel of the tongue image in the RGB space.

[0143] In a possible implementation, a color conversion matrix is ​​generated using a fitted polynomial regression model, including: splitting coefficients in the fitted polynomial regression model to obtain a polynomial regression model after coefficient splitting; and combining the coefficients of the polynomial regression model after coefficient splitting into a color conversion matrix.

[0144] The specific process of generating a color conversion matrix is ​​as follows Fig.14 As shown. First, extract the coefficients in the fitted polynomial regression model. These coefficients are key parameters that describe the law of color change and determine how the color value is adjusted under different lighting conditions and shooting angles; secondly, combine the coefficients in the polynomial regression model into a standardized color conversion matrix. This matrix will represent the mapping relationship from the color value of the image to be processed to the color value of the standard colorimetric card. Usually, the dimension of the color conversion matrix depends on the dimension of the color space (RGB space is a 3x3 matrix). Finally, output the final color conversion matrix (M) for subsequent calculation of tongue image color conversion.

[0145] in, Fig.14 middle is the fitting function after the polynomial regression model coefficients are split, where R, G, B represent the input red, green, and blue components respectively, and a, b, c, e, f, g, i, j, and k are model coefficients. And after matrix conversion, the formula f is expressed in the form of matrix multiplication, the matrix Matrix is ​​output, and the matrix Matrix is ​​converted into a color conversion matrix M, where .

[0146] S106 , correcting the color values ​​of the pixels in the image to be processed using the color conversion matrix to obtain a corrected image.

[0147] Use the color conversion matrix to perform color correction on all pixels in the image to be processed. Apply the color conversion matrix to the RGB values ​​of the image to be processed and generate a color-corrected image. In this way, each pixel in the image to be processed will have its color value adjusted according to the result of the polynomial fitting, ensuring that the corrected image matches the standard color value of the colorimetric card, thereby generating the final corrected image.

[0148] The generated color conversion matrix is ​​used to perform color correction conversion on the RGB values ​​in the image to be processed, and the color values ​​of all pixels in the image to be processed are adjusted to the corrected color values, thereby generating the final corrected image. Fig.15 As shown, the specific operation process is as follows: read the image to be processed and extract the RGB value of each pixel in the image to be processed. The original color values ​​in these images to be processed will be adjusted to the standard state in subsequent steps; use the color conversion matrix to perform color conversion on the RGB value of each pixel; recombine the corrected color values ​​of all pixels in the image to be processed into a complete image. At this point, the colors of all pixels in the corrected image have been mapped and adjusted to the colors in the standard colorimetric card; finally, output the color-converted corrected image. The color distribution of the corrected image is more in line with the actual standard and can be used for subsequent intelligent analysis and auxiliary diagnosis of traditional Chinese medicine tongue diagnosis. Among them, when using the color conversion matrix for color conversion, the following matrix operation is applied to each pixel:

[0149]

[0150] in:

[0151] is the RGB value of the original pixel in the image to be processed, is the corrected pixel value in the corrected image; is the corresponding element in the color conversion matrix M.

[0152] Furthermore, after the color correction is completed, the corrected image is saved to the built-in storage space of the color correction robot. The stored corrected image data will be properly saved by the system for subsequent use or further analysis. Ensure the security and traceability of the corrected image data, including the version information, generation date and time of the corrected image. In addition, the hash value (such as MD5) of the generated file is checked for integrity and regularly backed up to a secure storage medium or cloud to prevent data loss or damage.

[0153] For subsequent use or further analysis, the color-corrected corrected image is output to the built-in storage space of the color correction robot to ensure the security and traceability of the corrected tongue image data. To ensure that the corrected image strikes a balance between quality and storage efficiency, the image is stored in a lossless compressed PNG format and uses a traceable file naming format that contains version information, date, time, and image number. For example: Corrected_ver001_20240101_001.png. To ensure the security of the image, the hash value (MD5) of the generated file is checked for integrity after saving, and is regularly backed up to a secure storage medium or cloud to prevent data loss.

[0154] In order to allow users to intuitively view and evaluate the effect of color correction, a visualization function of the corrected image is designed. This function can help users quickly understand the results of color correction and determine whether it meets the expected standards.

[0155] The saved correction image is read from the storage space and output to the robot's built-in display. Through the display, the user can intuitively view and evaluate the effect of color correction and judge whether the correction image meets the expected standards. This function provides a direct interactive method, making the color correction process more transparent and easy to understand, ensuring that the final tongue image meets the quality requirements of TCM tongue diagnosis.

[0156] In order to intuitively display the effect of color correction, a convenient evaluation and analysis interface will be provided for users. The saved correction image will be read from the built-in storage space of the color correction robot and output to the robot's built-in display for visual display. Through the display, users can clearly see the color-corrected tongue image and quickly evaluate the correction effect by comparing it with the original image.

[0157] The image color correction method provided in the embodiment of the present application mainly includes the following advantages:

[0158] First, the accuracy and adaptability of the color correction model are improved by adjusting the weight of the color value.

[0159] The color values ​​are fitted by a polynomial regression model, and the fitting process is optimized by combining a weight adjustment algorithm for color correction. A correction mechanism that can be adjusted adaptively is provided for the color deviation problem of the image to be processed under different lighting conditions and shooting angles. The weight adjustment algorithm optimizes the color values ​​of different color blocks so that the model can more accurately reflect the color change rules between color blocks during fitting, thereby achieving more accurate color correction. The polynomial regression model fitting method based on weight adjustment has strong adaptability and can cope with complex lighting and angle changes. It has extremely high practical value and innovation.

[0160] Second, by detecting a large amount of raw color value data extracted by the segmentation module, the stability and reliability of the system are enhanced.

[0161] Based on the polynomial regression model fitting of a large amount of color value raw data, a large amount of color value raw data is used to fit the polynomial regression model to build a high-precision color correction model. With rich color value raw data, the model can more comprehensively capture the color change rules in the actual shooting scene. This makes the polynomial regression model show better stability and reliability under different shooting conditions and environments. A large amount of color value raw data covers more lighting conditions and shooting angle changes, making the model more accurate when dealing with complex lighting conditions and different angles. This precise fitting capability significantly improves the accuracy of color correction, making the final color correction result closer to the actual color value.

[0162] Third, realize the automation of colorimetric card detection and color block segmentation to improve efficiency and accuracy.

[0163] In the image color correction process, a modular and automated solution for processing colorimetric cards and color block data in images is proposed by combining the colorimetric card detection model with the color block segmentation model, which provides a large amount of color value raw data for the subsequent color correction module. The colorimetric card detection model is used to automatically identify the position and boundary of the colorimetric card in the image to be processed, while the color block segmentation model further accurately segments each color block in the colorimetric card area image. Through this organic combination, the system can efficiently and accurately extract the color information of the colorimetric card, greatly reducing the workload of manual annotation and intervention. This automated detection and segmentation process is highly flexible and scalable, and can be applied to a variety of image processing scenarios. It not only improves processing efficiency, but also ensures the accuracy of the results, and is suitable for image color correction tasks.

[0164] Fourth, the robot system can realize full process automation and integration, reduce labor costs and improve overall processing efficiency.

[0165] The colorimetric card detection and segmentation module and the color correction module are embedded in the robot system to form a full-stage integrated robot image color correction system solution. The robot automatically collects the image to be processed through the camera, and performs the processes of colorimetric card detection, color block segmentation, color correction, etc. in sequence to ensure that each step is seamlessly connected. The system design covers the entire process from image acquisition to processing, correction, storage, and final visualization. Medical users only need to use the robot system to complete the data acquisition and color correction of the entire image. This highly automated design not only reduces manual intervention and reduces operating costs, but also improves processing speed and accuracy, greatly improves the stability and reliability of the system, and is suitable for a variety of medical and scientific research application scenarios. The storage and visualization functions embedded in the system also enable users to easily manage data and evaluate the correction effect in real time, further enhancing the practicality and convenience of the system.

[0166] Fig.16 FIG. 1 is a schematic diagram of a structure of an image color correction device in an embodiment of the present application. Fig.16 As shown, the image color correction device 40 provided in the embodiment of the present application mainly includes: an image acquisition module 41, a color value extraction module 42, a weight setting module 43, a fitting module 44, a matrix generation module 45 and a color correction module 46.

[0167] Among them, the image acquisition module 41 is used to acquire the image to be processed, wherein the image to be processed includes a colorimetric card area, and the colorimetric card area includes multiple color blocks; the color value extraction module 42 is used to extract the color value of each pixel in the multiple color blocks to obtain a color value list; the weight setting module 43 is used to set a weight coefficient for each color value in the color value list; the fitting module 44 is used to fit a polynomial regression model using a color value list with a weight coefficient to obtain a fitted polynomial regression model; the matrix generation module 45 is used to generate a color conversion matrix using the fitted polynomial regression model, and the color conversion matrix is ​​used to represent the mapping relationship between the color values ​​in the image to be processed and the standard color values ​​in the colorimetric card; the color correction module 46 is used to correct the color values ​​of the pixels in the image to be processed using the color conversion matrix to obtain a corrected image.

[0168] The embodiment of the present application provides a device for correcting image color, which is used to perform the following process: obtaining an image to be processed, wherein the image to be processed includes a colorimetric card area, and the colorimetric card area includes multiple color blocks; extracting the color values ​​of each pixel in the multiple color blocks to obtain a color value list; setting weight coefficients for each color value in the color value list; fitting a polynomial regression model using the color value list with weight coefficients to obtain a fitted polynomial regression model; using the fitted polynomial regression model to generate a color conversion matrix, the color conversion matrix is ​​used to represent the mapping relationship between the color values ​​in the image to be processed and the standard color values ​​in the colorimetric card; using the color conversion matrix to correct the color values ​​of each pixel in the image to be processed to obtain a corrected image. This embodiment improves the color accuracy of the image, ensures the reliability and authenticity of the image information, and thus provides reliable data support for subsequent intelligent analysis and auxiliary diagnosis.

[0169] In a possible implementation, the color value extraction module 42 is specifically used to detect the image to be processed using a colorimetric card detection model to obtain the position information of the colorimetric card area; crop the image to be processed based on the position information of the colorimetric card area to obtain a colorimetric card area image; identify and segment the colorimetric card area image using a color block segmentation model to obtain the coordinates of each pixel in each color block; based on the coordinates of each pixel in each color block, extract the color value of each pixel in each color block to obtain a color value list.

[0170] In a possible implementation, the color value extraction module 42 is specifically used to extract the color value of each pixel in the image to be processed based on the coordinate value of each pixel; establish an association relationship between the number of the color block and the color value of each pixel, and store them in a color value list.

[0171] In one possible implementation, the weight setting module 43 is specifically used to set a weight coefficient for each color value in the color value list, including: counting the color value of each pixel in the color value list to obtain the color distribution of each channel; analyzing based on the color distribution of each channel to obtain the first parameter and the second parameter corresponding to each channel, wherein the first parameter is half of the channel width, and the second parameter is the channel value with the largest proportion in the channel; calculating the weight coefficient of each channel based on the first parameter and the second parameter corresponding to each channel and the set weight calculation formula.

[0172] In one possible implementation, the weight calculation formula is as follows

[0173] Among them, k represents the weight coefficient, is the natural logarithm, is the first parameter, is the second parameter.

[0174] In one possible implementation, the fitting module 44 is specifically used to fit the color values ​​and color block numbers in the color value list to obtain an initial polynomial regression model; the initial polynomial regression model is iteratively adjusted using the weight coefficients of each color value in the color value list to obtain a fitted polynomial regression model, wherein the fitted polynomial regression model includes a polynomial regression model that minimizes the loss function.

[0175] In a possible implementation, the matrix generation module 45 is specifically used to split the coefficients in the fitted polynomial regression model to obtain a polynomial regression model after coefficient splitting; and combine the coefficients of the polynomial regression model after coefficient splitting into a color conversion matrix.

[0176] The image color correction device provided in the embodiment of the present application can execute the image color correction method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0177] Fig.17 is a schematic diagram of the structure of an electronic device provided in this embodiment. The electronic device may include an image color correction device, such as Fig.17 As shown, the electronic device 500 includes a processor 510, a memory 520, an input device 530, and an output device 540; the number of the processor 510 in the electronic device can be one or more. Fig.17 A processor 510 is taken as an example; the processor 510, the memory 520, the input device 530 and the output device 540 in the electronic device can be connected via a bus or other means. Fig.17 The example of connecting through bus is taken in the following.

[0178] The memory 520, as a computer-readable storage medium, can be used to store software programs, computer executable programs and modules, such as program instructions / modules corresponding to the image color correction method in the embodiment of the present invention. The processor 510 executes various functional applications and data processing of the electronic device by running the software programs, instructions and modules stored in the memory 520, that is, implements the image color correction method provided in the embodiment of the present invention.

[0179] The memory 520 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system and at least one application required for a function; the data storage area may store data created according to the use of the terminal, etc. In addition, the memory 520 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some instances, the memory 520 may further include a memory remotely arranged relative to the processor 510, and these remote memories may be connected to the electronic device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0180] The input device 530 may be used to receive input digital or character information and generate signal input related to user settings and function control of the electronic device, and may include a keyboard, a mouse, etc. The output device 540 may include a display device such as a display screen.

[0181] This embodiment also provides a storage medium containing computer executable instructions, and when the computer executable instructions are executed by a computer processor, they are used to implement the image color correction method provided by the embodiment of the present invention.

[0182] Of course, the computer executable instructions of a storage medium including computer executable instructions provided by an embodiment of the present invention are not limited to the method operations described above, and can also execute related operations in the image color correction method provided by any embodiment of the present invention.

[0183] Through the above description of the implementation methods, the technicians in the relevant field can clearly understand that the present invention can be implemented by means of software and necessary general hardware, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, a read-only memory (ROM), a random access memory (RAM), a flash memory (FLASH), a hard disk or an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0184] It is worth noting that in the embodiment of the above-mentioned image color correction device, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention.

[0185] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0186] The above description is only a specific implementation of the present application, so that those skilled in the art can understand or implement the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments described herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for correcting image color, characterized in that: The method comprises: Acquire an image to be processed, wherein the image to be processed includes a colorimetric card area, and the colorimetric card area includes a plurality of color blocks; Extracting the color value of each pixel in the plurality of color blocks to obtain a color value list; Setting a weight coefficient for each color value in the color value list, wherein the setting of the weight coefficient for each color value in the color value list includes: performing statistics on the color value of each pixel in the color value list to obtain the color distribution of each channel; performing analysis based on the color distribution of each channel to obtain a first parameter and a second parameter corresponding to each channel, wherein the first parameter is half of the channel width, and the second parameter is the channel value with the largest proportion in the channel; calculating the weight coefficient of each channel based on the first parameter and the second parameter corresponding to each channel and a set weight calculation formula; The polynomial regression model is fitted using the color value list with weight coefficients to obtain the fitted polynomial regression model; Generate a color conversion matrix using the fitted polynomial regression model, wherein the color conversion matrix is ​​used to represent the mapping relationship between the color values ​​in the image to be processed and the standard color values ​​in the colorimetric card; The color conversion matrix is ​​used to correct the color values ​​of the pixels in the image to be processed to obtain a corrected image.

2. The method according to claim 1, characterized in that The step of extracting the color value of each pixel in the plurality of color blocks to obtain a color value list includes: The colorimetric card detection model is used to detect the image to be processed to obtain the position information of the colorimetric card area; Cutting the image to be processed based on the position information of the colorimetric card area to obtain a colorimetric card area image; Using a color block segmentation model to identify and segment the colorimetric card area image, and obtain the coordinates of each pixel in each color block; Based on the coordinates of each pixel point in each color block, the color value of each pixel point in each color block is extracted to obtain a color value list.

3. The method according to claim 2, characterized in that The step of extracting the color value of each pixel in each color block based on the coordinates of each pixel in each color block to obtain a color value list includes: Extracting the color value of each pixel in the image to be processed based on the coordinate value of each pixel; An association relationship is established between the number of the color block and the color value of each pixel, and the association relationship is stored in a color value list.

4. The method according to claim 1, characterized in that The calculation formula for setting the weight is as follows: Among them, k represents the weight coefficient, is the natural logarithm, is the first parameter, is the second parameter, It is the channel value of the current channel of the pixel.

5. The method according to claim 4, characterized in that The fitting of the polynomial regression model using the color value list with weight coefficients to obtain the fitted polynomial regression model includes: Using the color values ​​in the color value list and the numbers of the color blocks to perform fitting, an initial polynomial regression model is obtained; The initial polynomial regression model is iteratively adjusted using the weight coefficients of the respective color values ​​in the color value list to obtain a fitted polynomial regression model, wherein the fitted polynomial regression model includes a polynomial regression model that minimizes a loss function.

6. The method according to claim 5, characterized in that The method of generating a color conversion matrix using the fitted polynomial regression model includes: Splitting the coefficients in the fitted polynomial regression model to obtain a polynomial regression model after coefficient splitting; Combine the coefficients of the polynomial regression model after coefficient splitting into a color transformation matrix.

7. A device for correcting image color, characterized in that: The device comprises: An image acquisition module, used for acquiring an image to be processed, wherein the image to be processed includes a colorimetric card area, and the colorimetric card area includes a plurality of color blocks; A color value extraction module, used to extract the color value of each pixel in the plurality of color blocks to obtain a color value list; A weight setting module, used for setting a weight coefficient for each color value in the color value list, wherein the setting of the weight coefficient for each color value in the color value list comprises: performing statistics on the color value of each pixel in the color value list to obtain the color distribution of each channel; performing analysis based on the color distribution of each channel to obtain a first parameter and a second parameter corresponding to each channel, wherein the first parameter is one-half of the channel width, and the second parameter is the channel value with the largest proportion in the channel; calculating the weight coefficient of each channel based on the first parameter and the second parameter corresponding to each channel and a set weight calculation formula; A fitting module, used for fitting a polynomial regression model using a color value list with weight coefficients to obtain a fitted polynomial regression model; A matrix generation module, used to generate a color conversion matrix using the fitted polynomial regression model, wherein the color conversion matrix is ​​used to represent the mapping relationship between the color values ​​in the image to be processed and the standard color values ​​in the colorimetric card; The color correction module is used to correct the color values ​​of the pixels in the image to be processed by using the color conversion matrix to obtain a corrected image.

8. A color correction robot, characterized in that: The color correction robot comprises: one or more processors; A storage device for storing one or more programs; A camera, used for collecting images to be processed and transmitting them to the processor; When the one or more programs are executed by the one or more processors, the one or more processors implement the image color correction method according to any one of claims 1 to 6.

9. A storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the image color correction method according to any one of claims 1 to 6 is implemented.

Images