Method and system based on tongue diagnosis image recognition

Abstract

The invention relates to the technical field of computer-aided diagnosis, in particular to a method and system based on tongue diagnosis image recognition, and the method comprises the following steps: generating a color correction matrix and an initial space coordinate system through a standard color palette, capturing a tongue body image, correcting the color and posture of the tongue body image, constructing a pixel-level multi-spectral feature vector, and obtaining a multi-spectral feature vector; and dividing pixels through a K-means clustering algorithm, extracting area, color and shape features, fusing the area, color and shape features into a comprehensive vector, matching a discrimination rule, and outputting a tongue picture classification analysis result. According to the method, image colors are standardized through a calibration color plate, accurate tongue color analysis is ensured, position offset and physical jitter are compensated in combination with posture reading, pixel-level multi-spectral feature vectors are constructed to mine tongue picture details, objective data such as quantitative segmentation and calculation area and colors are subjected to clustering algorithm quantitative segmentation, and form and color feature comprehensive judgment is fused. A standardized analysis result is output, and conversion from subjective experience to objective data driving is achieved.

Description

Technical Field

[0001] This invention relates to the field of computer-aided diagnostic technology, and in particular to a method and system based on tongue diagnosis image recognition. Background Technology

[0002] Computer-aided diagnostic technology is an important branch of medical informatics and artificial intelligence. It primarily involves using computer image processing, pattern recognition, and medical knowledge bases to analyze and judge clinical symptoms or medical images. Its core aspects include the acquisition and preprocessing of medical image data, feature extraction and recognition, the establishment and reasoning of diagnostic models, and the output of decision support. This technology aims to achieve objectivity and standardization in diagnosis through the digital and intelligent analysis of physiological signals and image information, broadly covering medical image recognition, clinical data mining, and pathological image-assisted analysis. Traditional tongue diagnosis, in the context of Traditional Chinese Medicine (TCM), refers to inferring the body's internal pathological state by observing the patient's tongue shape, color, and coating thickness. It addresses the diagnostic needs based on the acquisition and analysis of tongue features. Traditional tongue diagnosis typically relies on the physician's visual observation and experience to extract and analyze information. Common methods include manually comparing tongue images to identify symptom characteristics or making manual judgments based on TCM tongue diagnosis theory, thereby completing the auxiliary diagnosis of the patient's health status.

[0003] The existing diagnostic process relies excessively on the physician's personal experience and subjective visual judgment, which can easily affect the accuracy and consistency of the diagnostic results. For example, when observing the same tongue under different lighting conditions, its color and luster will show significant differences, leading to different judgments by physicians. At the same time, even slight changes in angle or position during collection can cause visual distortion of the tongue's morphological characteristics. Furthermore, the descriptive conclusions of traditional tongue diagnosis are difficult to quantify accurately and make longitudinal comparisons, making it difficult to have objective evidence for the dynamic tracking of subtle changes in the condition. This non-standardized diagnostic method limits its application in scenarios that require high consistency and repeatability. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of existing technologies and to propose a method and system based on tongue diagnosis image recognition.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: a method based on tongue diagnosis image recognition, comprising the following steps: S1: The mapping relationship between the red, green, and blue channel values ​​of the standard color chart image generated by illuminating the standard color chart with a monochromatic light source and a white light source and the pre-stored standard color values ​​is used to generate a color correction matrix, and an initial spatial coordinate system is constructed based on the collected initial attitude readings. S2: Capture tongue images and collect real-time posture readings, call the color correction matrix to transform the color values ​​of the tongue images, and generate a multispectral image sequence and a set of real-time posture readings; S3: Generate a perspective compensation matrix and attitude compensation transformation parameters based on the deviation between the real-time attitude reading set and the initial spatial coordinate system. Perform global perspective correction on the multispectral image sequence. Then, select the initial tongue region from the white light tongue image. At the same time, call the attitude compensation transformation parameters to correct the initial tongue region. Calculate the second-order central moment of the initial tongue region. Derive the principal axis tilt angle of the tongue. Based on this, construct a rotation matrix to perform principal axis alignment transformation on the initial tongue region. Combine pixel brightness values ​​to construct a pixel-level multispectral feature vector set. S4: Input the set of pixel-level multispectral feature vectors into the K-means clustering algorithm to classify pixel categories, calculate the area ratio and average color value of the pixel categories, extract the shape values ​​of the initial tongue region, and fuse them into a comprehensive feature vector. Match the comprehensive feature vector with the discrimination rules and output the tongue image classification analysis results.

[0006] As a further aspect of the present invention, the color correction matrix includes red channel correction coefficients, green channel correction coefficients, and blue channel correction coefficients; the initial spatial coordinate system specifically comprises three-axis rotation components and three-axis translation components; the multispectral image sequence includes white light tongue image frames and monochromatic light tongue image frames; the real-time attitude reading set specifically refers to real-time gyroscope readings and real-time accelerometer readings; the attitude compensation transformation parameters include rotation matrices and translation vectors; the pixel-level multispectral feature vector set includes pixel position indices and multispectral response vectors; the comprehensive feature vector includes region area percentage values, average color values, and tongue shape values; and the tongue image classification analysis results specifically refer to tongue image category identifiers and confidence scores.

[0007] As a further aspect of the present invention, the step of obtaining the initial spatial coordinate system specifically includes: S101: Obtain a standard color swatch and illuminate it with a monochromatic light source and a white light source respectively. Simultaneously acquire standard color swatch images, analyze the standard color swatch images, extract the pixel values ​​of the red, green and blue channels within the image frame, and combine the extracted pixel values ​​into a multidimensional array to generate a standard color swatch image pixel value matrix. S102: Call the standard color swatch image pixel value matrix and obtain the pre-stored standard color values. Construct a linear transformation equation between the multiple channel values ​​in the standard color swatch image pixel value matrix and the color components corresponding to the standard color values. Solve the linear transformation equation to obtain the transformation coefficients. Organize the transformation coefficients into a third-order matrix to generate a color correction matrix. S103: Collect initial attitude readings of roll angle, pitch angle and yaw angle data from the attitude sensor. Based on the initial attitude readings, define the roll angle, pitch angle and yaw angle data as the first, second and third rotation axes of the spatial coordinate system, respectively, and use them as a reference to construct the initial spatial coordinate system.

[0008] As a further aspect of the present invention, the steps for obtaining the multispectral image sequence and the real-time attitude reading set are specifically as follows: S201: The image sensor captures a single frame of tongue image, and the attitude sensor collects real-time attitude readings simultaneously. The color information of multiple pixels in the tongue image is deconstructed into red, green and blue three-channel intensity values ​​to generate an original tongue RGB image. The original data from the gyroscope and accelerometer in the real-time attitude readings are integrated to establish an initial attitude parameter set. S202: Obtain a preset color correction matrix, construct a pixel color vector based on the red, green and blue three-channel intensity values ​​of each pixel position in the original tongue RGB image, call the color correction matrix and the pixel color vector to perform matrix multiplication, replace the red, green and blue three-channel intensity values ​​with the transformation vector generated by the operation, and obtain the corrected spectral data frame; S203: For the multiple calibration spectral data frames acquired within a continuous sampling period and the initial attitude parameter set, stack the multiple calibration spectral data frames along the time axis to construct a multispectral image sequence, and retrieve the acquisition timestamp corresponding to the initial attitude parameter set and match it with the image frame index of the multispectral image sequence to generate the multispectral image sequence and the real-time attitude reading set.

[0009] As a further aspect of the present invention, the step of obtaining the pixel-level multispectral feature vector set specifically includes: S301: Calculate the homography matrix based on the deviation between the real-time attitude reading set and the initial spatial coordinate system, eliminate perspective distortion caused by the tilt of the shooting angle, calculate the zero-order moment M00 and the first-order moments M10 and M01 of the initial tongue region to determine the centroid coordinates, and calculate the deflection angle α of the tongue principal axis relative to the vertical direction by combining the second-order central moments μ20, μ11 and μ02, and generate attitude compensation transformation parameters including rotation and translation components. S302 calls the white light tongue image of the multispectral image sequence, sets the pixel chromaticity range and the lower limit threshold of brightness, traverses all pixels of the white light tongue image, removes background pixels that do not conform to the pixel chromaticity range and the lower limit threshold of brightness, filters to obtain the initial tongue region, calls the posture compensation transformation parameters to perform inverse geometric transformation on the pixel coordinates of the initial tongue region, and obtains the corrected tongue region pixel set. S303: Traverse the pixel positions recorded in the pixel set of the corrected tongue region, extract pixel brightness values ​​one by one from the N spectral channel images of the multispectral image sequence according to the pixel positions, arrange the extracted N pixel brightness values ​​in order of spectral channel wavelength from shortest to longest to form an N-dimensional feature vector, aggregate the N-dimensional feature vectors of all the pixels, and construct a pixel-level multispectral feature vector set.

[0010] As a further aspect of the present invention, the steps for obtaining the tongue image classification analysis results are specifically as follows: S401: For the set of pixel-level multispectral feature vectors, iteratively divide them according to the Euclidean distance between the vectors until the cluster centers are stable, forming several pixel categories. Calculate the ratio of the number of pixels in each of the pixel categories to the total number of pixels in the initial tongue region, and calculate the area ratio. At the same time, perform an arithmetic mean operation on the color values ​​of all pixels in each of the pixel categories to obtain the average color value. Finally, combine the area ratio of all pixel categories with the average color value to generate a pixel category attribute set. S402: Based on the contour coordinates of the initial tongue region, calculate the ratio of its perimeter to its area as a shape value, call the pixel category attribute set, and serialize and concatenate the shape value with the area ratio and the average color value in the pixel category attribute set to establish a comprehensive feature vector. S403: The area ratio, average color value, and shape value of the comprehensive feature vector are compared one by one with the preset value range threshold for the differentiated tongue image category in the discrimination rule. When all the values ​​of the comprehensive feature vector fall within the preset threshold range of a tongue image category, the matching is determined to be successful, and the tongue image classification analysis result is obtained.

[0011] As a further aspect of the present invention, in the initial tongue region screening step, the method for setting the pixel chromaticity range and the lower limit threshold of brightness is specifically as follows: Acquire the white light tongue image, calculate the three-channel pixel value distribution histogram of the white light tongue image in the hue, saturation and brightness color space, and generate the hue distribution map, saturation distribution map and brightness distribution map; Analyze the brightness distribution map, use a peak detection algorithm to identify the main peak and secondary peak in the brightness distribution map, define the brightness value corresponding to the main peak as the background brightness center, and define the brightness value corresponding to the secondary peak as the target brightness center; The lower limit threshold of brightness is calculated based on the target brightness center and the preset brightness standard deviation multiple. At the same time, the pixels whose brightness values ​​are in the neighborhood of the target brightness center are analyzed, and the distribution range of the pixels in the hue distribution map and the saturation distribution map is determined. The pixel distribution range is then used as the pixel chromaticity range. Based on the calculated lower limit of brightness and the pixel chromaticity range, each pixel in the white light tongue image is traversed and judged, and the pixels that meet the conditions are selected to generate the initial tongue region. The preset brightness standard deviation multiple is an empirical value determined based on statistical analysis of multiple tongue images, and the value ranges from 2 to 4.

[0012] As a further aspect of the present invention, the condition for determining the stability of cluster centers in the step of dividing pixel categories using the K-means clustering algorithm is specifically as follows: K multispectral response vectors are randomly selected from the set of pixel-level multispectral feature vectors as initial cluster centers; For each of the multispectral response vectors in the pixel-level multispectral feature vector set, calculate the Euclidean distance with the multiple cluster centers, and assign the multispectral response vector to the pixel category corresponding to the nearest cluster center; Recalculate the mean of all multispectral response vectors within each pixel category, and update the mean to the current cluster center; Repeat the assignment and update steps, and determine whether the clustering process has converged using the following formula:

[0013] Where ΔC represents the average change of all cluster centers between two consecutive iterations, K represents the preset number of pixel categories, and k is the pixel category index from 1 to K. This represents the cluster center vector of the k-th pixel category at the t-th iteration. This represents the cluster center vector of the pixel category in the previous iteration, i.e., the (t-1)th iteration. The Euclidean norm of a vector. This represents the preset cluster center convergence threshold; The preset number of pixel categories K is determined based on the common classification numbers of tongue coating and coating color in the theory of tongue diagnosis, and the value ranges from 3 to 7. When the ΔC is less than or equal to the When the clustering process converges, several pixel categories are generated.

[0014] As a further aspect of the present invention, the calculation steps for the confidence score in the tongue image classification analysis results are as follows: Once the comprehensive feature vector is successfully matched with the discrimination rule of a certain tongue image category, the preset numerical range threshold of the tongue image category is obtained, including the upper and lower limits of each feature. The confidence score is calculated by retrieving the area percentage, average color, and tongue shape values ​​from the comprehensive feature vector, and using the following formula:

[0015] Among them, S conf The confidence score represents the total number of dimensions of the comprehensive feature vector, and j is the feature index from 1 to M. j The preset weight coefficients representing the j-th feature, and the sum of all weight coefficients is 1, v j U represents the actual value of the j-th dimension in the comprehensive feature vector. j L represents the upper limit of the preset numerical range of the j-th feature that matches the tongue image category. j The lower limit of the preset numerical range of the representative feature, δ represents the smoothing factor set to avoid a denominator of zero; The preset weighting coefficient w of the j-th feature j It is determined by the contribution of differential features to the tongue image classification results, through expert scoring or machine learning model training. The calculated confidence score is combined with the tongue image category identifier of the tongue image category to generate the tongue image classification analysis result.

[0016] A system based on tongue diagnosis image recognition, the system being used to implement the above-described method based on tongue diagnosis image recognition, the system comprising: The benchmark construction module is used to generate a color correction matrix by mapping the red, green, and blue channel values ​​of the standard color chart image generated by illuminating the standard color chart with a monochromatic light source and a white light source to the pre-stored standard color values. Based on the acquired initial posture readings, it constructs an initial spatial coordinate system and transmits the color correction matrix and the initial spatial coordinate system to the tongue image acquisition and color calibration module. The tongue image acquisition and color correction module is used to capture tongue images and collect real-time posture readings. It calls the color correction matrix to transform the color values ​​of the tongue image, generates a multispectral image sequence and a set of real-time posture readings, and transmits the multispectral image sequence, the set of real-time posture readings, and the initial spatial coordinate system to the posture correction and feature construction module. The attitude correction and feature construction module is used to generate attitude compensation transformation parameters based on the deviation between the real-time attitude reading set and the initial spatial coordinate system, select the initial tongue region from the white light tongue image of the multispectral image sequence, and simultaneously call the attitude compensation transformation parameters to correct the initial tongue region. The module combines pixel brightness values ​​to construct a pixel-level multispectral feature vector set and transmits it to the comprehensive discrimination analysis module. The comprehensive discriminant analysis module is used to input the set of pixel-level multispectral feature vectors into the K-means clustering algorithm to classify pixel categories, calculate the area ratio and average color value of the pixel categories, extract the shape values ​​of the initial tongue region, and fuse them into a comprehensive feature vector. The module then matches the comprehensive feature vector with the discrimination rules and outputs the tongue image classification analysis results.

[0017] Compared with the prior art, the advantages and positive effects of the present invention are as follows: In this invention, standard color charts under monochrome and white light sources are calibrated to standardize image colors, ensuring the accuracy and consistency of tongue color analysis. Combined with posture readings to compensate for positional offsets during the acquisition process, the interference of physical jitter on tongue morphology recognition is eliminated. Pixel-level multispectral feature vectors are constructed to deeply mine subtle feature information of the tongue image. Then, clustering algorithms are used to quantify and segment the tongue image features, calculate objective data such as the area ratio and color mean of each region, and integrate morphological and color features for comprehensive discrimination, outputting standardized analysis results, thus realizing the transformation from subjective experience-based to objective data-driven analysis. Attached Figure Description

[0018] Figure 1 This is an overall flowchart of the tongue diagnosis image recognition method of the present invention; Figure 2 This is a flowchart illustrating the construction process of the color correction matrix and initial spatial coordinate system of this invention. Figure 3 This is a flowchart illustrating the process of acquiring multispectral image sequences and real-time attitude reading sets according to the present invention. Figure 4 This is a flowchart of the tongue region correction and feature vector construction process of the present invention; Figure 5 This is a flowchart of the output of tongue image classification and analysis results in this invention. Detailed Implementation

[0019] In the description of this invention, the system architecture relationships or data processing flows indicated by terms such as "layer," "module," "interface," "data flow," "client," and "server" are all defined based on the architecture diagram or flowchart corresponding to the embodiments. This way of describing is only used to clearly illustrate the logical relationships between the elements in the technical solution, and not to limit the physical deployment form. The term "multiple" includes two or more technical units, including but not limited to multiple data nodes, processing threads, service instances, or functional components and other scalable elements. The specific number is determined according to the actual business scenario and needs to be specifically specified.

[0020] Please see Figure 1 and Figure 2 This invention provides a method for tongue diagnosis image recognition, comprising the following steps: S1: The mapping relationship between the red, green, and blue channel values ​​of the standard color chart image generated by illuminating the standard color chart with a monochromatic light source and a white light source and the pre-stored standard color values ​​is used to generate a color correction matrix, and an initial spatial coordinate system is constructed based on the collected initial attitude readings. The color correction matrix includes red channel correction coefficients, green channel correction coefficients, and blue channel correction coefficients; The initial spatial coordinate system consists of three rotational components and three translational components. The specific steps for obtaining the initial spatial coordinate system are as follows: S101: Obtain a standard color swatch and illuminate it with a monochromatic light source and a white light source respectively. Simultaneously acquire standard color swatch images, analyze the standard color swatch images, extract the pixel values ​​of the red, green and blue channels within the image frame, and combine the extracted pixel values ​​into a multidimensional array to generate a standard color swatch image pixel value matrix. S102: Call the standard color swatch image pixel value matrix and obtain the pre-stored standard color values. Construct a linear transformation equation between the multiple channel values ​​in the standard color swatch image pixel value matrix and the corresponding color components of the standard color values. Solve the linear transformation equation to obtain the transformation coefficients. Organize the transformation coefficients into a third-order matrix form to generate a color correction matrix. S103: Collect initial attitude readings of roll angle, pitch angle and yaw angle data from the attitude sensor. Based on the initial attitude readings, define the roll angle, pitch angle and yaw angle data as the first, second and third rotation axes of the spatial coordinate system, respectively, and use them as a reference to construct the initial spatial coordinate system.

[0021] In the color correction matrix generation stage, a standard X-RiteColorCheckerClassic color chart is first prepared, containing 24 standard color patches. The CIELAB standard color values ​​for each patch are pre-stored in a local database. In a closed environment with a constant illuminance of 500 Lux and a color temperature of 6500 K, the standard color chart is placed 30 cm directly in front of the image acquisition device. The image acquisition device uses a CMOS sensor with a resolution of 1920x1080 pixels. First, a 630nm monochromatic red LED light source is turned on to illuminate the standard color chart, and 10 frames of images are captured and stored. Then, the red light source is turned off, and a 525nm monochromatic green LED light source and a 465nm monochromatic blue LED light source are turned on sequentially to illuminate the color chart, and 10 frames of images are captured for each. Finally, all monochromatic light sources are turned off, and a white LED light source is turned on, and 10 frames of images are captured again.

[0022] Taking a frame under white light illumination as an example, the system analyzes the acquired images. First, it locates 24 standard color patch regions in the image and calculates the arithmetic mean of the red (R), green (G), and blue (B) pixel values ​​for all pixels within each color patch region. For example, the table below shows an example of the pre-stored standard RGB color values ​​and the actual RGB values ​​obtained through image acquisition and analysis for the color patch 'Darkskin'. Similarly, the table below shows an example of the standard values ​​and acquired values ​​for the color patch 'Bluesky'. The table below also shows the standard values ​​and actual acquired values ​​for three non-linearly related color patches, such as 'Darkskin', 'Bluesky', and 'Bluishgreen'.

[0023] Based on these three sets of data, construct a system of linear transformation equations: , Similarly, by solving this system of linear equations, a 3×3 transformation coefficient matrix is ​​obtained. In this embodiment, by performing least squares fitting calculations on all 24 color patch data, a more universal color correction matrix is ​​finally obtained, with the following specific values:

[0024] This matrix is ​​the color correction matrix used in subsequent steps. The first row [0.92, -0.11, 0.05] is the red channel correction coefficient, the second row [0.08, 0.95, -0.03] is the green channel correction coefficient, and the third row [-0.04, 0.07, 0.91] is the blue channel correction coefficient.

[0025] In the initial spatial coordinate system construction phase, the image acquisition device integrates a six-axis attitude sensor (MPU-6050). With the device stationary on a horizontal testing platform, 100 data points are continuously acquired from the attitude sensor. Each data point includes readings of roll, pitch, and yaw angles. The arithmetic mean of these 100 data points is calculated to eliminate random noise. For example, the calculated average is: roll 0.12°, pitch -0.08°, and yaw 25.3°. Using this set of initial attitude readings as a reference, a right-handed coordinate system is defined: the initial rotational state is defined with a roll angle of 0.12° as the X-axis (first rotational axis), a pitch angle of -0.08° as the Y-axis (second rotational axis), and a yaw angle of 25.3° as the Z-axis (third rotational axis). Simultaneously, the device's translational position in three-dimensional space (0, 0, 0) is recorded as the reference for the translational components. Thus, an initial spatial coordinate system was constructed, which includes three-axis rotational components (0.12°, -0.08°, 25.3°) and three-axis translational components (0, 0, 0).

[0026] Please see Figure 1 and Figure 3 S2: Capture tongue images and acquire real-time attitude readings, call the color correction matrix to transform the color values ​​of the tongue images, and generate a multispectral image sequence and a set of real-time attitude readings; The multispectral image sequence includes white light tongue image frames and monochromatic light tongue image frames; The real-time attitude reading set specifically refers to the real-time readings of the gyroscope and the real-time readings of the accelerometer. The specific steps for obtaining the multispectral image sequence and real-time attitude reading set are as follows: S201: The monitoring image sensor captures a single frame of tongue image, and the monitoring attitude sensor collects real-time attitude readings simultaneously. The color information of multiple pixels in the tongue image is deconstructed into red, green and blue three-channel intensity values ​​to generate the original tongue RGB image. The original data of gyroscope and accelerometer in the real-time attitude reading are integrated to establish the initial attitude parameter set. S202: Obtain the preset color correction matrix, construct the pixel color vector based on the red, green and blue three-channel intensity values ​​of each pixel position in the original tongue RGB image, call the color correction matrix and the pixel color vector to perform matrix multiplication, replace the red, green and blue three-channel intensity values ​​with the transformation vector generated by the operation, and obtain the correction spectral data frame. S203: For multiple calibration spectral data frames and initial attitude parameter sets acquired within a continuous sampling period, stack the multiple calibration spectral data frames along the time axis to construct a multispectral image sequence, and retrieve the acquisition timestamp corresponding to the initial attitude parameter set and the image frame index of the multispectral image sequence for pairing, thereby generating a multispectral image sequence and a set of real-time attitude readings.

[0027] When the user points the device at the tongue and starts the acquisition program, the image sensor captures an image of the tongue at a rate of 30 frames per second, while the attitude sensor acquires real-time attitude readings at a frequency of 100 times per second. The system processes one frame of the image and the synchronized attitude data. Assume that at timestamp T = 0.033 seconds, the image sensor captures a frame of the tongue image and simultaneously monitors a set of real-time attitude readings from the attitude sensor. For this frame of the tongue image, a pixel P in the tongue tip region is selected for processing. The original color information of this pixel is deconstructed into the intensity values ​​of the red, green, and blue channels, specifically (210, 125, 110), which is the pixel color vector in the original RGB image of the tongue. At the same time, the initial attitude parameter set at that moment is established by integrating the raw gyroscope data (e.g., angular velocity of (0.5 rad / s, -0.2 rad / s, 0.1 rad / s)) collected by the attitude sensor at T=0.033 seconds with the raw accelerometer data (e.g., acceleration of (0.01g, 0.02g, -0.99g)).

[0028] Next, obtain the preset color correction matrix generated in step S1. Construct a 1x3 pixel color vector V_orig from the red, green, and blue channel intensity values ​​(210, 125, 110) of pixel P. Then, call the color correction matrix M_calib generated in step S1:

[0029] The color correction matrix is ​​multiplied by the pixel color vector V_orig. The calculation process is as follows: R_corr=0.92*210+(-0.11)*125+0.05*110=193.2-13.75+5.5=184.95; G_corr=0.08*210+0.95*125+(-0.03)*110=16.8+118.75-3.3=132.25; B_corr=(-0.04)*210+0.07*125+0.91*110=-8.4+8.75+100.1=100.45; The generated transform vector V_corr = [184.95, 132.25, 100.45]. The result is rounded to obtain new RGB three-channel intensity values ​​(185, 132, 100), which replace the original intensity values. This process is repeated for each pixel in the image to generate a frame of corrected spectral data. This process is also applicable to tongue image frames acquired under monochromatic light sources (e.g., red or green light) to generate corresponding monochromatic tongue image frames.

[0030] Within a continuous sampling period, e.g., 1 second, the system acquires 30 frames of calibrated spectral data (including white light tongue image frames and monochromatic light tongue image frames) and 100 sets of initial attitude parameters. The system stacks these 30 frames of calibrated spectral data along the time axis in chronological order of acquisition, constructing a multispectral image sequence. Subsequently, the system retrieves the acquisition timestamp of each frame and pairs it with the attitude parameter set whose acquisition time is closest to the first frame in the 100 sets of initial attitude parameters. For example, if the timestamp of the first frame is T=0.033s, it is paired with the attitude parameter set whose acquisition time is closest to 0.033s; if the timestamp of the second frame is T=0.066s, it is paired with the attitude parameter set whose acquisition time is closest to 0.066s. In this way, each frame index of the multispectral image sequence is precisely paired with a real-time attitude reading, ultimately generating a multispectral image sequence and a set of real-time attitude readings containing 30 pairs of "calibrated spectral data frame - real-time attitude reading" data.

[0031] Please see Figure 1 and Figure 4 S3: Based on the deviation between the real-time attitude reading set and the initial spatial coordinate system, a perspective compensation matrix and attitude compensation transformation parameters are generated, and global perspective correction is performed on the multispectral image sequence. Then, the initial tongue region is selected from the white light tongue image, and the attitude compensation transformation parameters are called to correct the initial tongue region. The second central moment of the initial tongue region is calculated, the principal axis tilt angle of the tongue is derived, and a rotation matrix is ​​constructed to align the principal axis of the tongue, and a set of pixel-level multispectral feature vectors is constructed. The attitude compensation transformation parameters include the rotation matrix and the translation vector; The pixel-level multispectral feature vector set includes pixel location indices and multispectral response vectors; The specific steps for obtaining the pixel-level multispectral feature vector set are as follows: S301: Perform global perspective correction. Based on the real-time attitude reading set, the system extracts the attitude data of a certain frame (e.g., frame 15) from the real-time attitude reading set. Assume that the acquisition device is detected to be tilted relative to the initial spatial coordinate system (e.g., the pitch angle deviation is -10°, i.e., the lens is tilted downwards). At this time, physical alignment alone cannot eliminate the perspective distortion of the image (i.e., the base of the tongue appears wider than it actually is, and the tip of the tongue appears narrower than it actually is). The system uses this angle deviation to construct a perspective compensation matrix and performs inverse perspective projection transformation on the entire frame image to restore the image viewpoint to the standard frontal plane. S302: Perform tongue region segmentation and principal axis calculation. Utilize a perspective-corrected white-light tongue image and determine the brightness threshold using the "bimodal method" (as described above: primary peak for background 45, secondary peak for tongue 175, threshold 130). Combine this with chromaticity range removal to obtain a binarized initial tongue region mask. At this point, although perspective correction is achieved, the patient may exhibit head tilting or tongue deviation. To obtain accurate shape values, principal axis alignment is necessary. The system then performs image moment analysis on this binarized region: Calculate the centroid: Calculate the coordinates (x, y) of the white pixels in the statistical region, and calculate the zero-order moment M00 (area) and the first-order moments M10 and M01; The coordinates of the centroid are Cx = M10 / M00, Cy = M01 / M00; Calculate the deflection angle: Calculate the second-order central moments μ20 = ∑(x - Cx)2, μ02 = ∑(y - Cy)2, μ11 = ∑(x - Cx)(y - Cy); use the formula Solve for the deflection angle α of the principal axis of the tongue by using α = 0.5Xarctan(2μ11 / (μ20-μ02)); Alignment is performed: Assuming α = 8° (8 degrees to the right), the system constructs a rotation matrix to rotate the tongue region counterclockwise by 8 degrees with the centroid as the origin; Through the above steps, the obtained pixel set of the corrected tongue region not only eliminates the shooting angle error of the device, but also eliminates the patient's own posture error, thus achieving standardization. S303: Traverse the pixel positions recorded in the pixel set of the tongue region for correction, extract pixel brightness values ​​one by one from the N spectral channels of the multispectral image sequence according to the pixel positions, arrange the extracted N pixel brightness values ​​in order of spectral channel wavelength from shortest to longest to form an N-dimensional feature vector, aggregate the N-dimensional feature vectors of all pixels, and construct a pixel-level multispectral feature vector set. In the initial tongue region screening step, the specific method for setting the pixel chromaticity range and lower limit threshold of brightness is as follows: Acquire a white light tongue image, calculate the three-channel pixel value distribution histogram of the white light tongue image in the hue, saturation and brightness color spaces, and generate a hue distribution map, a saturation distribution map and a brightness distribution map; Analyze the brightness distribution map, use the peak detection algorithm to identify the main peak and secondary peak in the brightness distribution map, define the brightness value corresponding to the main peak as the background brightness center, and define the brightness value corresponding to the secondary peak as the target brightness center; Based on the target brightness center and the preset brightness standard deviation multiple, the lower limit threshold of brightness is calculated. At the same time, the pixels whose brightness values ​​are in the neighborhood of the target brightness center are analyzed, and the distribution range of the pixels in the hue distribution map and saturation distribution map is determined. The pixel distribution range is then used as the pixel chromaticity range. Based on the calculated lower limit of brightness and pixel chromaticity range, each pixel in the white light tongue image is traversed and judged, and the pixels that meet the conditions are selected to generate the initial tongue region. The preset brightness standard deviation multiple is an empirical value determined based on statistical analysis of multiple tongue images, with a range of 2 to 4.

[0032] The system extracts attitude data for a specific frame (e.g., frame 15, timestamp T=0.5s) from the real-time attitude reading set. After processing, the attitude angles obtained are: roll angle 0.32°, pitch angle -0.15°, and yaw angle 25.8°. Simultaneously, it monitors the initial spatial coordinate system constructed in step S1, with a reference attitude of (0.12°, -0.08°, 25.3°). The real-time attitude data is spatially aligned with the initial spatial coordinate system, and the deviation values ​​of the two in the three rotational degrees of freedom are calculated: roll angle deviation = 0.32° - 0.12° = 0.20°, pitch angle deviation = -0.15° - (-0.08°) = -0.07°, yaw angle deviation = 25.8° - 25.3° = 0.50°. Based on these deviation values, the system constructs a perspective compensation matrix (HomographyMatrix) to perform inverse perspective projection transformation on the entire frame image to eliminate perspective distortion caused by the tilt of the acquisition device's shooting angle (such as overhead or side shots), ensuring that the geometric proportions of the tongue's projection on the two-dimensional plane are true.

[0033] Next, the white-light tongue image frame (frame 15) from the multispectral image sequence is used to filter the initial tongue region. For this, the pixel chromaticity range and lower limit threshold for brightness need to be set. This setting process first involves converting the white-light tongue image from RGB space to HSV (hue, saturation, brightness) space. Then, the histograms of pixel value distribution across the H, S, and V channels of the entire image are calculated, generating hue, saturation, and brightness distribution maps. Analysis of the brightness distribution map reveals a bimodal shape, with a primary peak at brightness value 45 and a secondary peak at brightness value 175. The system defines the brightness value of 45 corresponding to the primary peak as the background brightness center and the brightness value of 175 corresponding to the secondary peak as the target brightness center.

[0034] The calculation of the lower limit threshold for brightness is based on the target brightness center and a preset brightness standard deviation multiple. The preset brightness standard deviation multiple was determined based on a statistical experiment conducted on a database of 500 tongue image samples under different lighting conditions and individual differences. In the experiment, standard deviation multiples of 2.0, 2.5, 3.0, 3.5, and 4.0 were used to calculate the precision, recall, and F1 score of the segmentation results. Experimental data showed that the F1 score reached its peak when the multiple was 3.0, indicating the best segmentation effect. Therefore, in this embodiment, the preset brightness standard deviation multiple was determined to be 3.0. The calculated standard deviation of pixel brightness near the target brightness center 175 was 15. Therefore, the lower limit threshold for brightness = 175 - 3.0 * 15 = 130. Simultaneously, pixels with brightness values ​​within the neighborhood of the target brightness center 175 were analyzed, and the distribution intervals of these pixels in the hue distribution map and saturation distribution map were statistically analyzed. Statistical results show that the hue values ​​of these pixels are mainly distributed in the range of [0, 0.1] and [0.9, 1.0], while the saturation values ​​are distributed in the range of [0.3, 0.8]. This range is set as the pixel chromaticity range. Based on the calculated lower limit threshold of 130 and the above pixel chromaticity range, the system traverses every pixel of the white-light tongue image in this frame; if a pixel's V value is greater than 130, its H value is within [0, 0.1] or [0.9, 1.0], and its S value is within [0.3, 0.8], then the pixel is retained; otherwise, it is discarded. All the retained pixels together constitute the initial tongue region. Subsequently, the previously generated pose compensation transformation parameters (rotation matrix R) are called to perform an inverse geometric transformation on the coordinates of each pixel in the initial tongue region, resulting in the corrected tongue region pixel set.

[0035] Finally, a pixel-level multispectral feature vector set is constructed. Assume that the multispectral image sequence in this embodiment contains 5 spectral channels (N=5), ordered from shortest to longest wavelength: blue light channel (465nm), green light channel (525nm), white light channel, red light channel (630nm), and near-infrared channel (850nm). The system iterates through each pixel position recorded within the pixel set of the tongue region being rectified. For example, for pixel position (x, y), the system extracts the pixel brightness value at that position from each of the 5 spectral channel images. The extracted values ​​might be: blue light channel brightness 95, green light channel brightness 150, white light channel brightness 178, red light channel brightness 205, and near-infrared channel brightness 130. These 5 brightness values ​​are arranged sequentially to form a 5-dimensional feature vector. This vector is a multispectral response vector. The 5-dimensional feature vectors of all pixels within the tongue region are aggregated to finally construct a pixel-level multispectral feature vector set containing all pixel position indices and their corresponding multispectral response vectors.

[0036] Please see Figure 1 and Figure 5S4: Input the set of pixel-level multispectral feature vectors to the K-means clustering algorithm to classify pixel categories, calculate the area ratio and average color value of pixel categories, extract the shape values ​​of the initial tongue region, and fuse them into a comprehensive feature vector. Match the comprehensive feature vector with the discrimination rules and output the tongue image classification analysis results.

[0037] The comprehensive feature vector includes the region area percentage, average color value, and tongue shape value; The results of tongue image classification analysis specifically refer to the tongue image category identifier and confidence score; The specific steps for obtaining the results of tongue image classification analysis are as follows: S401: For a set of pixel-level multispectral feature vectors, iteratively divide the data according to the Euclidean distance between the vectors until the cluster centers are stable, forming several pixel categories. Calculate the ratio of the number of pixels in each pixel category to the total number of pixels in the initial tongue region, and calculate the area ratio. At the same time, perform an arithmetic mean operation on the color values ​​of all pixels in each pixel category to obtain the average color value. Finally, combine the area ratio of all pixel categories with the average color value to generate a pixel category attribute set. S402: Based on the contour coordinates of the initial tongue region, calculate the ratio of its perimeter to its area as the shape value, call the pixel category attribute set, and serialize and concatenate the shape value with the area ratio and average color value in the pixel category attribute set to establish a comprehensive feature vector. S403: The area ratio, average color value, and shape value of the comprehensive feature vector are compared one by one with the preset value range thresholds for the differentiated tongue image categories in the discrimination rules. When all the values ​​of the comprehensive feature vector fall within the preset threshold range of a tongue image category, the matching is determined to be successful, and the tongue image classification analysis results are obtained. In the K-means clustering algorithm's steps for classifying pixel categories, the specific criterion for determining the stability of cluster centers is as follows: K multispectral response vectors are randomly selected from the set of pixel-level multispectral feature vectors as initial cluster centers; For each multispectral response vector in the pixel-level multispectral feature vector set, calculate the Euclidean distance to multiple cluster centers, and assign the multispectral response vector to the pixel category corresponding to the nearest cluster center; Recalculate the mean of all multispectral response vectors within each pixel category and update the mean to the current cluster center; Repeat the assignment and update steps, and determine whether the clustering process has converged using the following formula:

[0038] Where ΔC represents the average change of all cluster centers between two consecutive iterations, K represents the preset number of pixel categories, and k is the pixel category index from 1 to K. This represents the cluster center vector of the k-th pixel category at the t-th iteration. This represents the cluster center vector of the pixel category in the previous iteration, i.e., the (t-1)th iteration. The Euclidean norm of a vector. This represents the preset cluster center convergence threshold; The preset number of pixel categories K is determined based on the common classification numbers of tongue coating and coating color in the theory of tongue diagnosis, and the value ranges from 3 to 7. When ΔC is less than or equal to When the clustering process has converged, several pixel categories are generated. The specific steps for calculating the confidence score in the tongue image classification analysis results are as follows: Once the comprehensive feature vector is successfully matched with the discrimination rule of a certain tongue image category, the preset numerical range threshold of the tongue image category is obtained, including the upper and lower limits of each feature. The area percentage, average color, and tongue shape values ​​from the comprehensive feature vector are retrieved, and the confidence score is calculated using the following formula:

[0039] Among them, S conf The confidence score represents the total number of dimensions in the composite feature vector, M represents the total number of dimensions, j is the feature index from 1 to M, and w j The preset weight coefficients representing the j-th feature, and the sum of all weight coefficients is 1, v j U represents the actual value of the j-th dimension in the comprehensive feature vector. j L represents the upper limit of the preset numerical range of the j-th feature that matches the tongue image category. j The lower limit of the preset numerical range of the representative feature, δ represents the smoothing factor set to avoid a denominator of zero; The preset weight coefficient w of the j-th feature j It is determined by the contribution of differential features to the tongue image classification results, through expert scoring or machine learning model training. The calculated confidence score is combined with the tongue image category identifier to generate the tongue image classification analysis result.

[0040] The system inputs a set of pixel-level multispectral feature vectors into the K-means clustering algorithm. The preset number of pixel categories K is determined based on the number of common combinations of tongue coating texture (e.g., thick, thin, moist, dry, greasy) and coating color (e.g., white, yellow, gray-black) in traditional Chinese medicine tongue diagnosis theory. To cover common cases, a pre-clustering experiment is conducted on 1000 tongue image samples already annotated by experts. In the experiment, K values ​​were set to 3, 4, 5, 6, and 7, and the silhouette coefficient and Calinski-Harabasz index were used as evaluation indicators. The experimental results show that when K=5, the combined score of the two indicators is the highest, indicating that the clustering effect at this time best matches the classification of traditional Chinese medicine theory. Therefore, K=5 is set in this embodiment. The algorithm first randomly selects 5 multispectral response vectors from the feature vector set as initial cluster centers. Then, for each vector in the set, its Euclidean distance to these 5 cluster centers is calculated, and it is assigned to the pixel category corresponding to the nearest cluster center. After all vectors have been assigned, the mean of all vectors within each pixel category is recalculated, and this mean is used to update the new cluster centers. This assignment and update step is repeated.

[0041] After each iteration, the convergence of the clustering process is determined by the following formula:

[0042] This formula is used to calculate the average change of all cluster centers between two consecutive iterations.

[0043] Where ΔC represents the average change; K is the preset number of pixel categories, which is 5 in this example; and k is the pixel category index, ranging from 1 to 5. This represents the cluster center vector of the k-th pixel category at the current t-th iteration; it is a 5-dimensional vector. This represents the cluster center vector of this category in the previous iteration (t-1th iteration); This represents the calculation of the Euclidean norm (i.e., Euclidean distance) between two vectors. This is the preset cluster center convergence threshold. This threshold is set based on an experiment balancing convergence speed and accuracy, by monitoring different... The number of iterations and the stability of the final clustering results at values ​​(e.g., 0.1, 0.01, 0.001) are used to determine the stability of the final clustering results. It is a value that can converge stably within 100 iterations with minimal fluctuations in the result.

[0044] Example: Suppose that the changes of the 5 cluster centers in the 10th and 11th iterations are as follows (for simplicity, a two-dimensional vector is used as an example): …(Calculate ΔC for the other 3 centers:) ; Assume the distances to the other three centers are 0.15, 0.18, and 0.12, respectively.

[0045] .

[0046] Because 0.1582 > 0.01, the clustering process is determined not to have converged, and the next iteration continues. When the calculated ΔC value in a certain iteration is 0.008, since 0.008 ≤ 0.01, the clustering process is determined to have converged, forming 5 pixel categories. Subsequently, the proportion of pixels in each category to the total number of pixels in the initial tongue region is calculated, and the area proportion is determined. At the same time, the arithmetic mean of the color values ​​(in CIELAB space) of all pixels in each category is performed to obtain the average color value.

[0047] Next, based on the contour coordinates of the initial tongue region, the ratio of its perimeter (e.g., 800 pixels) to its area (e.g., 30,000 square pixels) is calculated, resulting in a shape value of 800 / 30,000 ≈ 0.0267. This shape value is then serialized and concatenated with the area proportions and average color values ​​of the five pixel categories to establish a comprehensive feature vector, for example: [shape value, area of ​​class 1, class 1L, class 1a, class 1b, area of ​​class 2, ..., class 5b].

[0048] Finally, the comprehensive feature vector is compared with the discrimination rules. The discrimination rules are preset in the database. For example, the threshold values ​​for the numerical range of 'pale tongue with thin white coating' are: shape value [0.02, 0.03], area ratio of the main coating color category (assuming it is category 1) [0.7, 0.9], L value of the main coating color, a value [-5, 5], b value, etc. When all the values ​​of the comprehensive feature vector fall within the preset threshold range of a certain tongue image category, the match is determined to be successful.

[0049] After a successful match, use the formula:

[0050] Calculate the confidence score. This formula is used to quantify the feature value v. j Within its preset effective range The closer the feature value is to the midpoint of the interval, the higher its contribution to the overall confidence score, thus making the score results more discriminative. Example: Assume the weight of the shape value w1 = 0.2, and its range... The actual value v1 = 0.0267; the area proportion weight of Class 1 w2 = 0.3, and its range The actual value is v2=0.75.

[0051] Calculate the score for the first feature: Center point = (0.03 + 0.02) / 2 = 0.025; Half distance = (0.03 - 0.02) / 2 = 0.005; Scoring items ; Calculate the score for the second feature: Center point = (0.9 + 0.7) / 2 = 0.8; Half distance = (0.9 - 0.7) / 2 = 0.1; Scoring items ; The total confidence score S is obtained by summing the scores of all M dimensions. conf For example, the final calculated score is 0.89. This result indicates that the confidence level for identifying the current tongue image as "pale tongue with thin white coating" is 89%. Finally, the system outputs the tongue image category identifier "pale tongue with thin white coating" and the confidence score of 0.89, together generating the tongue image classification analysis result.

[0052] A system based on tongue diagnosis image recognition, the system being used to execute the above-described method based on tongue diagnosis image recognition, the system comprising: The benchmark construction module is used to map the red, green, and blue channel values ​​of the standard color chart image generated by illuminating the standard color chart with a monochromatic light source and a white light source to the pre-stored standard color values, generate a color correction matrix, and construct an initial spatial coordinate system based on the acquired initial posture readings, and then transfer the color correction matrix and the initial spatial coordinate system to the tongue image acquisition and color calibration module. The tongue image acquisition and color correction module is used to capture tongue images and collect real-time posture readings. It calls the color correction matrix to transform the color values ​​of the tongue images, generates a multispectral image sequence and a set of real-time posture readings, and then passes the multispectral image sequence, the set of real-time posture readings, and the initial spatial coordinate system to the posture correction and feature construction module. The attitude correction and feature construction module is used to generate attitude compensation transformation parameters based on the deviation between the real-time attitude reading set and the initial spatial coordinate system, select the initial tongue region from the white light tongue image of the multispectral image sequence, and call the attitude compensation transformation parameters to correct the initial tongue region. It combines pixel brightness values ​​to construct a set of pixel-level multispectral feature vectors and passes them to the comprehensive discrimination analysis module. The comprehensive discriminant analysis module is used to input a set of pixel-level multispectral feature vectors to the K-means clustering algorithm to classify pixel categories, calculate the area ratio and average color value of pixel categories, extract the shape values ​​of the initial tongue region, and fuse them into a comprehensive feature vector. The comprehensive feature vector is then matched with the discrimination rules to output the tongue image classification analysis results.

[0053] The above embodiments illustrate preferred embodiments of the present invention. Any equivalent adjustments to the technical solution based on software engineering methods are within the scope of protection, including but not limited to: implementing algorithm logic using different programming languages, refactoring functional modules into services, adjusting data interaction protocols, and optimizing resource scheduling strategies. Any implementation scheme derived from reasonable modifications to the data processing flow, service call chain, or system architecture layer without departing from the core technology of the present invention should be considered within the scope of protection defined by the claims of the present invention.

Claims

1. A method based on tongue diagnosis image recognition, characterized in that, Includes the following steps: S1: The mapping relationship between the red, green, and blue channel values ​​of the standard color chart image generated by illuminating the standard color chart with a monochromatic light source and a white light source and the pre-stored standard color values ​​is used to generate a color correction matrix, and an initial spatial coordinate system is constructed based on the collected initial attitude readings. S2: Capture tongue images and collect real-time posture readings, call the color correction matrix to transform the color values ​​of the tongue images, and generate a multispectral image sequence and a set of real-time posture readings; S3: Generate a perspective compensation matrix and attitude compensation transformation parameters based on the deviation between the real-time attitude reading set and the initial spatial coordinate system. Perform global perspective correction on the multispectral image sequence. Then, select the initial tongue region from the white light tongue image. At the same time, call the attitude compensation transformation parameters to correct the initial tongue region. Calculate the second-order central moment of the initial tongue region. Derive the principal axis tilt angle of the tongue. Based on this, construct a rotation matrix to perform principal axis alignment transformation on the initial tongue region. Combine pixel brightness values ​​to construct a pixel-level multispectral feature vector set. S4: Input the set of pixel-level multispectral feature vectors into the K-means clustering algorithm to classify pixel categories, calculate the area ratio and average color value of the pixel categories, extract the shape values ​​of the initial tongue region, and fuse them into a comprehensive feature vector. Match the comprehensive feature vector with the discrimination rules and output the tongue image classification analysis results.

2. The method based on tongue diagnosis image recognition according to claim 1, characterized in that, The color correction matrix includes red channel correction coefficients, green channel correction coefficients, and blue channel correction coefficients. The initial spatial coordinate system specifically consists of three-axis rotation components and three-axis translation components. The multispectral image sequence includes white light tongue image frames and monochromatic light tongue image frames. The real-time attitude reading set specifically refers to real-time gyroscope readings and real-time accelerometer readings. The attitude compensation transformation parameters include rotation matrices and translation vectors. The pixel-level multispectral feature vector set includes pixel position indices and multispectral response vectors. The comprehensive feature vector includes region area percentage values, average color values, and tongue shape values. The tongue image classification analysis results specifically refer to tongue image category identifiers and confidence scores.

3. The method based on tongue diagnosis image recognition according to claim 2, characterized in that, The specific steps for obtaining the initial spatial coordinate system are as follows: S101: Obtain a standard color swatch and illuminate it with a monochromatic light source and a white light source respectively. Simultaneously acquire standard color swatch images, analyze the standard color swatch images, extract the pixel values ​​of the red, green and blue channels within the image frame, and combine the extracted pixel values ​​into a multidimensional array to generate a standard color swatch image pixel value matrix. S102: Call the standard color swatch image pixel value matrix and obtain the pre-stored standard color values. Construct a linear transformation equation between the multiple channel values ​​in the standard color swatch image pixel value matrix and the color components corresponding to the standard color values. Solve the linear transformation equation to obtain the transformation coefficients. Organize the transformation coefficients into a third-order matrix to generate a color correction matrix. S103: Collect initial attitude readings of roll angle, pitch angle and yaw angle data from the attitude sensor. Based on the initial attitude readings, define the roll angle, pitch angle and yaw angle data as the first, second and third rotation axes of the spatial coordinate system, respectively, and use them as a reference to construct the initial spatial coordinate system.

4. The method based on tongue diagnosis image recognition according to claim 3, characterized in that, The specific steps for obtaining the multispectral image sequence and the real-time attitude reading set are as follows: S201: The image sensor captures a single frame of tongue image, and the attitude sensor collects real-time attitude readings simultaneously. The color information of multiple pixels in the tongue image is deconstructed into red, green and blue three-channel intensity values ​​to generate an original tongue RGB image. The original data from the gyroscope and accelerometer in the real-time attitude readings are integrated to establish an initial attitude parameter set. S202: Obtain a preset color correction matrix, construct a pixel color vector based on the red, green and blue three-channel intensity values ​​of each pixel position in the original tongue RGB image, call the color correction matrix and the pixel color vector to perform matrix multiplication, replace the red, green and blue three-channel intensity values ​​with the transformation vector generated by the operation, and obtain the corrected spectral data frame; S203: For the multiple calibration spectral data frames acquired within a continuous sampling period and the initial attitude parameter set, stack the multiple calibration spectral data frames along the time axis to construct a multispectral image sequence, and retrieve the acquisition timestamp corresponding to the initial attitude parameter set and match it with the image frame index of the multispectral image sequence to generate the multispectral image sequence and the real-time attitude reading set.

5. The method based on tongue diagnosis image recognition according to claim 4, characterized in that, The specific steps for obtaining the pixel-level multispectral feature vector set are as follows: S301: Calculate the homography matrix based on the deviation between the real-time attitude reading set and the initial spatial coordinate system, eliminate perspective distortion caused by the tilt of the shooting angle, calculate the zero-order moment M00 and the first-order moments M10 and M01 of the initial tongue region to determine the centroid coordinates, and calculate the deflection angle α of the tongue principal axis relative to the vertical direction by combining the second-order central moments μ20, μ11 and μ02, and generate attitude compensation transformation parameters including rotation and translation components. S302 calls the white light tongue image of the multispectral image sequence, sets the pixel chromaticity range and the lower limit threshold of brightness, traverses all pixels of the white light tongue image, removes background pixels that do not conform to the pixel chromaticity range and the lower limit threshold of brightness, filters to obtain the initial tongue region, calls the posture compensation transformation parameters to perform inverse geometric transformation on the pixel coordinates of the initial tongue region, and obtains the corrected tongue region pixel set. S303: Traverse the pixel positions recorded in the pixel set of the corrected tongue region, extract pixel brightness values ​​one by one from the N spectral channel images of the multispectral image sequence according to the pixel positions, arrange the extracted N pixel brightness values ​​in order of spectral channel wavelength from shortest to longest to form an N-dimensional feature vector, aggregate the N-dimensional feature vectors of all the pixels, and construct a pixel-level multispectral feature vector set.

6. The method based on tongue diagnosis image recognition according to claim 5, characterized in that, The specific steps for obtaining the tongue image classification analysis results are as follows: S401: For the set of pixel-level multispectral feature vectors, iteratively divide them according to the Euclidean distance between the vectors until the cluster centers are stable, forming several pixel categories. Calculate the ratio of the number of pixels in each of the pixel categories to the total number of pixels in the initial tongue region, and calculate the area ratio. At the same time, perform an arithmetic mean operation on the color values ​​of all pixels in each of the pixel categories to obtain the average color value. Finally, combine the area ratio of all pixel categories with the average color value to generate a pixel category attribute set. S402: Based on the contour coordinates of the initial tongue region, calculate the ratio of its perimeter to its area as a shape value, call the pixel category attribute set, and serialize and concatenate the shape value with the area ratio and the average color value in the pixel category attribute set to establish a comprehensive feature vector. S403: The area ratio, average color value, and shape value of the comprehensive feature vector are compared one by one with the preset value range threshold for the differentiated tongue image category in the discrimination rule. When all the values ​​of the comprehensive feature vector fall within the preset threshold range of a tongue image category, the matching is determined to be successful, and the tongue image classification analysis result is obtained.

7. The method based on tongue diagnosis image recognition according to claim 5, characterized in that, In the initial tongue region screening step, the method for setting the pixel chromaticity range and the lower limit threshold of brightness is as follows: Acquire the white light tongue image, calculate the three-channel pixel value distribution histogram of the white light tongue image in the hue, saturation and brightness color space, and generate the hue distribution map, saturation distribution map and brightness distribution map; Analyze the brightness distribution map, use a peak detection algorithm to identify the main peak and secondary peak in the brightness distribution map, define the brightness value corresponding to the main peak as the background brightness center, and define the brightness value corresponding to the secondary peak as the target brightness center; The lower limit threshold of brightness is calculated based on the target brightness center and the preset brightness standard deviation multiple. At the same time, the pixels whose brightness values ​​are in the neighborhood of the target brightness center are analyzed, and the distribution range of the pixels in the hue distribution map and the saturation distribution map is determined. The pixel distribution range is then used as the pixel chromaticity range. Based on the calculated lower limit of brightness and the pixel chromaticity range, each pixel in the white light tongue image is traversed and judged, and the pixels that meet the conditions are selected to generate the initial tongue region. The preset brightness standard deviation multiple is an empirical value determined based on statistical analysis of multiple tongue images, and the value ranges from 2 to 4.

8. The method based on tongue diagnosis image recognition according to claim 6, characterized in that, In the step of classifying pixel categories using the K-means clustering algorithm, the specific criterion for determining the stability of cluster centers is as follows: K multispectral response vectors are randomly selected from the set of pixel-level multispectral feature vectors as initial cluster centers; For each of the multispectral response vectors in the pixel-level multispectral feature vector set, calculate the Euclidean distance with the multiple cluster centers, and assign the multispectral response vector to the pixel category corresponding to the nearest cluster center; Recalculate the mean of all multispectral response vectors within each pixel category, and update the mean to the current cluster center; Repeat the assignment and update steps, and determine whether the clustering process has converged using the following formula: ， Where ΔC represents the average change of all cluster centers between two consecutive iterations, K represents the preset number of pixel categories, and k is the pixel category index from 1 to K. This represents the cluster center vector of the k-th pixel category at the t-th iteration. This represents the cluster center vector of the pixel category in the previous iteration, i.e., the (t-1)th iteration. The Euclidean norm of a vector. This represents the preset cluster center convergence threshold; The preset number of pixel categories K is determined based on the common classification numbers of tongue coating and coating color in the theory of tongue diagnosis, and the value ranges from 3 to 7. When the ΔC is less than or equal to the When the clustering process converges, several pixel categories are generated.

9. The method based on tongue diagnosis image recognition according to claim 6, characterized in that, The specific steps for calculating the confidence score in the tongue image classification analysis results are as follows: Once the comprehensive feature vector is successfully matched with the discrimination rule of a certain tongue image category, the preset numerical range threshold of the tongue image category is obtained, including the upper and lower limits of each feature. The confidence score is calculated by retrieving the area percentage, average color, and tongue shape values ​​from the comprehensive feature vector, and using the following formula: ， Among them, S conf The confidence score represents the total number of dimensions of the comprehensive feature vector, and j is the feature index from 1 to M. j The preset weight coefficients representing the j-th feature, and the sum of all weight coefficients is 1, v j U represents the actual value of the j-th dimension in the comprehensive feature vector. j L represents the upper limit of the preset numerical range of the j-th feature that matches the tongue image category. j The lower limit of the preset numerical range of the representative feature, δ represents the smoothing factor set to avoid a denominator of zero; The preset weighting coefficient w of the j-th feature j It is determined by the contribution of differential features to the tongue image classification results, through expert scoring or machine learning model training. The calculated confidence score is combined with the tongue image category identifier of the tongue image category to generate the tongue image classification analysis result.

10. A system based on tongue diagnosis image recognition, characterized in that, The system is used to implement the tongue diagnosis image recognition method according to any one of claims 1-9, the system comprising: The benchmark construction module is used to generate a color correction matrix by mapping the red, green, and blue channel values ​​of the standard color chart image generated by illuminating the standard color chart with a monochromatic light source and a white light source to the pre-stored standard color values. Based on the acquired initial posture readings, it constructs an initial spatial coordinate system and transmits the color correction matrix and the initial spatial coordinate system to the tongue image acquisition and color calibration module. The tongue image acquisition and color correction module is used to capture tongue images and collect real-time posture readings. It calls the color correction matrix to transform the color values ​​of the tongue image, generates a multispectral image sequence and a set of real-time posture readings, and transmits the multispectral image sequence, the set of real-time posture readings, and the initial spatial coordinate system to the posture correction and feature construction module. The attitude correction and feature construction module is used to generate attitude compensation transformation parameters based on the deviation between the real-time attitude reading set and the initial spatial coordinate system, select the initial tongue region from the white light tongue image of the multispectral image sequence, and simultaneously call the attitude compensation transformation parameters to correct the initial tongue region. The module combines pixel brightness values ​​to construct a pixel-level multispectral feature vector set and transmits it to the comprehensive discrimination analysis module. The comprehensive discriminant analysis module is used to input the set of pixel-level multispectral feature vectors into the K-means clustering algorithm to classify pixel categories, calculate the area ratio and average color value of the pixel categories, extract the shape values ​​of the initial tongue region, and fuse them into a comprehensive feature vector. The module then matches the comprehensive feature vector with the discrimination rules and outputs the tongue image classification analysis results.

Images