A tongue diagnosis image segmentation method based on computer vision

Abstract

The application discloses a tongue image segmentation method based on computer vision. In the application, a tongue color criticality index feedback term is introduced, so that the segmentation contour can judge the tongue color type in real time according to the pixels already segmented in the current region in the iteration process. If a criticality feature such as purple tongue or cyanotic tongue is found, the feedback term automatically drives the contour to contract inward, and actively excludes the dark interference structure which does not belong to the main region of the tongue. On the contrary, if the current region is too small and presents a light white tongue feature, the feedback term allows the contour to expand moderately to ensure the integrity of the tongue surface information. The segmentation and color recognition feedback mechanism makes the final output tongue region optimal in both geometric shape and traditional Chinese medicine semantics, so that the contour edge is more consistent with the real tongue boundary, and the segmentation result naturally meets the definition requirements of the main region of the tongue in traditional Chinese medicine tongue diagnosis.

Description

Technical Field

[0001] This invention belongs to the field of image recognition technology, specifically a method for segmenting tongue diagnosis images based on computer vision. Background Technology

[0002] Tongue diagnosis is an important component of traditional Chinese medicine's visual diagnosis. The color, coating, and sublingual veins of the tongue directly reflect the state of Qi and blood, as well as the pathological changes in the internal organs. With the development of computer vision technology, automatic tongue image segmentation has become a crucial prerequisite for the digitization of TCM tongue diagnosis. Currently, commonly used tongue segmentation methods include threshold-based segmentation, edge detection-based segmentation, graph theory-based segmentation, and semantic segmentation based on deep learning. These methods can achieve good segmentation results under ideal lighting and standard tongue morphology.

[0003] However, current technologies for tongue image segmentation typically rely solely on statistical features of grayscale, edges, or color, lacking a dynamic feedback mechanism with TCM tongue color diagnostic indicators. This leads to the segmentation contour being easily skewed by interfering structures such as dark ecchymosis at the tongue edges, visible sublingual veins, or dark red areas on the inner wall of the mouth, resulting in the inaccurate inclusion of non-tongue tissues in the segmentation results. Furthermore, traditional methods often rely on a single geometric change threshold for segmentation stopping, leading to poor repeatability of segmentation results from multiple acquisitions of the same patient. Additionally, the lack of mutual verification and closed-loop adjustment capabilities between processing steps means that deviations at any step can accumulate and propagate downstream, failing to meet the requirements of quantitative stability and semantic consistency in TCM clinical tongue diagnosis. Summary of the Invention

[0004] The purpose of this invention is to provide a computer vision-based method for tongue diagnosis image segmentation in order to solve the problems mentioned above.

[0005] The technical solution adopted in this invention is as follows: A tongue diagnosis image segmentation method based on computer vision, comprising the following steps: S1: Acquire tongue images and extract ambient lighting parameters. A D65 standard light source with ring illumination is used to acquire the original tongue image through a CMOS sensor, and the global average brightness and color temperature deviation of the image are recorded.

[0006] S2: Correct the uneven brightness areas of the image based on the illumination parameters of S1, convert the RGB image to the Lab color space, and extract the L channel for subsequent tongue-background separation.

[0007] S3: Obtain a binarized mask on the a channel of the Lab space using the Otsu's method, remove small noise regions in the oral cavity using morphological opening operation, and output the initial contour of the tongue.

[0008] S4: Using the initial contour from S3 as the starting boundary of the active contour model, an energy function is defined that includes a contour length term, a region grayscale consistency term, and a tongue color crisis index feedback term. The tongue color crisis index calculates the weight scores for pale white, red, dark purple, and bluish-purple tongues in real time based on the Lab values ​​of pixels within the current segmentation region. These scores, in turn, drive contour expansion or contraction. Iteration stops when the contour change between two adjacent iterations is less than one pixel and the tongue color index fluctuation is less than 5%, outputting a finely segmented tongue region. This mechanism of mutual feedback between segmentation and tongue color quantization can eliminate the boundary drift problem caused by the thickness of the tongue coating.

[0009] S5: Within the tongue region obtained in S4, locate the course of the sublingual vein using a grayscale projection curve. If the vein region is over-segmented or omitted, trigger local contour adjustment and revert to the last valid iteration of S4 for re-execution.

[0010] S6: Perform hole filling and edge smoothing on the validated tongue mask. The seed filling algorithm removes tiny holes inside the tongue, and cubic spline interpolation smooths the contour edges to obtain the final tongue segmentation result.

[0011] S7: Pass the segmentation results of S6 to the tongue color quantitative analysis module, and output the mean values ​​of L, a, and b of the tongue body region to calculate the tongue color crisis index, which serves as the input feature for subsequent syndrome differentiation.

[0012] In a preferred embodiment, in step S1, the tongue image acquisition uses a ring-shaped lighting device with a standard D65 light source, the color temperature of which is fixed at 6500K, and a polarizing filter is installed in front of the light source to eliminate reflections on the tongue surface. During shooting, the distance between the tongue and the lens is kept at 15 cm, and three frames are continuously captured after autofocus. The frame with the highest clarity is taken as the original tongue image.

[0013] In a preferred embodiment, in step S2, the presence of excessively dark or bright areas in the image is first determined based on the global average brightness. If the average brightness is below 50 or above 200, automatic exposure compensation is triggered. Then, the RGB image is converted to the Lab color space, and only the L channel is extracted for subsequent segmentation. To address the uneven illumination problem, a brightness correction method based on local statistics is used. The L channel image is divided into a 16x16 grid, and the difference between the average brightness of each grid and the global average is calculated. The correction coefficient for each pixel is obtained through bilinear interpolation. Finally, the corrected L channel image is output to S3.

[0014] In a preferred embodiment, in step S3, the corrected L-channel image output from S2 is used as input. First, a global threshold is calculated on the a-channel. The pixel values ​​of the a-channel are evenly divided into 256 levels from the minimum to the maximum value. The proportion of pixels in each level to the total number of pixels is calculated, and then the gray level that maximizes the inter-class variance is found as the segmentation threshold. A binary mask is generated based on this threshold. Pixels greater than the threshold are marked as 1, representing candidate tongue regions, and pixels less than the threshold are marked as 0, representing background. Next, a morphological opening operation is performed on the binary mask using a circular structuring element with a radius of 5 pixels, first eroding and then dilating, to remove isolated small noise regions generated by the oral cavity wall and teeth. Finally, the largest connected component is extracted as the initial contour of the tongue, and the boundary coordinates of this contour are passed to S4 as the starting contour.

[0015] In a preferred embodiment, in step S4, a level set framework is selected to implicitly represent the contour as the zero level set of the level set function.

[0016] The energy function includes a basic term and a tongue color crisis index feedback term. The basic term consists of a contour length term and a region grayscale consistency term. The contour length term penalizes overly convoluted edges to ensure smooth segmentation results. The region grayscale consistency term adopts the idea of ​​the Chan-Vese model, assuming uniform grayscale inside the tongue and uniform grayscale in the background, and is characterized by the mean grayscale values ​​of the L channels inside and outside the level set, respectively.

[0017] The tongue color crisis index feedback term calculates the tongue color index in real time based on the Lab values ​​of all pixels within the current segmentation region in each iteration. The index level then drives the contour to contract or expand. Specifically, if the calculated tongue color index for the current region is high, indicating a potential dark purple or bluish-purple tongue crisis, the feedback term generates a negative driving force, causing the contour to contract inward to prevent dark ecchymosis or sublingual veins from being mistakenly included in the main tongue region. If the tongue color index is low, corresponding to a pale white or bright red tongue, the feedback term allows the contour to expand moderately outward to cover the entire tongue surface.

[0018] In a preferred embodiment, in step S4, before each iteration begins, the membership degree and weighted total score of the four tongue colors are recalculated based on the pixel set inside the current contour, and then the feedback velocity function in the level set evolution equation is updated.

[0019] The evolution equation itself is composed of contour curvature, internal and external grayscale differences, and tongue color feedback speed. The contour curvature term keeps the edges smooth, while the grayscale difference term pushes the contour to move towards the actual tongue edge; The tongue color feedback speed acts as an additional moderating force, its magnitude and sign determined by the real-time value of the tongue color crisis index. A positive feedback speed causes the contour to expand outwards, while a negative speed causes it to contract inwards. This makes the segmentation process no longer purely geometrically driven, but incorporates semantic knowledge from traditional Chinese medicine tongue diagnosis. For example, if the initial contour inadvertently includes part of the dark red oral cavity wall, the α-channel values ​​in these areas are high, leading to an increase in the membership degree of the current region's dark purple tongue and a higher tongue color index. The feedback term immediately drives the contour to contract inwards, thus excluding the dark red area. Conversely, if the current region is too small and the tongue color exhibits pale characteristics, it indicates that the pale parts of the tongue's edge may have been missed, and the feedback term encourages the contour to expand outwards.

[0020] The evolution equation for the level set with tongue color feedback is: Parameter definition: It is a level set function, whose zero level set represents the current contour.

[0021] This represents the iteration time step.

[0022] It is a Dirac function, which is non-zero only near the contour.

[0023] The weight of the contour length term controls the edge smoothness.

[0024] The curvature of the contour.

[0025] and These are the grayscale consistency weights for the inside and outside of the contour, respectively.

[0026] The image pixel grayscale values ​​are represented using the L channel in Lab color space.

[0027] and These are the average gray values ​​of the L channel inside and outside the current contour, respectively.

[0028] This is the adjustment coefficient for the intensity of tongue color feedback.

[0029] The feedback speed function for the tongue color crisis index.

[0030] The specific expression for the feedback velocity function is: Parameter definition: The preset weights for pale white tongue, reddish-brown tongue, dark purple tongue, and bluish-purple tongue are 5, 8, 15, and 25, respectively.

[0031] The membership degree of the four tongue colors in the current segmentation region is calculated by the fuzzy membership function of the pixel Lab value and the standard threshold range, and the sum of the four membership degrees is 1.

[0032] It is a sign function; when it takes a negative value, it drives contour contraction, and when it takes a positive value, it drives expansion. Its absolute value is determined by the weighted mean.

[0033] In a preferred embodiment, in step S4, the iteration continues until both conditions are met simultaneously.

[0034] The first condition is that the maximum change in the contour position between two adjacent iterations is less than one pixel, which indicates that the geometry has converged.

[0035] The second condition is that the relative volatility of the Tongue Color Criticality Index is less than 5%. Volatility is defined as the absolute value of the difference between the mean of the Tongue Color Index in the current iteration and that in the previous iteration, divided by the mean of the previous iteration.

[0036] The second condition is added because simply looking at geometric changes may not be enough—sometimes the outline moves very little, but the internal pixel composition is still changing slowly, such as the uneven thickness of the tongue coating causing oscillations in the tongue color statistics. Only when both geometry and tongue color are stable can the segmentation be considered truly complete.

[0037] After stopping the iteration, the current zero-level set is output as the finely segmented tongue region. This region is passed to S5 for sublingual vein localization and is also output to the subsequent tongue color quantification module, realizing a closed-loop feedback between segmentation and color recognition.

[0038] The formula for calculating the volatility of the tongue color index in the iteration stopping condition is: Parameter definition: For the first The relative rate of change of the tongue color index in each iteration.

[0039] For the first The mean tongue color crisis index of all pixels in the segmented region at the end of the next iteration is calculated as follows: in The total number of pixels in the region. For pixels This is an indicator function for a pale tongue; the same applies to the others.

[0040] in The total number of pixels in the region. For pixels This is an indicator function for a pale tongue; the same applies to the others.

[0041] It should be a very small positive number to prevent the denominator from being zero. The iteration terminates when the maximum change in the contour position is less than one pixel.

[0042] In a preferred embodiment, in step S5, the sublingual vein is located within the mask using a grayscale projection curve. First, the cumulative grayscale value of each column of pixels within the mask is calculated to obtain a vertical projection curve. The column coordinates corresponding to the lowest point of the curve are found as the midline of the tongue. Then, the cumulative grayscale value of each row of pixels within a 10-pixel range to the left and right of the midline is calculated to obtain a horizontal projection curve. The two troughs on this curve correspond to the start and end points of the sublingual vein, respectively. Based on these two points, a region of interest (ROI) for the vein is defined, and the Lab values ​​of the pixels within the region are extracted. If the area of ​​the vein region is less than 5% of the total area of ​​the tongue, or the average a value of the vein region is greater than 20 and the b value is less than 5, it is determined to be a valid vein region. Otherwise, the vein extraction is considered incomplete, triggering a local contour adjustment signal, returning to the last valid iteration of S4, and re-execution.

[0043] In a preferred embodiment, in step S6, a seed filling algorithm is used to fill all holes inside the mask. Specifically, a seed pixel located inside the mask is selected and diffused to its four neighbors until it encounters the mask boundary. All pixels that have diffused through this area are marked as filling regions. For burrs on the mask boundary caused by uneven tongue coating thickness, a cubic spline interpolation method is used for edge smoothing. Thirty control points are selected at equal intervals from the mask contour, and cubic spline fitting is performed on the horizontal and vertical coordinates of these control points to obtain smooth closed curves. Then, the pixels inside the curve are reassigned to 1, and the pixels outside are reassigned to 0, generating the final tongue segmentation mask.

[0044] In a preferred embodiment, in step S7, the L, a, and b components of all pixels within the mask region are extracted. The mean of the L channel, the mean of the a channel, the mean of the b channel, and the standard deviation of the three channels are calculated respectively. These statistics, along with the total number of pixels in the mask region, are packaged into a tongue feature vector and passed to the tongue color quantification analysis module. This module determines the tongue color type and calculates the tongue color crisis index based on the L, a, and b mean values, while retaining the segmentation mask itself for subsequent tongue coating analysis and sublingual vein analysis as a location reference.

[0045] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are: 1. This invention introduces a tongue color crisis index feedback term, enabling the segmented contour to determine the tongue color type in real time based on the pixels already segmented within the current region during the iteration process. If critical features such as a dark purple or bluish-purple tongue are detected, the feedback term automatically drives the contour to shrink inward, actively eliminating dark interference structures that do not belong to the main tongue region. Conversely, if the current region is too small and exhibits pale tongue characteristics, the feedback term allows the contour to expand moderately to ensure the integrity of the tongue surface information. This mechanism of mutual feedback between segmentation and color identification ensures that the final output tongue region achieves optimal performance in both geometric shape and TCM semantics. Not only does the contour edge more closely match the actual tongue boundary, but the segmentation result also naturally conforms to the definition requirements of the main tongue region in TCM tongue diagnosis.

[0046] 2. In this invention, the stopping condition requires both a contour position change of less than one pixel and a tongue color index fluctuation of less than 5%. This ensures that the segmentation results will not fluctuate due to different iteration counts, allowing for highly repeatable segmentation of tongue images from the same patient acquired at different time points. Simultaneously, the output precisely segmented region is used for sublingual vein localization. When vein extraction is incomplete, local contour adjustment is triggered, and the process is reverted for re-execution. This closed-loop feedback design between the preceding and following steps prevents erroneous segmentation results from propagating to downstream modules. The final output L, a, and b statistics accurately reflect the color characteristics of the tongue without being affected by uneven tongue coating thickness or boundary burrs. This allows the method to be stably deployed in TCM smart wearable devices or TCM hospital monitoring systems, providing reliable segmentation results for subsequent tongue color criticality index calculation, tongue coating texture analysis, and sublingual vein grading. Attached Figure Description

[0047] Figure 1 This is a schematic diagram illustrating the process principle of the present invention; Figure 2 This is a diagram of the active contour evolution interface for tongue color feedback in this invention. Detailed Implementation

[0048] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0049] Example: Reference Figure 1-2 A computer vision-based tongue diagnosis image segmentation method includes the following steps: S1: Acquire tongue images and extract ambient lighting parameters. A D65 standard light source with ring illumination is used to acquire the original tongue image through a CMOS sensor, and the global average brightness and color temperature deviation of the image are recorded.

[0050] S2: Correct the uneven brightness areas of the image based on the illumination parameters of S1, convert the RGB image to the Lab color space, and extract the L channel for subsequent tongue-background separation.

[0051] S3: Obtain a binarized mask on the a channel of the Lab space using the Otsu's method, remove small noise regions in the oral cavity using morphological opening operation, and output the initial contour of the tongue.

[0052] S4: Using the initial contour from S3 as the starting boundary of the active contour model, an energy function is defined that includes a contour length term, a region grayscale consistency term, and a tongue color crisis index feedback term. The tongue color crisis index calculates the weight scores for pale white, red, dark purple, and bluish-purple tongues in real time based on the Lab values ​​of pixels within the current segmentation region. These scores, in turn, drive contour expansion or contraction. Iteration stops when the contour change between two adjacent iterations is less than one pixel and the tongue color index fluctuation is less than 5%, outputting a finely segmented tongue region. This mechanism of mutual feedback between segmentation and tongue color quantization can eliminate the boundary drift problem caused by the thickness of the tongue coating.

[0053] S5: Within the tongue region obtained in S4, locate the course of the sublingual vein using a grayscale projection curve. If the vein region is over-segmented or omitted, trigger local contour adjustment and revert to the last valid iteration of S4 for re-execution.

[0054] S6: Perform hole filling and edge smoothing on the validated tongue mask. The seed filling algorithm removes tiny holes inside the tongue, and cubic spline interpolation smooths the contour edges to obtain the final tongue segmentation result.

[0055] S7: Pass the segmentation results of S6 to the tongue color quantitative analysis module, and output the mean values ​​of L, a, and b of the tongue body region to calculate the tongue color crisis index, which serves as the input feature for subsequent syndrome differentiation.

[0056] In step S1, the tongue image acquisition uses a ring-shaped lighting device with a standard D65 light source, the color temperature is fixed at 6500K, and a polarizing filter is installed in front of the light source to eliminate reflections on the tongue surface. During shooting, the distance between the tongue and the lens is kept at 15 cm, and after automatic focusing, 3 frames are taken continuously. The frame with the highest clarity is taken as the original tongue image.

[0057] In step S2, the image is first assessed for excessively dark or bright areas based on the global average brightness. If the average brightness is below 50 or above 200, automatic exposure compensation is triggered. Then, the RGB image is converted to the Lab color space, and only the L channel is extracted for subsequent segmentation. To address the uneven illumination issue, a brightness correction method based on local statistics is employed. The L channel image is divided into a 16x16 grid, and the difference between the average brightness of each grid and the global average is calculated. Bilinear interpolation is used to obtain the correction coefficient for each pixel. Finally, the corrected L channel image is output to step S3.

[0058] In step S3, the corrected L-channel image output from S2 is used as input. First, a global threshold is calculated on the a-channel. The pixel values ​​of the a-channel are evenly divided into 256 levels from the minimum to the maximum value. The proportion of pixels in each level to the total number of pixels is calculated, and then the gray level that maximizes the inter-class variance is found as the segmentation threshold. A binary mask is generated based on this threshold. Pixels greater than the threshold are marked as 1, representing the tongue candidate region, and pixels less than the threshold are marked as 0, representing the background. Next, a morphological opening operation is performed on the binary mask using a circular structuring element with a radius of 5 pixels, first eroding and then dilating, to remove isolated small noise regions generated by the oral cavity wall and teeth. Finally, the largest connected component is extracted as the initial contour of the tongue, and the boundary coordinates of this contour are passed to S4 as the starting contour.

[0059] In step S4, a level set framework is selected to implicitly represent the contour as the zero level set of the level set function.

[0060] The energy function includes a basic term and a tongue color crisis index feedback term. The basic term consists of a contour length term and a region grayscale consistency term. The contour length term penalizes overly convoluted edges to ensure smooth segmentation results. The region grayscale consistency term adopts the idea of ​​the Chan-Vese model, assuming uniform grayscale inside the tongue and uniform grayscale in the background, and is characterized by the mean grayscale values ​​of the L channels inside and outside the level set, respectively.

[0061] The tongue color crisis index feedback term calculates the tongue color index in real time based on the Lab values ​​of all pixels within the current segmentation region in each iteration. The index level then drives the contour to contract or expand. Specifically, if the calculated tongue color index for the current region is high, indicating a potential dark purple or bluish-purple tongue crisis, the feedback term generates a negative driving force, causing the contour to contract inward to prevent dark ecchymosis or sublingual veins from being mistakenly included in the main tongue region. If the tongue color index is low, corresponding to a pale white or bright red tongue, the feedback term allows the contour to expand moderately outward to cover the entire tongue surface.

[0062] In step S4, before each iteration begins, the membership degree and weighted total score of the four tongue colors are recalculated based on the pixel set inside the current contour, and then the feedback velocity function in the level set evolution equation is updated.

[0063] The evolution equation itself is composed of contour curvature, internal and external grayscale differences, and tongue color feedback speed. The contour curvature term keeps the edges smooth, while the grayscale difference term pushes the contour to move towards the actual tongue edge; The tongue color feedback speed acts as an additional moderating force, its magnitude and sign determined by the real-time value of the tongue color crisis index. A positive feedback speed causes the contour to expand outwards, while a negative speed causes it to contract inwards. This makes the segmentation process no longer purely geometrically driven, but incorporates semantic knowledge from traditional Chinese medicine tongue diagnosis. For example, if the initial contour inadvertently includes part of the dark red oral cavity wall, the α-channel values ​​in these areas are high, leading to an increase in the membership degree of the current region's dark purple tongue and a higher tongue color index. The feedback term immediately drives the contour to contract inwards, thus excluding the dark red area. Conversely, if the current region is too small and the tongue color exhibits pale characteristics, it indicates that the pale parts of the tongue's edge may have been missed, and the feedback term encourages the contour to expand outwards.

[0064] The evolution equation for the level set with tongue color feedback is: Parameter definition: It is a level set function, whose zero level set represents the current contour.

[0065] This represents the iteration time step.

[0066] It is a Dirac function, which is non-zero only near the contour.

[0067] The weight of the contour length term controls the edge smoothness.

[0068] The curvature of the contour.

[0069] and These are the grayscale consistency weights for the inside and outside of the contour, respectively.

[0070] The image pixel grayscale values ​​are represented using the L channel in Lab color space.

[0071] and These are the average gray values ​​of the L channel inside and outside the current contour, respectively.

[0072] This is the adjustment coefficient for the intensity of tongue color feedback.

[0073] The feedback speed function for the tongue color crisis index.

[0074] The specific expression for the feedback velocity function is: Parameter definition: The preset weights for pale white tongue, reddish-brown tongue, dark purple tongue, and bluish-purple tongue are 5, 8, 15, and 25, respectively.

[0075] The membership degree of the four tongue colors in the current segmentation region is calculated by the fuzzy membership function of the pixel Lab value and the standard threshold range, and the sum of the four membership degrees is 1.

[0076] It is a sign function; when it takes a negative value, it drives contour contraction, and when it takes a positive value, it drives expansion. Its absolute value is determined by the weighted mean.

[0077] In step S4, the iteration continues until both conditions are met simultaneously.

[0078] The first condition is that the maximum change in the contour position between two adjacent iterations is less than one pixel, which indicates that the geometry has converged.

[0079] The second condition is that the relative volatility of the Tongue Color Criticality Index is less than 5%. Volatility is defined as the absolute value of the difference between the mean of the Tongue Color Index in the current iteration and that in the previous iteration, divided by the mean of the previous iteration.

[0080] The second condition is added because simply looking at geometric changes may not be enough—sometimes the outline moves very little, but the internal pixel composition is still changing slowly, such as the uneven thickness of the tongue coating causing oscillations in the tongue color statistics. Only when both geometry and tongue color are stable can the segmentation be considered truly complete.

[0081] After stopping the iteration, the current zero-level set is output as the finely segmented tongue region. This region is passed to S5 for sublingual vein localization and is also output to the subsequent tongue color quantification module, realizing a closed-loop feedback between segmentation and color recognition.

[0082] The formula for calculating the volatility of the tongue color index in the iteration stopping condition is: Parameter definition: For the first The relative rate of change of the tongue color index in each iteration.

[0083] For the first The mean tongue color crisis index of all pixels in the segmented region at the end of the next iteration is calculated as follows: in The total number of pixels in the region. For pixels This is an indicator function for a pale tongue; the same applies to the others.

[0084] in The total number of pixels in the region. For pixels This is an indicator function for a pale tongue; the same applies to the others.

[0085] It should be a very small positive number to prevent the denominator from being zero. The iteration terminates when the maximum change in the contour position is less than one pixel.

[0086] In step S5, the sublingual vein is located within the mask using a grayscale projection curve. First, the cumulative grayscale value of each column of pixels within the mask is calculated to obtain the vertical projection curve. The column coordinates corresponding to the lowest point of the curve are found as the midline of the tongue. Then, the cumulative grayscale value of each row of pixels within a 10-pixel range to the left and right of the midline is calculated to obtain the horizontal projection curve. The two troughs on this curve correspond to the start and end points of the sublingual vein, respectively. Based on these two points, the region of interest for the vein is defined, and the Lab values ​​of the pixels within the region are extracted. If the area of ​​the vein region is less than 5% of the total area of ​​the tongue, or the average a value of the vein region is greater than 20 and the b value is less than 5, it is determined to be a valid vein region. Otherwise, the vein extraction is considered incomplete, triggering a local contour adjustment signal, returning to the last valid iteration of S4, and re-execution.

[0087] In step S6, a seed filling algorithm is used to fill all the holes inside the mask. Specifically, a seed pixel located inside the mask is selected and diffused to its four neighbors until it encounters the mask boundary. All pixels that have diffused through this area are marked as filling regions. For the burrs on the mask boundary caused by uneven tongue coating thickness, a cubic spline interpolation method is used for edge smoothing. Thirty control points are selected at equal intervals from the mask contour, and cubic spline fitting is performed on the horizontal and vertical coordinates of these control points to obtain smooth closed curves. Then, the pixels inside the curve are reassigned to 1, and the pixels outside are reassigned to 0, generating the final tongue segmentation mask.

[0088] In step S7, the L, a, and b components of all pixels within the mask region are extracted. The mean of the L channel, the mean of the a channel, the mean of the b channel, and the standard deviation of each of the three channels are calculated. These statistics, along with the total number of pixels in the mask region, are packaged into a tongue feature vector and passed to the tongue color quantification analysis module. This module determines the tongue color type and calculates the tongue color severity index based on the L, a, and b mean values, while retaining the segmentation mask itself for subsequent tongue coating analysis and sublingual vein analysis as a localization reference.

[0089] From the above, we can conclude that: This invention introduces a tongue color crisis index feedback term, enabling the segmented contour to determine the tongue color type in real time based on the pixels already segmented within the current region during the iteration process. If critical features such as a dark purple or bluish-purple tongue are detected, the feedback term automatically drives the contour to shrink inward, actively eliminating dark interference structures that do not belong to the main tongue region. Conversely, if the current region is too small and exhibits pale tongue characteristics, the feedback term allows the contour to expand moderately to ensure the integrity of the tongue surface information. This mechanism of mutual feedback between segmentation and color identification ensures that the final output tongue region achieves optimal performance in both geometric shape and TCM semantics. Not only does the contour edge more closely match the actual tongue boundary, but the segmentation result also naturally conforms to the definition requirements of the main tongue region in TCM tongue diagnosis.

[0090] In this invention, the stopping condition requires both a contour position change of less than one pixel and a tongue color index fluctuation of less than 5%. This ensures that the segmentation results do not fluctuate due to different iteration counts, allowing for highly repeatable segmentation of tongue images from the same patient acquired at different time points. Simultaneously, the output precisely segmented region is used for sublingual vein localization. When vein extraction is incomplete, local contour adjustment is triggered, and the process is reverted for re-execution. This closed-loop feedback design between the preceding and following steps prevents erroneous segmentation results from propagating to downstream modules. The final output L, a, and b statistics accurately reflect the color characteristics of the tongue without being affected by uneven tongue coating thickness or boundary burrs. This allows the method to be stably deployed in TCM smart wearable devices or TCM hospital monitoring systems, providing reliable segmentation results for subsequent tongue color criticality index calculation, tongue coating texture analysis, and sublingual vein grading.

[0091] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the term "comprising" or any other variations thereof is intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the element.

[0092] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for segmenting tongue diagnosis images based on computer vision, characterized in that: The method includes the following steps: S1: Acquire tongue images and extract ambient lighting parameters; use D65 standard light source ring lighting, acquire original tongue images through CMOS sensor, and record the global average brightness and color temperature deviation of the image; S2: Correct the uneven brightness areas of the image according to the illumination parameters of S1, convert the RGB image to the Lab color space, and extract the L channel for subsequent tongue and background separation; S3: Obtain a binarized mask on the a channel of the Lab space using the Otsu's method, remove small noise regions in the oral cavity using morphological opening operation, and output the initial contour of the tongue. S4: Using the initial contour of S3 as the starting boundary of the active contour model, the energy function is defined to include a contour length term, a region grayscale consistency term, and a tongue color crisis index feedback term. The tongue color crisis index calculates the weight scores of pale white tongue, red tongue, dark purple tongue, and bluish-purple tongue in real time based on the Lab value of the pixels in the current segmentation region. This score, in turn, drives the contour expansion or contraction. When the contour change between two adjacent iterations is less than one pixel and the tongue color index fluctuation is less than five percent, the iteration stops and the finely segmented tongue region is output. S5: Within the tongue region obtained in S4, locate the course of the sublingual vein using a grayscale projection curve; if the vein region is over-segmented or omitted, trigger local contour adjustment and revert to the last valid iteration of S4 to re-execute. S6: Perform hole filling and edge smoothing on the verified tongue mask; use seed filling algorithm to remove tiny holes inside the tongue, and use cubic spline interpolation to smooth the contour edges to obtain the final tongue segmentation result; S7: Pass the segmentation results of S6 to the tongue color quantitative analysis module, and output the mean values ​​of L, a, and b of the tongue body region to calculate the tongue color crisis index, which serves as the input feature for subsequent syndrome differentiation.

2. The tongue diagnosis image segmentation method based on computer vision as described in claim 1, characterized in that: In step S1, the tongue image acquisition uses a ring-shaped light source with a standard D65 light source and a fixed color temperature of 6500K. A polarizing filter is installed in front of the light source to eliminate reflections on the tongue surface. During shooting, the distance between the tongue and the lens is kept at 15cm. After automatic focusing, three frames are continuously captured, and the frame with the highest clarity is taken as the original tongue image.

3. The tongue diagnosis image segmentation method based on computer vision as described in claim 1, characterized in that: In step S2, firstly, it is determined whether there are excessively dark or excessively bright areas in the image based on the global average brightness value. If the average brightness value is lower than 50 or higher than 200, automatic exposure compensation is triggered. Then, the RGB image is converted to the Lab color space, and only the L channel is extracted for subsequent segmentation. To address the problem of uneven illumination, a brightness correction method based on local statistics is adopted. The L channel image is divided into a 16x16 grid, and the difference between the average brightness value of each grid and the global average is calculated. The correction coefficient of each pixel is obtained through bilinear interpolation. Finally, the corrected L channel image is output to S3.

4. The tongue diagnosis image segmentation method based on computer vision as described in claim 1, characterized in that: In step S3, the corrected L-channel image output from S2 is used as input; firstly, a global threshold is calculated on the a-channel, and the pixel values ​​of the a-channel are evenly divided into 256 levels from the minimum to the maximum value. The proportion of the number of pixels in each level to the total number of pixels is calculated, and then the gray level that maximizes the inter-class variance is found as the segmentation threshold. A binary mask is generated based on the threshold. Pixels larger than the threshold are marked as 1 to represent candidate tongue regions, and pixels smaller than the threshold are marked as 0 to represent the background. Then, a morphological opening operation is performed on the binary mask using a circular structuring element with a radius of 5 pixels, first eroding and then dilating, to remove isolated small noise regions generated by the oral cavity wall and teeth. Finally, the largest connected component is extracted as the initial contour of the tongue, and the boundary coordinates of this contour are passed to S4 as the starting contour.

5. The tongue diagnosis image segmentation method based on computer vision as described in claim 1, characterized in that: In step S4, a level set framework is selected to implicitly represent the contour as the zero level set of the level set function. The energy function includes a basic term and a tongue color crisis index feedback term. The basic term consists of a contour length term and a region grayscale consistency term. The contour length term is used to penalize overly curved edges to ensure smooth segmentation results. The region grayscale consistency term adopts the idea of ​​the Chan-Vese model, assuming that the grayscale inside the tongue is uniform and the background grayscale is uniform, and is characterized by the mean grayscale of the L channel inside and outside the level set, respectively. The tongue color crisis index feedback item calculates the tongue color index in real time based on the Lab value of all pixels in the current segmentation area in each iteration, and then drives the contour to shrink or expand according to the index level. The specific logic is as follows: if the calculated tongue color index for the current area is high, it means that there may be a purplish-dark tongue or bluish-purple tongue crisis in the area. At this time, the feedback item will generate a negative driving force to make the outline shrink inward, so as to avoid including the dark ecchymosis on the edge of the tongue or the sublingual veins into the main area of ​​the tongue. If the tongue color index is low, corresponding to a pale white tongue or a reddish-brown tongue, the feedback item will allow the outline to expand outward appropriately to cover the entire tongue surface.

6. The tongue diagnosis image segmentation method based on computer vision as described in claim 1, characterized in that: In step S4, before each iteration begins, the membership degree and weighted total score of the four tongue colors are recalculated based on the pixel set inside the current contour, and then the feedback velocity function in the level set evolution equation is updated. The evolution equation itself is composed of contour curvature, internal and external grayscale difference, and tongue color feedback speed; the contour curvature term keeps the edge smooth, and the grayscale difference term pushes the contour to move towards the real tongue edge; The evolution equation for the level set with tongue color feedback is: Parameter definition: It is a level set function, whose zero level set represents the current contour; This represents the iteration time step. For Dirac functions, non-zero values ​​are only found near the contour; The weight of the contour length term controls the edge smoothness; The curvature of the contour; and These are the grayscale consistency weights for the inside and outside of the contour, respectively; For the image pixel grayscale values, use the L channel in the Lab color space; and These are the average gray values ​​of the L channel inside and outside the current contour, respectively; This is the adjustment coefficient for the intensity of tongue color feedback; The feedback velocity function for the tongue color crisis index; The specific expression for the feedback velocity function is: Parameter definition: The preset weights for pale white tongue, reddish-brown tongue, dark purple tongue, and bluish-purple tongue are 5, 8, 15, and 25, respectively. The membership degrees of the four tongue colors in the current segmentation region are calculated by the fuzzy membership function of the pixel Lab value and the standard threshold range, and the sum of the four membership degrees is 1. It is a sign function; when it takes a negative value, it drives contour contraction, and when it takes a positive value, it drives expansion. Its absolute value is determined by the weighted mean.

7. The tongue diagnosis image segmentation method based on computer vision as described in claim 1, characterized in that: In step S4, the iteration continues until both conditions are met simultaneously. The first condition is that the maximum change in the contour position between two adjacent iterations is less than one pixel, which indicates that the geometry has converged; The second condition is that the relative volatility of the tongue color crisis index is less than 5%; volatility is defined as the absolute value of the difference between the mean of the tongue color index in the current iteration and the mean of the previous iteration, divided by the mean of the previous iteration. After stopping the iteration, the current zero level set is output as the finely segmented tongue region; this region is passed to S5 for sublingual vein localization, and is also output to the subsequent tongue color quantification module, realizing a closed-loop feedback between segmentation and color recognition. The formula for calculating the volatility of the tongue color index in the iteration stopping condition is: Parameter definition: For the first The relative rate of change of the tongue color index in each iteration; For the first The mean tongue color crisis index of all pixels in the segmented region at the end of the next iteration is calculated as follows: in The total number of pixels in the region. For pixels This is an indicator function for pale tongue; the same applies to the others. in The total number of pixels in the region. For pixels This is an indicator function for pale tongue; the same applies to the others. It should be a very small positive number to prevent the denominator from being zero; when The iteration terminates when the maximum change in the contour position is less than one pixel.

8. The tongue diagnosis image segmentation method based on computer vision as described in claim 1, characterized in that: In step S5, the sublingual vein is located within the mask using a grayscale projection curve. First, the cumulative grayscale value of each column of pixels within the mask is calculated to obtain the vertical projection curve. The column coordinates corresponding to the lowest point of the curve are found as the midline of the tongue. Then, the cumulative grayscale value of each row of pixels within a 10-pixel range to the left and right of the midline is calculated to obtain the horizontal projection curve. The two troughs on this curve correspond to the starting and ending points of the sublingual vein, respectively. Based on these two points, the region of interest for the vein is defined, and the Lab values ​​of the pixels within the region are extracted. If the area of ​​the vein region is less than 5% of the total area of ​​the tongue, or the average a value of the vein region is greater than 20 and the b value is less than 5, it is determined to be a valid vein region. Otherwise, the vein extraction is considered incomplete, triggering a local contour adjustment signal, returning to the last valid iteration of S4, and re-execution.

9. The tongue diagnosis image segmentation method based on computer vision as described in claim 1, characterized in that: In step S6, a seed filling algorithm is used to fill all the holes inside the mask. Specifically, a seed pixel located inside the mask is selected and diffused to its 4-neighborhood until it encounters the mask boundary. All pixels that have diffused through the mask are marked as filling areas. For the burrs on the mask boundary caused by uneven tongue coating thickness, a cubic spline interpolation method is used to smooth the edges. 30 control points are selected at equal intervals from the mask contour. The horizontal and vertical coordinates of these control points are fitted with cubic splines to obtain smooth closed curves. Then, the pixels inside the curve are reassigned to 1, and the pixels outside are reassigned to 0, generating the final tongue segmentation mask.

10. The tongue diagnosis image segmentation method based on computer vision as described in claim 1, characterized in that: In step S7, the L, a, and b components of all pixels within the mask area are extracted; the mean of the L channel, the mean of the a channel, the mean of the b channel, and the standard deviation of the three channels are calculated respectively; these statistics, together with the total number of pixels in the mask area, are packaged into a tongue feature vector and passed to the tongue color quantification analysis module; this module determines the tongue color type and calculates the tongue color crisis index based on the L, a, and b mean values, while retaining the segmentation mask itself for subsequent tongue coating analysis and sublingual vein analysis as a positioning reference.

Images