Colorectal cancer risk screening system based on tongue picture analysis

Abstract

The invention relates to the technical field of data analysis and screening, and particularly discloses a colorectal cancer risk screening system based on tongue picture analysis, which comprises the following steps: obtaining a tongue extension-free tongue image and an expiration sample, carrying out denoising, color recovery and segmentation on the tongue image to obtain independent tongue picture features, extracting hue saturation, texture and morphological parameters, and normalizing to generate tongue picture feature vectors; markers such as methane are detected from an expiration sample through tunable semiconductor laser absorption spectrum and photoacoustic spectrum, concentration vectors are generated after calibration, the two types of vectors are subjected to principal component analysis dimension reduction and cross-modal attention weighted fusion to obtain joint features, then the joint features are input into a double-branch deep learning model, a risk probability value is obtained after fusion output, and the risk probability value is obtained. Dividing low, medium and high risks and outputting suggestions. According to the invention, a standard and objective data analysis screening technical field path can be constructed, the precision defect of a traditional screening system is made up, the accessibility disorder of examination is overcome, and a technical solution is provided for early discovery, early diagnosis and early treatment of colorectal cancer.

Description

Technical Field

[0001] This invention relates to the field of data analysis and screening technology, and in particular to a colorectal cancer risk screening system based on tongue image analysis. Background Technology

[0002] In the field of colorectal cancer risk screening, traditional screening methods have obvious limitations. For example, the fecal occult blood test relies on stool sample testing and is easily affected by diet and drugs, leading to false positives or false negatives, resulting in insufficient screening accuracy. Although colonoscopy and pathological biopsy are highly accurate, they are invasive, costly, and complicated to operate, and patients have poor compliance, making them unsuitable for early screening of large populations.

[0003] Meanwhile, traditional tongue image analysis, as an auxiliary diagnostic tool, relies heavily on the subjective judgment of physicians and is greatly affected by lighting conditions, shooting angle, and individual experience differences. It lacks standardized tongue image acquisition, preprocessing, and feature extraction processes, making it impossible to achieve objective quantification of tongue image features. Furthermore, existing tongue image analysis methods mostly rely solely on tongue image information and do not combine human metabolic indicators to construct a multimodal assessment system, making it difficult to comprehensively reflect the physiological changes in the process of colorectal cancer development, thus limiting the accuracy and stability of screening.

[0004] In addition, existing multimodal screening technologies often employ simple splicing or fixed-weight fusion methods for feature fusion, failing to fully explore the correlation and complementarity between different modal features. Furthermore, risk assessment models are mostly single-branch network structures, which cannot simultaneously and accurately capture both subtle local features and globally correlated features, further reducing the accuracy and efficiency of colorectal cancer risk screening.

[0005] Therefore, there is an urgent need for a colorectal cancer risk screening system based on tongue image analysis to solve the above problems. Summary of the Invention

[0006] The purpose of this invention is to provide a colorectal cancer risk screening system based on tongue image analysis, comprising: The first acquisition module is used to acquire images of the tongue and exhalation samples in the tongue-free state; The second acquisition module is used to remove noise from the tongue image using a deep learning denoising network, obtain a corrected tongue image, enhance the color and texture details of the tongue body through an image enhancement algorithm, and accurately separate the tongue body from the background area based on an instance segmentation model to obtain independent tongue image features. The third acquisition module is used to extract the hue saturation distribution features, tongue texture features, and tongue morphology features of the tongue body and tongue coating based on the independent tongue image features, and to obtain the tongue image feature vector based on the hue saturation distribution features, tongue texture features, and tongue morphology features of the tongue body and tongue coating. The concentration acquisition module is used to perform quantitative analysis of exhaled biomarkers on the exhaled sample and obtain the exhaled biomarker concentration vector. The fusion module is used to fuse and generate joint features based on the tongue image feature vector and the exhaled biomarker concentration vector, and to establish a dual-branch deep learning model. The joint features are input into the dual-branch deep learning model. The API-Net branch extracts local detail features of the tongue image, and the TransFG branch learns global semantic associations of the tongue image. The outputs of the two branches are fused and the risk probability value is calculated. The output module is used to automatically classify low-risk, medium-risk, and high-risk levels based on the risk probability value and output corresponding suggestions.

[0007] Furthermore, the second acquisition module is specifically used for: The signal-to-noise ratio (SNR) of the RGB channels of the original tongue image is calculated. When the SNR is detected to be lower than 40 dB, the U-Net denoising network is triggered to process the image. A noise distribution model is constructed by analyzing the deviation between the local pixel intensity and the mean, and a corrected tongue image with an SNR of not less than 50 dB is output. Extract the hue and saturation distribution histogram of the corrected tongue image. When the hue mean is detected to deviate from the preset range, perform color restoration processing and output a standard tongue image with a stable color temperature within the standard color temperature range. In the Lab color space, the contrast of the tongue area is quantified. When the contrast value is lower than the preset contrast value, texture enhancement is activated. The Gaussian kernel parameters are dynamically adjusted according to the local image variance. The enhanced image is input into the MaskR-CNN segmentation model, and the tiny holes are filled through morphological closing operations. The output is an independent tongue image feature with a segmentation accuracy no lower than the preset accuracy value.

[0008] Furthermore, the third acquisition module is specifically used for: Calculate the average hue and saturation values ​​of the tongue body region and the tongue coating region in the HSV color space, and obtain the hue and saturation parameters; Extract the gray-level co-occurrence matrix of the gray-level image of the tongue surface, and calculate the three texture parameters of the matrix: contrast, energy, and entropy. The serration index of the tongue edge contour and the depth ratio of tongue surface cracks were measured, and morphological parameters were obtained. The hue saturation parameters, texture parameters, and shape parameters are combined into a multi-dimensional feature vector in a fixed order, and the values ​​of each dimension are normalized.

[0009] Furthermore, the concentration acquisition module is specifically used for: The concentrations of methane and ethane in exhaled breath samples were detected using tunable semiconductor laser absorption spectroscopy. The concentration values ​​of volatile organic compounds such as acetone and isoprene were quantitatively analyzed using photoacoustic spectroscopy. The instrument is periodically calibrated using a built-in standard gas calibration system, and system errors are monitored in real time using an internal standard gas channel. The concentration values ​​of each gas are arranged in the order of methane, ethane, acetone, and isoprene to form a concentration vector, and the concentration values ​​are logarithmically transformed to improve the data distribution characteristics.

[0010] Furthermore, the fusion module is specifically used for: Principal component analysis was performed to reduce the dimensionality of the tongue image feature vector and the exhaled biomarker concentration vector, and the principal components whose cumulative contribution rate reached the threshold were retained. The correlation coefficient between the two sets of features is calculated using the canonical correlation analysis algorithm, and the feature projection direction is determined based on the principle of maximum correlation. A cross-modal attention mechanism was used to calculate the association weights between tongue image features and expiratory markers, where the attention weights of tongue image features were learned through a multilayer perceptron network. The two types of features are weighted and fused according to the attention weights to generate a joint feature vector, the dimension of which is half of the sum of the dimensions of the original features.

[0011] Furthermore, the output module is specifically used for: The joint feature vector is simultaneously input into two branches of the network: API-Net and TransFG. The API-Net branch uses a dilated convolutional structure to capture local details of the tongue image, while the TransFG branch uses a self-attention mechanism to learn global semantic associations. A feature interaction mechanism is established between the two branch networks, and the mutual enhancement of local and global features is achieved through the cross attention module; The output features of the two branches are merged, and a gating mechanism is used to dynamically adjust the contribution ratio of local features and global features; The fused features are mapped to a single risk score through a fully connected layer, and the score is converted into a risk probability value in the range of 0-1.

[0012] Furthermore, the present invention also discloses a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement a colorectal cancer risk screening system based on tongue image analysis.

[0013] Furthermore, the present invention also discloses a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, enables the operation of a colorectal cancer risk screening system based on tongue image analysis.

[0014] The beneficial effects of this application are as follows: This invention acquires tongue images and exhalation samples in a tongue-free state, and through image preprocessing, feature extraction, exhalation biomarker analysis, dual-modal feature fusion, and dual-branch deep learning model calculation, ultimately achieves colorectal cancer risk level classification and recommendation output. It aims to build a standardized, objective, and automated data analysis and screening technology path, make up for the accuracy defects of traditional screening systems, overcome accessibility barriers, and provide a brand-new technical solution for early detection, early diagnosis, and early treatment of colorectal cancer. Attached Figure Description

[0015] Figure 1 This is a schematic diagram of the system structure proposed in an embodiment of the present invention.

[0016] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0017] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0018] like Figure 1 As shown, this application provides a colorectal cancer risk screening system based on tongue image analysis, including: The first acquisition module 1 is used to execute step S1, specifically to acquire a tongue image and an exhalation sample in the tongue-free state; The second acquisition module 2 is used to execute step S2, specifically to remove noise from the tongue image using a deep learning denoising network, obtain a corrected tongue image, enhance the color and texture details of the tongue body through an image enhancement algorithm, and accurately separate the tongue body from the background area based on an instance segmentation model to obtain independent tongue image features. The third acquisition module 3 is used to execute step S3, specifically to extract the hue saturation distribution features, tongue texture features, and tongue morphology features of the tongue body and tongue coating based on the independent tongue image features, and to obtain the tongue image feature vector based on the hue saturation distribution features, tongue texture features, and tongue morphology features of the tongue body and tongue coating. Concentration acquisition module 4 is used to execute step S4, specifically to perform quantitative analysis of exhaled biomarkers on the exhaled sample and obtain the exhaled biomarker concentration vector. The fusion module 5 is used to execute step S5, specifically to fuse and generate joint features based on the tongue image feature vector and the exhaled biomarker concentration vector, and to establish a dual-branch deep learning model. The joint features are input into the dual-branch deep learning model, and the local detail features of the tongue image are extracted through the API-Net branch, and the global semantic association of the tongue image is learned through the TransFG branch. The outputs of the two branches are fused and the risk probability value is calculated. Output module 6 is used to execute step S6, specifically to automatically classify low-risk, medium-risk and high-risk levels according to the risk probability value, and output corresponding suggestions.

[0019] As described in the functions of the modules above, this invention acquires tongue images and exhalation samples in a tongue-free state, and through image preprocessing, feature extraction, exhalation biomarker analysis, dual-modal feature fusion, and dual-branch deep learning model calculation, ultimately achieves colorectal cancer risk level classification and recommendation output. It aims to build a standardized, objective, and automated data analysis and screening technology path, make up for the accuracy defects of traditional screening systems, overcome accessibility barriers to examinations, and provide a brand-new technical solution for early detection, early diagnosis, and early treatment of colorectal cancer.

[0020] In the field of colorectal cancer risk screening, traditional screening systems such as fecal occult blood tests suffer from insufficient accuracy and are easily affected by factors such as diet and medication, leading to false positives or false negatives. While colonoscopy and other examinations offer higher accuracy, they are invasive, costly, and have poor patient compliance, making it difficult to achieve early screening for large populations. Furthermore, traditional tongue image analysis relies heavily on subjective judgment by physicians, influenced by factors such as lighting conditions, shooting angle, and physician experience. It lacks standardized image acquisition and feature extraction procedures and relies solely on tongue image features for disease risk assessment without incorporating other physiological indicators, thus limiting screening accuracy. To address these technical issues, this invention proposes a dual-modal fusion solution combining tongue image and exhalation. Through standardized sample collection, refined image preprocessing, multi-dimensional feature extraction and fusion, and a targeted dual-branch deep learning model, it achieves accurate and convenient colorectal cancer risk screening.

[0021] First, acquiring images of the tongue and exhalation samples in a tongue-free state is the basic data collection step of the entire screening system, providing raw data support for subsequent feature extraction and risk assessment. The tongue images, obtained without the tongue protruding, are captured using an industrial camera with a resolution of at least 20 megapixels and a large-aperture macro lens, under a ring-shaped shadowless light source with automatically adjusted illumination intensity. This equipment and lighting scheme are chosen because traditional tongue image acquisition often results in the loss of tongue color and texture details due to uneven illumination and insufficient resolution. The 20-megapixel industrial camera ensures high image clarity, the large-aperture macro lens clearly captures subtle features such as tongue papillae and tongue coating texture, and the ring-shaped shadowless light source with automatic illumination intensity adjustment avoids shadow interference, ensuring consistent illumination of tongue images under different acquisition scenarios and laying a standardized foundation for subsequent image preprocessing and feature extraction. Simultaneously, breath samples are collected using a breath sampling device treated with an anti-adsorption coating. This coating prevents volatile organic compounds in the breath from adsorbing onto the inner wall of the sampling device, avoiding detection errors caused by sample component loss and ensuring the accuracy of breath biomarker concentration detection.

[0022] After acquiring the original tongue image and exhalation sample, the tongue image needs to be preprocessed to eliminate noise, enhance features, and separate the tongue from the background. This involves using a deep learning denoising network to remove noise from the tongue image, obtaining a corrected tongue image, enhancing the tongue's color and texture details using image enhancement algorithms, and accurately separating the tongue from the background region based on an instance segmentation model to obtain independent tongue image features. The core objective is to address potential noise interference, color distortion, feature blurring, and background redundancy issues in the original tongue image, providing high-quality image data for subsequent accurate extraction of tongue image features.

[0023] After obtaining independent tongue image features, key features need to be extracted and a tongue image feature vector constructed. This involves extracting the hue and saturation distribution features of the tongue body and coating, tongue texture features, and tongue morphology features based on the independent tongue image features, and then obtaining the tongue image feature vector based on these features. The purpose is to transform image features into numerical features that can be used for model calculations, and to comprehensively capture the physiological information related to colorectal cancer risk in the tongue image through multi-dimensional feature extraction.

[0024] While extracting tongue features, it is necessary to perform quantitative analysis of exhaled biomarkers on the exhaled breath samples to obtain the concentration vector of exhaled biomarkers. This is because the metabolic process in colorectal cancer patients may change, which may lead to abnormal concentrations of certain volatile organic compounds in the exhaled breath. By detecting the concentration of these biomarkers, information on tongue features can be supplemented, thereby improving the accuracy of risk screening.

[0025] After obtaining the tongue image feature vector and the exhalation biomarker concentration vector, it is necessary to fuse these two types of vectors to generate joint features. The purpose of this step is to make full use of the complementary information of the tongue image and exhalation modalities, avoid screening errors caused by incomplete feature information of a single modality, and improve the model's ability to identify colorectal cancer risk.

[0026] After generating joint features, a dual-branch deep learning model needs to be established. The joint features are input into the model, and the API-Net branch extracts local detailed features of the tongue image, while the TransFG branch learns the global semantic association of the tongue image. The outputs of the two branches are fused and the risk probability value is calculated. This step is the core of achieving accurate risk assessment of colorectal cancer. By capturing local subtle features and global association information through the dual-branch structure, the model's ability to identify risk features is improved.

[0027] Finally, based on the calculated risk probability values, the system automatically classifies risks into low-risk (<0.3), medium-risk (0.3-0.7), and high-risk (≥0.7) levels and outputs corresponding suggestions. This step transforms the model calculation results into information that can be referenced in actual clinical applications, realizing the practical value of the screening system. The risk probability thresholds were determined based on statistical analysis of a large number of clinical sample data. By calculating the risk probability of samples with known colorectal cancer status, the correspondence between different probability intervals and actual disease status was analyzed. Finally, the classification criteria were determined as follows: <0.3 corresponds to low risk, 0.3-0.7 corresponds to medium risk, and ≥0.7 corresponds to high risk. This standard can ensure the sensitivity of screening while taking into account specificity, reducing the risk of missed diagnosis and misdiagnosis. Corresponding recommendations are formulated according to different risk levels. For example, for low-risk levels, it is recommended to conduct screening of this system regularly (e.g., annually); for medium-risk levels, it is recommended to further conduct fecal occult blood tests or colonoscopy; and for high-risk levels, it is recommended to conduct colonoscopy as soon as possible to clarify the diagnosis. These recommendations are based on clinical diagnosis and treatment guidelines to ensure their scientificity and practicality, providing clear directions for follow-up treatment for clinicians and examinees, and ultimately realizing the clinical application value of colorectal cancer risk screening.

[0028] In one embodiment, the second acquisition module 2 is specifically used for: The first output unit is used to execute step S21, specifically to calculate the signal-to-noise ratio (SNR) of the RGB channels of the original tongue image. When the SNR is detected to be lower than 40 dB, the U-Net denoising network is triggered to process the noise distribution model by analyzing the deviation between the local pixel intensity and the mean, and outputs a corrected tongue image with an SNR of not less than 50 dB. The first extraction unit is used to execute step S22, specifically to extract the hue saturation distribution histogram of the corrected tongue image. When the hue mean is detected to deviate from the preset range (the preset range is 15 degrees to 45 degrees), the Retinex algorithm is used for color restoration processing, and a standard tongue image with a stable color temperature in the standard color temperature range (the standard color temperature range is 5300K to 5700K) is output. The adjustment unit is used to execute step S23, specifically to quantify the contrast of the tongue region in the Lab color space. When the contrast value is lower than the preset contrast value (the preset contrast value is 0.3), texture enhancement is activated. The Gaussian kernel parameters are dynamically adjusted according to the local image variance to ensure that the clarity of the boundary between the tongue papillae and the tongue coating texture meets the medical recognition standard. The second output unit is used to execute step S24, specifically to input the enhanced image into the MaskR-CNN segmentation model, fill the tiny holes through morphological closing operations, and output independent tongue image features with segmentation accuracy not lower than the preset accuracy value (the preset accuracy value is 98%).

[0029] As described above, each unit performs noise reduction, color restoration, texture enhancement, and tongue-background segmentation on the original tongue images acquired in the tongue-free state. This process yields independent tongue features with a segmentation accuracy of no less than 98%. The aim is to eliminate noise interference, color distortion, texture blurring, and background redundancy in the original tongue images. This provides high-quality, standardized image data support for the subsequent accurate extraction of the hue and saturation distribution features, texture features, and morphological features of the tongue body and coating, thereby ensuring the accuracy and reliability of colorectal cancer risk screening.

[0030] Raw tongue images are susceptible to quality issues due to various factors during acquisition. Industrial cameras shooting in low-light conditions or electronic noise from the equipment itself can introduce Gaussian noise, salt-and-pepper noise, and other noises into the image. These noises can mask the subtle textures of the tongue surface (such as the morphology of the tongue papillae), affecting the accuracy of subsequent feature extraction. On the other hand, deviations in the intensity adjustment of the ring-shaped shadowless light source or interference from external light in the acquisition environment can lead to color distortion in the tongue image, such as the hue mean deviating from the normal physiological range. Since tongue color (such as pale red or crimson) is an important indicator of human physiological state, color distortion can directly lead to deviations in subsequent risk assessments based on color features. At the same time, some subjects' tongue images may have insufficient contrast, resulting in blurred boundaries between the tongue papillae and the tongue coating texture, making it difficult to distinguish subtle structural differences. In addition, raw tongue images usually contain background areas such as oral mucosa and teeth. If these background areas are not effectively segmented, their features will be mixed into the tongue features, interfering with the results of subsequent feature analysis. Therefore, a series of image preprocessing steps are necessary to address the aforementioned issues in order to ensure the accuracy of subsequent tongue feature extraction and provide reliable basic data for colorectal cancer risk screening.

[0031] Specifically, step S21 is denoising, which involves calculating the signal-to-noise ratio (SNR) of the RGB channels of the original tongue image. When the SNR is detected to be below 40 dB, the U-Net denoising network is triggered. By analyzing the deviation between local pixel intensity and the mean, a noise distribution model is constructed, and a corrected tongue image with an SNR of no less than 50 dB is output. The original tongue image is obtained by using an industrial camera with a resolution of no less than 20 megapixels and a large aperture macro lens, under a ring-shaped shadowless light source with automatically adjusted illumination intensity, capturing a tongue image in a tongue-free state. The SNR is obtained by calculating the ratio of signal power to noise power in the three RGB channels. When this ratio is below 40 dB, it indicates that image noise has significantly interfered with the tongue features, and denoising processing needs to be initiated. The U-Net denoising network employs an encoder-decoder structure. In the encoding stage, the image is downsampled using four convolutional blocks (each containing two 3×3 convolutional layers, one batch normalization layer, and one ReLU activation function) to progressively extract noise features and tongue texture features. Each convolutional block is followed by a 2×2 max pooling layer to achieve downsampling. In the decoding stage, the feature map is upsampled using four deconvolutional blocks (each containing one 2×2 deconvolutional layer, two 3×3 convolutional layers, one batch normalization layer, and one ReLU activation function) and then skipped to connect with the corresponding feature map from the encoding stage. This fuses shallow texture features with deep noise features, thereby preserving the subtle texture of the tongue while removing noise. By analyzing the deviation between local pixel intensity and mean to construct a noise distribution model, the U-Net denoising network can learn the noise patterns of different regions in a targeted manner, avoiding excessive smoothing of the tongue texture. The final output of the corrected tongue image has a signal-to-noise ratio of no less than 50 dB, which can effectively eliminate noise interference and completely preserve key features such as tongue papillae and tongue coating texture, laying the foundation for subsequent color restoration and texture enhancement.

[0032] S22 involves color restoration processing, specifically extracting the hue and saturation distribution histogram of the corrected tongue image. When the hue mean value deviates from the preset range (15 to 45 degrees), the Retinex algorithm is used for color restoration, outputting a standard tongue image with a stable color temperature within the standard range (5300K to 5700K). The hue and saturation distribution histogram is obtained by statistically analyzing the pixel distribution frequency of the hue and saturation channels in the corrected tongue image within the HSV color space. The hue mean value is determined by calculating the average value of all pixels in the hue channel. The preset range of 15 to 45 degrees represents the normal hue range of a tongue based on statistical analysis of a large amount of tongue image data from healthy individuals. If the hue mean value deviates from this range, it indicates color distortion in the image (e.g., reddish or yellowish tones), requiring color restoration. The Retinex algorithm decomposes a corrected tongue image into a reflection component (reflecting the tongue's inherent color characteristics) and an illumination component (reflecting external lighting interference). The specific steps are as follows: First, the corrected tongue image is converted from the RGB color space to the HSV color space, processing only the luminance channel. Then, a Gaussian filter (with a Gaussian kernel size of 15×15 and a standard deviation of 2.0) is applied to the luminance channel image to obtain the illumination component. Finally, the original luminance channel image is divided by the illumination component to obtain the luminance channel corresponding to the reflection component. The processed HSV image is then converted back to the RGB color space to complete color restoration. This algorithm eliminates color distortion caused by uneven lighting and external light interference, restoring the tongue's color to its true physiological state. The final output standard tongue image color temperature is stable between 5300K and 5700K, which is close to the color temperature of natural light, accurately reflecting the tongue's inherent color characteristics and avoiding errors in subsequent hue and saturation parameter extraction due to color deviation, thus ensuring the accuracy of risk assessment based on color features.

[0033] S23 is a texture enhancement process, which quantifies the contrast of the tongue region in the Lab color space. When the contrast value is lower than the preset contrast value (0.3), texture enhancement is activated. The Gaussian kernel parameters are dynamically adjusted according to the local image variance to ensure that the clarity of the boundary between the tongue papillae and the tongue coating texture meets medical recognition standards. The tongue region is determined by preliminary contour detection of a standard tongue image (using the Canny edge detection algorithm with a threshold of 50-150). The contrast value is obtained by calculating the difference between the maximum and minimum pixel values ​​of the luminance channel (L channel) in the Lab color space and dividing by the sum of the two. The preset contrast value of 0.3 is the minimum clarity standard set based on medical diagnostic requirements. If it is lower than this value, the boundary between the tongue papillae and the tongue coating texture will be blurred, making it difficult to distinguish the differences in texture details. Texture enhancement employs an adaptive Gaussian sharpening algorithm based on local variance. First, the standard tongue image is converted from the RGB color space to the Lab color space, and the L-channel image is extracted. Then, the L-channel image is divided into 10×10 local image patches, and the variance of each local image patch (reflecting the richness of texture in that area) is calculated. If the local image patch variance is less than 0.05 (indicating blurred texture), the Gaussian kernel standard deviation is set to 0.5 to enhance the sharpening effect. If the local image patch variance is greater than 0.1 (indicating clear texture), the Gaussian kernel standard deviation is set to 1.0 to avoid over-sharpening. Finally, the L-channel image is processed using the Gaussian sharpening formula (original pixel value + 0.8 × (original pixel value - Gaussian filtered pixel value)) to complete the texture enhancement. By dynamically adjusting the Gaussian kernel parameters, differentiated enhancement strategies can be adopted for regions with different texture densities. This avoids over-sharpening of clear textures, which can lead to artifacts, while effectively enhancing the clarity of the boundaries of blurred textures. This ensures that details such as the shape of the tongue papillae and the thickness of the tongue coating meet medical recognition standards (i.e., the texture boundaries are clearly distinguishable and there are no obvious artifacts), providing a clear image foundation for subsequent texture parameter extraction.

[0034] S24 is the tongue segmentation process, which involves inputting the enhanced image into the MaskR-CNN segmentation model, filling in the tiny holes through morphological closing operations, and outputting independent tongue image features with a segmentation accuracy of no less than 98%. The enhanced image is a standard tongue image with texture enhancement processing. The MaskR-CNN segmentation model uses ResNet-50 as the backbone network, and its specific structure includes: a feature extraction network (extracting image features through ResNet-50 and outputting a 512×512×2048 feature map), a region proposal network (RPN, which generates candidate boxes through 3×3 convolutional layers and 1×1 classification and regression layers, with candidate box areas set to 128×128, 256×256, and 512×512, and aspect ratios set to 1:1, 1:2, and 2:1), a region of interest alignment (ROIAlign, which maps candidate boxes to feature maps and obtains a fixed-size feature map of 7×7×2048 through bilinear interpolation), and classification, regression, and mask generation (achieving candidate box classification and bounding box regression through two fully connected layers, and generating a 14×14 binary mask through a 1×1 convolutional layer and a sigmoid activation function). This model can accurately identify the tongue region and background regions (oral mucosa, teeth, etc.), and output preliminary segmentation results. Morphological closing operations use 3×3 rectangular structuring elements to fill tiny pores (such as small gaps in tongue surface cracks) within the tongue region through a process of first expanding and then eroding, eliminating discontinuous areas in the segmentation results. The final output of independent tongue image features contains only the tongue region; the background region is completely removed. Segmentation accuracy is evaluated by calculating the intersection-over-union (IoU) ratio of the segmentation results and the manually labeled results, ensuring it is no less than 98%. This avoids interference from background features in subsequent tongue image feature extraction while fully preserving morphological features such as the tongue edge contour and tongue surface cracks, providing an accurate regional basis for subsequent morphological parameter extraction.

[0035] Through the above preprocessing operations, the original tongue image can be transformed into independent tongue image features that are noise-free, color-standard, texture-clear, and free from background interference. Each processing step addresses the specific problems existing in the previous image, forming a progressive optimization process. This ensures that the subsequently extracted tongue body and tongue coating hue saturation distribution features, tongue image texture features, and tongue image morphology features can accurately reflect the examinee's physiological state, providing reliable feature data for the final calculation of colorectal cancer risk probability, and effectively improving the accuracy and stability of the screening system.

[0036] In one embodiment, the third acquisition module 3 is specifically used for: The first calculation unit is used to execute step S31, specifically to calculate the average hue and saturation values ​​of the tongue body region and the tongue coating region in the HSV color space, and to obtain the hue and saturation parameters; wherein the hue value range of the tongue body region is 0-50 degrees, the hue value range of the tongue coating region is 50-120 degrees, and the saturation threshold is set to 20%; The second extraction unit is used to execute step S32, specifically to extract the gray-level co-occurrence matrix of the gray-level image of the tongue surface, and calculate three texture parameters of the matrix: contrast, energy, and entropy. The contrast reflects the clarity of the texture, the energy reflects the uniformity of the image, and the entropy reflects the complexity of the texture. The measurement unit is used to execute step S33, specifically to measure the serration index of the tongue edge contour and the depth ratio of the tongue surface crack, and to obtain morphological parameters. The serration index is calculated by the edge curvature change rate, and the depth ratio is determined by the ratio of the crack pixel area to the total area of ​​the tongue. The first processing unit is used to execute step S34, specifically to combine the hue saturation parameter, texture parameter and morphology parameter into a multi-dimensional feature vector in a fixed order, and to normalize the values ​​of each dimension so that all feature values ​​are within the range of 0-1.

[0037] As described above, each unit extracts the hue saturation distribution features, texture features, and morphological features of the tongue body and coating step by step from the preprocessed independent tongue image features. Then, the three types of features are combined and normalized according to fixed rules to finally generate a tongue image feature vector in the range of 0-1. The aim is to transform the image-based tongue image features into quantifiable and computable numerical features, laying the data foundation for subsequent fusion with the exhaled biomarker concentration vector and input into the dual-branch deep learning model. This ensures the accuracy and consistency of feature input during the colorectal cancer risk assessment process and improves the objectivity and standardization of the screening system.

[0038] While independent tongue image features have eliminated noise, color distortion, and background interference, they are still essentially image data and cannot be directly used by deep learning models for risk calculation. Feature extraction is necessary to convert the physiological information related to colorectal cancer risk in the image into numerical parameters. In traditional Chinese medicine theory and modern medical research, the color of the tongue body and coating (corresponding to hue saturation distribution characteristics), the texture of the tongue surface (corresponding to tongue image texture characteristics), and the shape of the tongue (corresponding to tongue image morphology characteristics) are closely related to the health status of the human digestive system. Colorectal cancer patients, due to abnormal intestinal metabolism and changes in blood and qi circulation, will exhibit specific differences in their tongue appearance, such as a darker tongue body, disordered tongue coating texture, and obvious serrations on the tongue edges. If these features cannot be accurately extracted and quantified into feature vectors, deep learning models will lack effective input data and will struggle to accurately calculate risk probabilities. Therefore, a systematic feature extraction and vector construction process is essential to transform the visual information of the tongue image into numerical information that the model can recognize, in order to establish a correlation between tongue image features and colorectal cancer risk.

[0039] The average hue and saturation values ​​of the tongue body region and the tongue coating region in the HSV color space are calculated to obtain hue and saturation parameters. The hue value range for the tongue body region is set to 0-50 degrees, and the hue value range for the tongue coating region is set to 50-120 degrees. The saturation threshold is set to 20%. The independent tongue image features required for this step are obtained from tongue image data with a segmentation accuracy of no less than 98% after U-Net denoising, Retinex color restoration, texture enhancement, and Mask R-CNN segmentation processing. During the extraction process, the independent tongue image features are first converted from the RGB color space to the HSV color space. The HSV color space can more intuitively separate the hue, saturation, and brightness information of colors, facilitating accurate extraction of color features. Next, the tongue body and tongue coating regions are divided according to the preset hue value range. Pixels with hue values ​​in the range of 0-50 degrees are identified as the tongue body region, and pixels in the range of 50-120 degrees are identified as the tongue coating region. This range division is based on statistical analysis of a large number of tongue image data from healthy individuals and colorectal cancer patients, which can accurately distinguish the color boundaries of the two types of regions. Then, the average hue value and average saturation value of all pixels in the two regions are calculated respectively, resulting in four hue and saturation parameters: average hue value of the tongue body, average saturation value of the tongue body, average hue value of the tongue coating, and average saturation value of the tongue coating. At the same time, pixels with saturation values ​​below 20% are identified as low-saturation regions (possibly due to a thin tongue coating or image shadows). These pixels are removed when calculating the average value to avoid interference from low-saturation pixels on the accuracy of features. By using partitioning extraction and threshold filtering, the color characteristics of the tongue body and tongue coating can be accurately captured. For example, colorectal cancer patients may have an average hue value of the tongue body that deviates from the 0-50 degree range (such as leaning towards 60 degrees, which is manifested as a yellowish tongue body) and an average saturation value of the tongue coating that is higher than that of healthy people (which is manifested as a thick and greasy tongue coating). This provides a quantitative basis for the color dimension for subsequent risk assessment.

[0040] The gray-level co-occurrence matrix of the gray-level image of the tongue surface is extracted, and three texture parameters of the matrix, namely contrast, energy and entropy, are calculated. Contrast reflects the clarity of the texture, energy reflects the uniformity of the image, and entropy reflects the complexity of the texture. This step is based on independent tongue image features. First, the independent tongue image features are converted from the RGB color space to the grayscale space. The grayscale value of each pixel is calculated using the weighted average formula (grayscale value = 0.299×R + 0.587×G + 0.114×B) to obtain the grayscale image of the tongue surface. Then, a grayscale co-occurrence matrix is ​​constructed. Four directions with a pixel spacing of 1 and angles of 0 degrees, 45 degrees, 90 degrees, and 135 degrees are selected, and the grayscale levels are compressed to 16 levels (balancing computational efficiency and feature accuracy). The grayscale co-occurrence matrices for the four directions are calculated separately, and the average of the four matrices is taken as the final grayscale co-occurrence matrix. This matrix reflects the spatial distribution pattern of tongue texture in different directions. Finally, three texture parameters are calculated based on this matrix: contrast is obtained by multiplying the square of the grayscale difference between all pixels in the matrix and the diagonal pixels by the sum of the corresponding probabilities. The calculation formula is as follows: ; Where C represents contrast. Represents pixel pairs in the gray-level co-occurrence matrix The higher the contrast value, the clearer the tongue texture (e.g., clear tongue papillae texture in healthy individuals indicates a higher contrast value); energy is obtained by calculating the sum of the squares of the probabilities of all pixels in the matrix, and its calculation formula is: ; Where E represents energy, and a higher energy value indicates a more uniform tongue texture (e.g., colorectal cancer patients may have a lower energy value due to disordered tongue coating); the entropy value is obtained by calculating the negative value of the sum of the products of the probabilities of all pixels in the matrix and their corresponding correspondences, and its calculation formula is: ; Here, Q represents the entropy value. The higher the entropy value, the more complex the tongue texture (e.g., an increase in tongue surface cracks leads to a higher entropy value). These three parameters can comprehensively quantify the clarity, uniformity, and complexity of the tongue texture, capturing subtle texture differences that are difficult for traditional systems to identify, and supplementing risk assessment with texture dimension feature data.

[0041] The serration index of the tongue's edge contour and the depth ratio of tongue surface cracks are measured to obtain morphological parameters. The serration index is calculated by the rate of change of edge curvature, and the depth ratio is determined by the ratio of crack pixel area to the total area of ​​the tongue. The input data for this step is still independent tongue image features. First, the tongue edge contour is extracted using the Canny edge detection algorithm (threshold set to 50-150), resulting in an edge curve composed of continuous pixels. For the calculation of the serration index, the edge curve is divided into several continuous line segments of 5 pixels in length. The curvature value of each line segment is calculated (curvature = 1 / radius, calculated using the coordinates of the two endpoints and the midpoint of the line segment). Then, the absolute value of the difference in curvature values ​​between adjacent line segments is calculated. The average of all absolute differences is defined as the serration index. The larger the serration index, the more irregular the tongue edge (e.g., in some colorectal cancer patients, this may be due to poor blood circulation). The tongue exhibits pronounced serrations along its edges, with a serration index higher than that of healthy individuals. To calculate the depth ratio of tongue surface cracks, a threshold segmentation method (with a grayscale threshold set to 30) is first used to segment the cracked areas (pixels with grayscale values ​​below 30) in the grayscale image of the tongue surface. Then, the total number of pixels in the cracked areas (i.e., crack pixel area) is counted, along with the total number of pixels in the tongue body region within the independent tongue image features (i.e., the total tongue body area). The ratio of the crack pixel area to the total tongue body area is defined as the depth ratio. A higher depth ratio indicates more severe tongue surface cracks (e.g., in colorectal cancer patients, abnormal fluid metabolism may lead to increased tongue surface cracks and a higher depth ratio). These two morphological parameters quantify the microscopic morphological characteristics of the tongue, compensating for the shortcomings of traditional macroscopic morphological measurements and providing crucial morphological data for risk assessment.

[0042] The hue and saturation parameters, texture parameters, and shape parameters are combined into a multi-dimensional feature vector in a fixed order, and the values ​​of each dimension are normalized so that all feature values ​​are in the range of 0-1. First, a fixed order for feature combination is determined: the nine parameters extracted in the previous steps—four hue / saturation parameters, three texture parameters, and two morphology parameters—are arranged sequentially in the order of average hue value of the tongue body, average saturation value of the tongue body, average hue value of the tongue coating, average saturation value of the tongue coating, contrast, energy, entropy, serration index, and depth percentage, forming an initial multidimensional feature vector. Then, the Min-Max normalization algorithm is used to normalize the values ​​of each dimension of the initial feature vector. Normalization can eliminate the influence of differences in the dimensions of features in different dimensions. For example, the original range of the average hue value of the tongue body is 0-50 degrees, while the original range of contrast may be 0-1000. After normalization, both are in the same numerical range, avoiding the model's oversensitivity to certain features due to differences in numerical range. The fixed order of combination ensures that the feature vector structure of different samples is consistent, which is convenient for subsequent fusion with the exhaled biomarker concentration vector and input into the dual-branch deep learning model for calculation. The final generated tongue image feature vector not only fully preserves the three key feature information of tongue image—color, texture, and shape—but also has standardized and computable characteristics. It can provide accurate and unified feature input for subsequent dual-modal fusion and risk probability calculation, effectively improving the stability of model training and inference as well as the accuracy of risk screening.

[0043] In one embodiment, the concentration acquisition module 4 is specifically used for: The detection unit is used to perform step S41, specifically to detect the gas concentrations of methane and ethane in the breath sample using tunable semiconductor laser absorption spectroscopy. The analysis unit is used to perform step S42, specifically to quantitatively analyze the concentration values ​​of volatile organic compounds such as acetone and isoprene using photoacoustic spectroscopy. The monitoring unit is used to perform step S43, specifically to periodically calibrate the detection instrument through the built-in standard gas calibration system and to monitor the system error in real time using the internal standard gas channel; The second processing unit is used to execute step S44, specifically to arrange the gas concentration values ​​in the order of methane, ethane, acetone, and isoprene to form a concentration vector, and to perform logarithmic transformation on the concentration values ​​to improve the data distribution characteristics.

[0044] As described above, each unit functions by employing specific spectral techniques to quantitatively detect four biomarkers—methane, ethane, acetone, and isoprene—in breath samples. Combined with instrument calibration and data processing, a standardized breath biomarker concentration vector is generated. This aims to accurately capture the abnormal concentration characteristics of volatile organic compounds in the breath of colorectal cancer patients, providing reliable physiological metabolic index data for subsequent fusion with tongue image feature vectors and calculations using a dual-branch deep learning model. This complements tongue image features, enhancing the comprehensiveness and accuracy of colorectal cancer risk screening.

[0045] The development of colorectal cancer leads to an imbalance in the gut microbiota and metabolic disorders, which are reflected in the components of exhaled breath. Changes in the gut microbiota affect short-chain fatty acid metabolism, causing alterations in the production of gases such as methane and ethane. Meanwhile, abnormal proliferation of tumor cells disrupts normal lipid and energy metabolism, causing the concentrations of volatile organic compounds (VOCs) such as acetone and isoprene to deviate from healthy levels. These changes in the concentration of exhaled biomarkers objectively reflect abnormal physiological states and can serve as important supplementary indicators for colorectal cancer risk assessment. However, the concentrations of biomarkers in exhaled samples are usually low (mostly at the ppb level) and easily affected by factors such as adsorption from the collection device and errors in the detection instrument. Without accurate quantitative detection and data correction, the biomarker concentration data will be distorted, failing to provide effective support for risk assessment. Therefore, it is essential to ensure the accuracy and stability of exhaled biomarker concentration detection results through highly sensitive detection technologies, rigorous instrument calibration mechanisms, and scientific data processing systems to transform exhaled metabolic information into quantitative data that can be used for risk screening.

[0046] Tunable semiconductor laser absorption spectroscopy (TSLAS) was used to detect the concentrations of methane and ethane in exhaled breath samples. The required breath samples were collected from subjects in a tongue-free state using a breath sampling device treated with an anti-adsorption coating. This coating prevents methane and ethane from adsorbing onto the inner wall of the sampling device, ensuring the accuracy of the sample composition and concentration. The core principle of Tunable semiconductor laser absorption spectroscopy is based on the absorption characteristics of gas molecules to specific wavelengths of laser light. By adjusting the laser wavelength to scan the absorption spectrum of the target gas, the relationship between absorption intensity and gas concentration is calculated. In terms of technical parameter settings, a characteristic absorption wavelength of 1.653 μm was selected for methane, and a characteristic absorption wavelength of 1.690 μm was selected for ethane. These two wavelengths correspond to the strong absorption peaks of the two gas molecules, which can maximize the detection sensitivity. The laser output power was set to 5 mW to ensure that the detection signal strength is met while avoiding the increase in background noise caused by excessive laser energy. The optical path length of the detection system was set to 10 cm. Combined with a lock-in amplifier (detection accuracy up to 1 ppb), the absorption signal was amplified and acquired, ultimately realizing the quantitative detection of methane and ethane concentrations. The detection limit for both can reach 0.1 ppb. It can accurately capture the concentration changes that may occur in colorectal cancer patients, such as increased methane concentration (increased methanogens in the gut) and decreased ethane concentration (abnormal anaerobic metabolism), providing quantitative evidence at the level of gut microbiota metabolism for risk assessment.

[0047] The concentrations of acetone and isoprene were quantitatively analyzed using photoacoustic spectroscopy. The breath samples used in this step were the same as those described above, collected by a breath sampling device with an anti-adsorption coating. The working principle of photoacoustic spectroscopy is that gas molecules absorb light energy of a specific wavelength, converting the energy into heat energy through non-radiative transitions. This causes periodic changes in gas pressure, generating a photoacoustic signal. The intensity of the photoacoustic signal is linearly related to the gas concentration, and concentration quantification can be achieved by detecting the photoacoustic signal. A mid-infrared laser wavelength of 2.7 μm was selected for acetone, and a mid-infrared laser wavelength of 3.3 μm was selected for isoprene. These two wavelengths match the molecular vibrational absorption bands of the two volatile organic compounds, effectively distinguishing the target gas from other interfering gases. The photoacoustic cell volume was set to 5 mL to ensure sufficient reaction of the exhaled sample within the cell, while shortening the detection time (single detection time controlled within 1 minute). A microphone (sensitivity up to 1 μPa) was used to collect the photoacoustic signal, and the signal was filtered and amplified by a signal conditioning circuit (bandwidth set to 10 Hz-1 kHz) to achieve quantitative detection of acetone and isoprene concentrations, with detection limits of 0.5 ppb and 0.3 ppb, respectively. This technology can accurately capture the characteristics that may be present in colorectal cancer patients, such as elevated acetone concentration (abnormal energy metabolism, increased lipolysis) and decreased isoprene concentration (disordered cellular lipid synthesis metabolism), supplementing metabolic risk assessment data and complementing the concentration data of methane and ethane.

[0048] The instrument is periodically calibrated using a built-in standard gas calibration system, and system errors are monitored in real time using an internal standard gas channel. The built-in standard gas calibration system contains a mixture of standard gases with known concentrations of methane (10 ppm), ethane (5 ppm), acetone (2 ppm), and isoprene (1 ppm). The calibration cycle is set to once every 24 hours. During calibration, the standard gases are introduced into the detection system via an automatic sample introduction device, and the detection signals of the four gases are collected separately. A linear regression model between the detection signal intensity and the standard concentration is established. The instrument's detection parameters (such as laser wavelength offset and signal amplification coefficient) are adjusted based on the model deviation to ensure that the instrument's detection accuracy remains within the acceptable range (error ≤ 5%). Simultaneously, an internal standard gas channel is set up in the detection system, selecting nitrogen as the internal standard gas (nitrogen has a stable content in exhaled breath and does not interfere with other detection components). During each exhaled breath sample test, nitrogen gas at a fixed concentration (78% volume fraction) is simultaneously introduced as an internal standard, and the detection signal intensity of the internal standard gas is monitored in real time. If the detection signal intensity of the internal standard gas deviates from the preset range (±3%), a systematic error is determined to exist in the detection system (such as contamination of optical components or changes in ambient air pressure). The instrument's self-calibration program is immediately triggered, adjusting the detection parameters until the internal standard signal returns to the normal range. Through a dual mechanism of periodic calibration and real-time error monitoring, detection errors caused by instrument drift, environmental interference, and other factors can be effectively eliminated, ensuring the accuracy and reliability of methane, ethane, acetone, and isoprene concentration detection results. This avoids the impact of distorted concentration data on subsequent feature fusion and risk assessment.

[0049] The gas concentration values ​​were arranged in the order of methane, ethane, acetone, and isoprene to form a concentration vector, and a logarithmic transformation was performed on the concentration values ​​to improve the data distribution characteristics. First, the methane, ethane, acetone, and isoprene concentration values ​​obtained in the previous steps were strictly arranged in a fixed order to form an initial 4-dimensional concentration vector. This fixed order ensures that the concentration vector structure is consistent across different subjects, facilitating subsequent feature alignment and fusion with tongue image feature vectors. Since breath biomarker concentration data are often skewed (e.g., concentrations are low in most healthy individuals and significantly elevated in a few patients), directly using the raw concentration values ​​could lead to high-concentration outliers excessively influencing feature extraction results in subsequent principal component analysis and other feature processing steps. Therefore, a natural logarithmic transformation was used to process the concentration values. This transformation converts skewed concentration data into near-normal data, reducing the impact of outliers and narrowing the numerical range of concentration values ​​(e.g., converting the concentration range of 1 ppb to 100 ppb into a logarithmic range of 0 to 4.6). This facilitates subsequent fusion calculations with the tongue image feature vector (feature values ​​in the 0-1 range). The resulting breath biomarker concentration vector not only fully preserves the concentration information of the four biomarkers but also possesses standardized data distribution characteristics, enabling effective fusion with the tongue image feature vector. This provides comprehensive bimodal feature input for the dual-branch deep learning model, improving the accuracy and stability of colorectal cancer risk screening.

[0050] In one embodiment, the fusion module 5 is specifically used for: The third processing unit is used to execute step S51, specifically to perform principal component analysis dimensionality reduction on the tongue image feature vector and the exhaled biomarker concentration vector, and retain the principal components whose cumulative contribution rate reaches the threshold (the threshold is 95%). The determination unit is used to execute step S52, specifically to calculate the correlation coefficient between two sets of features using the canonical correlation analysis algorithm, and to determine the feature projection direction based on the principle of maximum correlation. The second calculation unit is used to execute step S53, specifically to calculate the association weight between tongue image features and exhalation markers using a cross-modal attention mechanism, wherein the attention weight of tongue image features is learned through a multilayer perceptron network. The generation unit is used to perform step S54, specifically to perform weighted fusion of the two types of features according to the attention weights to generate a joint feature vector, the dimension of which is half of the sum of the dimensions of the original features.

[0051] As described above, each unit performs principal component analysis for dimensionality reduction, canonical correlation analysis for feature association, cross-modal attention mechanism weight calculation, and weighted fusion on the tongue image feature vector and the exhalation biomarker concentration vector in sequence. Finally, a joint feature vector with a dimension that is half the sum of the original feature dimensions is generated. This aims to fully integrate the complementary information of the tongue image and exhalation modalities, eliminate feature redundancy, and strengthen key risk-related features. This provides high-quality, highly correlated fused features for subsequent dual-branch deep learning models, thereby improving the accuracy of colorectal cancer risk assessment and the computational efficiency of the model.

[0052] The tongue image feature vector (containing 9 dimensions of parameters including hue saturation, texture, and morphology) reflects the physiological characteristics of the human body surface, while the exhaled biomarker concentration vector (containing 4 dimensions of parameters including methane, ethane, acetone, and isoprene) reflects the metabolic characteristics of the human body. These two types of features reveal the risk of colorectal cancer from different physiological levels and have potential correlations and complementarities (e.g., a thick, greasy tongue coating and elevated methane concentration both point to an imbalance in the gut microbiota). However, directly concatenating the two types of features presents problems. On the one hand, the high feature dimensionality (13 dimensions in total) increases the computational load of the model, leading to a decrease in inference speed, and may also introduce redundant features (e.g., some tongue texture parameters have no significant correlation with exhaled biomarker concentrations), interfering with the model's learning of key features. On the other hand, the physical meaning and numerical distribution of the two types of features differ significantly (tongue image feature values ​​are in the range of 0-1, while exhaled biomarker concentrations, after logarithmic transformation, range from 0-4.6). Direct concatenation can cause the model to become oversensitive to exhaled features with larger numerical ranges, affecting the rationality of feature weight allocation. Therefore, a scientific feature fusion process is necessary to reduce dimensionality, strengthen correlation, and balance weights while retaining key information, so as to organically combine the two types of modal features and give full play to the synergistic advantages of the two modalities.

[0053] Principal component analysis (PCA) was performed on the tongue image feature vector and the exhaled biomarker concentration vector to reduce dimensionality, retaining the principal components whose cumulative contribution rate reached a threshold (95%). The tongue image feature vector required for this step was derived from a 9-dimensional normalized vector (eigenvalues ​​in the range of 0-1) extracted from independent tongue image features, and the exhaled biomarker concentration vector was derived from a 4-dimensional vector (values ​​in the range of 0-4.6) after quantitative detection and logarithmic transformation. The core principle of PCA is to map high-dimensional features to a low-dimensional space through linear transformation. The newly generated principal components are linear combinations of the original features, and the principal components are uncorrelated with each other, thus eliminating feature redundancy while retaining key information. When processing tongue image feature vectors, the covariance matrix of the nine original features is first calculated. Eigenvalue decomposition yields the eigenvalues ​​and eigenvectors of the covariance matrix, which are then sorted in descending order of eigenvalue (the larger the eigenvalue, the more original information the corresponding principal component contains). The cumulative contribution rate is then calculated (cumulative contribution rate = sum of the first k eigenvalues ​​ / sum of all eigenvalues). The process stops when the cumulative contribution rate reaches 95%, and the number of principal components to retain is determined (usually 4-5). The original 9-dimensional tongue image feature vector is projected onto the space formed by these principal components to obtain the dimensionality-reduced tongue image principal component vector. Similarly, principal component analysis is performed on the 4-dimensional exhaled biomarker concentration vector, retaining principal components with a cumulative contribution rate of 95% (usually 2-3), resulting in the dimensionality-reduced exhaled biomarker principal component vector. The 95% cumulative contribution rate threshold is set based on a large number of sample validations. This ensures that the principal components after dimensionality reduction retain more than 95% of the information of the original features, avoiding the loss of key risk features (such as the principal components corresponding to features strongly associated with colorectal cancer, such as the proportion of tongue surface crack depth and isoprene concentration). It also effectively reduces the feature dimensionality, reduces the subsequent model computation (such as reducing the original 13-dimensional features to 6-8 dimensions), and improves the model inference speed.

[0054] The correlation coefficient between two sets of principal component features (tongue image principal component vector and expiratory principal component vector) is calculated using canonical correlation analysis (CDI), and the feature projection direction is determined based on the principle of maximizing correlation. The purpose of CDI is to uncover the potential linear association between two types of modal features and find the projection direction that maximizes the correlation between the two types of features, making the projected features more risk-discriminating. Specifically, the reduced-dimensional tongue image principal component vector (denoted as X, dimension p) and expiratory principal component vector (denoted as Y, dimension q) are used as inputs to construct a CDI model. Then, the covariance matrix (containing the covariance matrix Cov(X,X) of X, the covariance matrix Cov(Y,Y) of Y, and the cross-covariance matrix Cov(X,Y) of X and Y) is solved. The canonical correlation coefficient and canonical variables are then solved through generalized eigenvalue decomposition. The canonical correlation coefficient reflects the degree of correlation between corresponding canonical variables, and the canonical variables are linear combinations of X and Y (e.g., U = ...). V= (where a and b are projection coefficient vectors); sort by canonical correlation coefficients from largest to smallest, and select the projection directions corresponding to the top k canonical correlation coefficients (usually k=2-3, to ensure coverage of the main correlation information). These projection directions ensure that the projected canonical variables of the tongue image have the greatest correlation with the canonical variables of the exhalation. For example, if the combination of tongue coating saturation and tongue papilla contrast in the principal components of the tongue image has the highest correlation with the combination of methane concentration and acetone concentration in the principal components of the exhalation (canonical correlation coefficient 0.75), then this projection direction will be selected first. The features projected through this direction can simultaneously reflect the correlation information of the two modalities, strengthen the correlation of features directly related to the risk of colorectal cancer, such as gut microbiota imbalance and abnormal tongue coating, and provide a more relevant feature basis for subsequent weighted fusion.

[0055] A cross-modal attention mechanism is employed to calculate the association weights between tongue image features and expiratory biomarkers, where the attention weights for tongue image features are learned through a multilayer perceptron network. The core of this mechanism is to dynamically allocate weights based on the importance of the two types of features to colorectal cancer risk assessment, avoiding the limitations of fixed weights. The inputs for this step are the canonical tongue image variables (denoted as U) and the canonical expiratory biomarkers (denoted as V) projected from classical correlation analysis. First, U and V are concatenated to obtain a concatenated vector Z = [U; V] (with dimensions k×2 determined in the previous step). Then, Z is input into the multilayer perceptron network (network structure: input layer to hidden layer 1 (16 neurons, ReLU activation function) to hidden layer 2 (8 neurons, ReLU activation function) to output layer (2 neurons, Softmax activation function)). The two neurons in the output layer correspond to the tongue image feature weights, respectively. Weighting of expiratory features ,and + =1. The training of the multilayer perceptron network is based on labeled samples (samples with known colorectal cancer status). The network parameters are adjusted by backpropagation through minimizing the risk probability prediction error (using the cross-entropy loss function), enabling the network to learn the contribution patterns of two types of features in different samples: for example, for samples with significant abnormalities in tongue coating but small changes in the concentration of exhaled biomarkers, the network will output a higher value. (e.g., 0.65) and lower (e.g., 0.35); for samples with significantly abnormal expiratory biomarker concentrations but minimal changes in tongue appearance, the network will output a lower value. (e.g., 0.3) and higher (e.g., 0.7). This dynamic weight calculation ensures that the fusion features of each type of sample highlight the modal information that is more critical to risk assessment, avoiding the problem of weakening key features caused by fixed weights.

[0056] The two types of features are weighted and fused according to the attention weights to generate a joint feature vector. The dimension of this vector is half of the sum of the dimensions of the original features. First, it is clarified that the sum of the dimensions of the original features is the sum of the dimensions of the tongue image feature vector (9) and the dimensions of the exhaled biomarker concentration vector (4), which is 13. Therefore, the dimension of the joint feature vector is set to 6 (13 / 2 rounded down to ensure that the dimension is controlled within a reasonable range, taking into account both information integrity and computational efficiency). In the specific fusion process, the tongue image typical variable U (dimension p') and the exhaled biomarker typical variable V (dimension q') after projection of the classical correlation analysis are respectively coupled with the corresponding attention weights. , Multiply by the product to obtain the weighted tongue image features U'= ×U, weighted expiratory characteristic V'= ×V; then, features of U' and V' are concatenated to obtain an initial fusion vector (dimension p'+q'); finally, the initial fusion vector is mapped to a 6-dimensional space through linear projection (projection matrix dimension is (p'+q')×6) to obtain the final joint feature vector. This projection matrix is ​​constructed through principal component analysis to ensure that the 6-dimensional features after projection retain more than 95% of the information of the initial fusion vector, while the dimension is strictly controlled to be half of the sum of the original feature dimensions, effectively reducing the model input dimension and improving computational efficiency. For example, if the dimension of U is 4, the dimension of V is 3, and the dimension of the initial fusion vector is 7, the linear projection mapping to a 6-dimensional joint feature vector not only retains the key correlation information in U' and V' (such as the correlation feature of "weighted tongue coating saturation-weighted methane concentration"), but also reduces the dimension from 7 to 6, further optimizing the model's computational efficiency. The resulting joint feature vector combines information from both tongue appearance and exhalation metabolism features. It has reasonable dimensions and dynamically adapted weights, providing accurate and efficient feature input for subsequent dual-branch deep learning models and significantly improving the model's accuracy in identifying colorectal cancer risk.

[0057] In one embodiment, the fusion module 5 is further configured to: The input unit is used to execute step S55, specifically to simultaneously input the joint feature vector into the API-Net and TransFG branch networks. The API-Net branch uses a dilated convolutional structure to capture local detail features of the tongue image, while the TransFG branch uses a self-attention mechanism to learn global semantic associations. The enhancement unit is used to execute step S56, specifically to establish a feature interaction mechanism between the two branch networks, and to achieve mutual enhancement of local features and global features through the cross attention module; The adjustment unit is used to execute step S57, specifically to fuse the output features of the two branches and dynamically adjust the contribution ratio of local features and global features using a gating mechanism. The transformation unit is used to perform step S58, specifically to map the fused features into a single risk score through a fully connected layer, and to use the Sigmoid function to convert the score into a risk probability value in the range of 0-1.

[0058] As described above, each unit functions by inputting the joint feature vector into the dual-branch deep learning model. The API-Net branch captures local details of the tongue image, while the TransFG branch learns global semantic associations. Combining feature interaction mechanisms and gating fusion strategies, the model ultimately outputs risk probability values ​​in the range of 0-1 through fully connected layers and the Sigmoid function. This approach aims to fully leverage the advantages of dual-branch networks in extracting both subtle local features and globally related features, enhance feature representation capabilities, ensure the accuracy of colorectal cancer risk assessment, and provide quantitative basis for subsequent risk level classification and recommendation output.

[0059] While the joint feature vector (with a dimension half the sum of the original feature dimensions) integrates information from both tongue appearance and exhalation modalities, it contains both local details directly related to risk, such as tongue papillae morphology and tongue coating texture, and globally correlated features, such as the overall tongue color distribution and the combination of exhalation marker concentrations. A single network branch cannot accurately capture both types of features simultaneously: relying solely on local feature extraction easily overlooks cross-modal global correlation information such as thick, greasy tongue coating and elevated methane concentration; relying solely on global feature learning easily misses key local risk features such as minute cracks on the tongue surface and localized discoloration of the tongue. Furthermore, the risk contribution of local and global features differs across samples (e.g., local crack features are more significant in some samples, while global metabolic abnormalities are more prominent in others). Using a fixed ratio for feature fusion can lead to biased risk assessments for some samples. Therefore, a dual-branch network structure must be used to extract both types of features separately, combined with dynamic interaction and fusion mechanisms, to comprehensively and accurately mine the risk information in the joint features and ensure the accuracy of risk probability calculations.

[0060] The joint feature vector is simultaneously input into both the API-Net and TransFG branches. The API-Net branch uses a dilated convolutional structure to capture local details of the tongue image, while the TransFG branch uses a self-attention mechanism to learn global semantic associations. The joint feature vector required in this step is derived from the feature vector generated after dimensionality reduction by principal component analysis, canonical correlation analysis, and cross-modal attention weighted fusion. Its dimension is half the sum of the original tongue image feature vector (9-dimensional) and the exhaled biomarker concentration vector (4-dimensional), i.e., 6-dimensional. The API-Net branch's structural design is optimized for local detail extraction: After receiving a 6-dimensional joint feature vector at the input layer, the features are first augmented and non-linearly transformed using a 1×1 convolutional layer (16 output channels, ReLU activation function). Then, three dilated convolutional layers with dilation rates of 1, 2, and 3 are applied, all with a kernel size of 3×3 and 32 output channels. These convolutions with different dilation rates expand the receptive field while accurately capturing local features at different scales (e.g., dilation rate 1 captures details of the tongue papillae, and dilation rate 3 captures the boundaries of the tongue coating texture). Each dilated convolutional layer is followed by a batch normalization layer and a ReLU activation function to prevent gradient vanishing and enhance non-linear expressive power. Finally, a 2×2 max-pooling layer (stride 2) downsamples the features, outputting a local feature map with a dimension of 32×(feature map size / 2). This feature map highlights local risk information in the joint features (e.g., feature responses corresponding to tongue surface cracks, feature activations corresponding to local anomalies in acetone concentration). The TransFG branch's structural design is optimized for global semantic association learning: the input layer also receives a 6-dimensional joint feature vector, which is first transformed through a 1×1 convolutional layer (16 output channels, ReLU activation function); then a self-attention module is connected, which calculates the association weights between each position in the feature map and all other positions (weight calculation uses the ScaledDot-ProductAttention mechanism, with the scaling factor set to the square root of the feature dimension, i.e., ...). =4), mining the global association between tongue image and exhalation features in the joint features (such as the cross-modal association between tongue color features and methane concentration features, and the global response between tongue texture features and isoprene concentration features); connecting a feedforward network (containing 2 fully connected layers, 32 hidden layer neurons, ReLU activation function, and 16 output layer neurons) after the attention module to further transform the attention-weighted features; finally, transforming the feature map into a 16-dimensional global feature vector through a global average pooling layer, which can completely retain the global risk association information in the joint features.

[0061] A feature interaction mechanism is established between the two branch networks, and the mutual enhancement of local and global features is achieved through the cross attention module. The input to the cross-attention module is the local feature map (32×H×W, where H and W are the height and width of the feature map) output by API-Net and the global feature vector (16-dimensional) output by TransFG. First, the global feature vector is transformed into a global feature map with the same number of channels as the local feature map through a 1×1 convolutional layer (32 output channels). Then, the attention weights between the local and global feature maps are calculated: the local and global feature maps are multiplied by a matrix (the local feature map is flattened to 32×(H×W), the global feature map is flattened to 32×(H×W), and the product dimension is (H×W)×(H×W)). The result is normalized by the Softmax function to obtain the cross-attention weight matrix. Then, the local feature map is multiplied by the weight matrix to obtain the local feature map after global feature enhancement. At the same time, the global feature map is multiplied by the transpose of the weight matrix to obtain the global feature map after local feature enhancement. Finally, the enhanced local feature map and the global feature map are convolved by 1×1 (keeping the number of output channels 32 and 16 respectively) to complete the feature interaction. This mechanism enables local features to be integrated with global correlation information (such as adjusting the response by combining detailed features of the tongue papilla with the global distribution of methane concentration), while simultaneously supplementing local detailed information with global features (such as optimizing the weights by combining global correlation features of exhalation markers with local features of tongue surface cracks), significantly improving the representational ability of both types of features and avoiding evaluation bias caused by one-sided information from a single branch feature.

[0062] The output features of the two branches are fused, and a gating mechanism is used to dynamically adjust the contribution ratio of local and global features. First, global average pooling is performed on the enhanced local feature map to obtain a 32-dimensional local feature vector. The enhanced global feature vector (16-dimensional) is then passed through a 1×1 convolutional layer (32 output channels, ReLU activation function) to adjust its dimension to match that of the local feature vector. Subsequently, a gating mechanism is constructed: the local and global feature vectors are concatenated into a 64-dimensional vector, which is then input into a fully connected layer with 16 neurons (ReLU activation function), and finally passed through a fully connected layer with an output dimension of 2 (Sigmoid activation function) to output the gating weights. (Local feature weights) and (Global feature weights), and + =1; The gating weights are learned dynamically by minimizing the risk probability prediction error (cross-entropy loss function) during model training. For example, for samples with significant local features (tongue surface cracks), It will be adjusted to 0.6-0.7. The value was 0.3-0.4; for samples with significant global features (abnormal combination of exhaled biomarkers), It will be adjusted to 0.3-0.4. The value is 0.6-0.7; finally, the two types of feature vectors are weighted and fused according to the gating weight, that is, the fused feature vector = ×Local feature vector+ Multiplying the global feature vector yields a 32-dimensional fused feature vector. This vector can dynamically balance the contributions of local and global risk information based on sample features, ensuring the relevance and effectiveness of the fused features.

[0063] The fused features are mapped to a single risk score using a fully connected layer, and the sigmoid function is used to convert this score into a risk probability value in the range of 0-1. First, the 32-dimensional fused feature vector is input into the fully connected layer: the first fully connected layer contains 64 neurons (ReLU activation function) to perform non-linear transformation and dimensionality enhancement on the fused features; the second fully connected layer contains 32 neurons (ReLU activation function) to further optimize the feature representation; the third fully connected layer contains 1 neuron (linear activation function) to map the 32-dimensional features to a single risk score (numerical range -5 to 5). The parameters of the fully connected layer are optimized through training on labeled samples (known colorectal cancer disease status) to ensure that the score is highly correlated with the actual risk (scores for diseased samples approach 5, and scores for healthy samples approach -5). Then, the single risk score is input into the sigmoid function, the formula of which is: ; Here, V represents the risk probability value, and S represents the risk score. This function compresses the score to the range of 0-1 to obtain the risk probability value. For example, when the risk score is 1, the probability value is approximately 0.73 (corresponding to high risk); when the score is 0, the probability value is 0.5 (corresponding to medium risk); and when the score is -1, the probability value is approximately 0.27 (corresponding to low risk). This probability value can intuitively quantify the examinee's colorectal cancer risk level and accurately match the subsequent risk level classification standards (low risk < 0.3, medium risk 0.3-0.7, high risk ≥ 0.7), providing a reliable quantitative basis for the final risk level determination and recommendation output.

[0064] The present invention also discloses a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement a colorectal cancer risk screening system based on tongue image analysis.

[0065] The present invention also discloses a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, enables the operation of a colorectal cancer risk screening system based on tongue image analysis.

[0066] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in this application and in the embodiments can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual-speed SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0067] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, apparatus, article, or method that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, apparatus, article, or method. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, apparatus, article, or method that includes that element.

[0068] The above description is merely a preferred embodiment of the present invention and does not limit the scope of this application. Any equivalent results or equivalent process transformations made based on the content of the present invention's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the scope of protection of this application.

Claims

1. A colorectal cancer risk screening system based on tongue image analysis, characterized in that, include: The first acquisition module is used to acquire images of the tongue and exhalation samples in the tongue-free state; The second acquisition module is used to remove noise from the tongue image using a deep learning denoising network, obtain a corrected tongue image, enhance the color and texture details of the tongue body through an image enhancement algorithm, and accurately separate the tongue body from the background area based on an instance segmentation model to obtain independent tongue image features. The third acquisition module is used to extract the hue saturation distribution features, tongue texture features, and tongue morphology features of the tongue body and tongue coating based on the independent tongue image features, and to obtain the tongue image feature vector based on the hue saturation distribution features, tongue texture features, and tongue morphology features of the tongue body and tongue coating. The concentration acquisition module is used to perform quantitative analysis of exhaled biomarkers on the exhaled sample and obtain the exhaled biomarker concentration vector. The fusion module is used to fuse and generate joint features based on the tongue image feature vector and the exhaled biomarker concentration vector, and to establish a dual-branch deep learning model. The joint features are input into the dual-branch deep learning model. The API-Net branch extracts local detail features of the tongue image, and the TransFG branch learns global semantic associations of the tongue image. The outputs of the two branches are fused and the risk probability value is calculated. The output module is used to automatically classify low-risk, medium-risk, and high-risk levels based on the risk probability value and output corresponding suggestions.

2. The colorectal cancer risk screening system based on tongue image analysis according to claim 1, characterized in that, The second acquisition module is specifically used for: The signal-to-noise ratio (SNR) of the RGB channels of the original tongue image is calculated. When the SNR is detected to be lower than 40 dB, the U-Net denoising network is triggered to process the image. A noise distribution model is constructed by analyzing the deviation between the local pixel intensity and the mean, and a corrected tongue image with an SNR of not less than 50 dB is output. Extract the hue and saturation distribution histogram of the corrected tongue image. When the hue mean is detected to deviate from the preset range, perform color restoration processing and output a standard tongue image with a stable color temperature within the standard color temperature range. In the Lab color space, the contrast of the tongue area is quantized. When the contrast value is lower than the preset contrast value, texture enhancement is activated. The Gaussian kernel parameters are dynamically adjusted according to the local image variance. The enhanced image is input into the MaskR-CNN segmentation model, and the tiny holes are filled through morphological closing operations. The output is an independent tongue image feature with a segmentation accuracy no lower than the preset accuracy value.

3. The colorectal cancer risk screening system based on tongue image analysis according to claim 1, characterized in that, The third acquisition module is specifically used for: Calculate the average hue and saturation values ​​of the tongue body region and the tongue coating region in the HSV color space, and obtain the hue and saturation parameters; Extract the gray-level co-occurrence matrix of the gray-level image of the tongue surface, and calculate the three texture parameters of the matrix: contrast, energy, and entropy. The serration index of the tongue edge contour and the depth ratio of tongue surface cracks were measured, and morphological parameters were obtained. The hue saturation parameters, texture parameters, and shape parameters are combined into a multi-dimensional feature vector in a fixed order, and the values ​​of each dimension are normalized.

4. The colorectal cancer risk screening system based on tongue image analysis according to claim 1, characterized in that, The concentration acquisition module is specifically used for: The concentrations of methane and ethane in exhaled breath samples were detected using tunable semiconductor laser absorption spectroscopy. The concentration values ​​of volatile organic compounds such as acetone and isoprene were quantitatively analyzed using photoacoustic spectroscopy. The instrument is periodically calibrated using a built-in standard gas calibration system, and system errors are monitored in real time using an internal standard gas channel. The concentration values ​​of each gas are arranged in the order of methane, ethane, acetone, and isoprene to form a concentration vector, and the concentration values ​​are logarithmically transformed to improve the data distribution characteristics.

5. The colorectal cancer risk screening system based on tongue image analysis according to claim 1, characterized in that, The fusion module is specifically used for: Principal component analysis was performed to reduce the dimensionality of the tongue image feature vector and the exhaled biomarker concentration vector, and the principal components whose cumulative contribution rate reached the threshold were retained. The correlation coefficient between the two sets of features is calculated using the canonical correlation analysis algorithm, and the feature projection direction is determined based on the principle of maximum correlation. A cross-modal attention mechanism was used to calculate the association weights between tongue image features and expiratory markers, where the attention weights of tongue image features were learned through a multilayer perceptron network. The two types of features are weighted and fused according to the attention weights to generate a joint feature vector, the dimension of which is half of the sum of the dimensions of the original features.

6. The colorectal cancer risk screening system based on tongue image analysis according to claim 5, characterized in that, The output module is specifically used for: The joint feature vector is simultaneously input into two branches of the network: API-Net and TransFG. The API-Net branch uses a dilated convolutional structure to capture local details of the tongue image, while the TransFG branch uses a self-attention mechanism to learn global semantic associations. A feature interaction mechanism is established between the two branch networks, and the mutual enhancement of local and global features is achieved through the cross attention module; The output features of the two branches are merged, and a gating mechanism is used to dynamically adjust the contribution ratio of local features and global features; The fused features are mapped to a single risk score through a fully connected layer, and the score is converted into a risk probability value in the range of 0-1.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it enables the operation of the system according to any one of claims 1 to 6.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it enables the operation of the system according to any one of claims 1 to 6.

Images