Method for analyzing correlation between tongue image recognition features and tumor prognosis in traditional Chinese medicine

Abstract

The application discloses a traditional Chinese medicine tongue feature and tumor prognosis association analysis method based on image recognition, comprising: standardized acquisition and quality four-dimensional gating; color standardization and device domain alignment; multi-spectral constraint de-mixing and dyeing event elimination; tongue segmentation and artifact mask; extracting tongue color, moss thickness, and ecchymosis quantitative features; combining micro-questionnaire, image interference detection, and treatment sensitive window to perform interference gating, generate feature dimension level mask; multi-modal hierarchical fusion modeling outputs recurrence and progression risk score; Bayesian update based on group hierarchical prior realizes cold start individualized baseline; deviation degree is controlled by one-sided cumulative sum and false discovery rate to output yellow, orange, and red three-level early warning, baseline abnormality masking update; taking image review day as a hard anchor point for closed-loop calibration. The application fills the monitoring gap between two image reviews by using the non-invasive and high-frequency acquisition characteristics of tongue, and realizes individualized tumor recurrence and progression risk grading early warning.

Description

Technical Field

[0001] This invention relates to the fields of medical artificial intelligence and objectification of traditional Chinese medicine tongue diagnosis, and in particular to a method for analyzing the correlation between TCM tongue features and tumor prognosis based on image recognition. Background Technology

[0002] Patients with malignant tumors require regular monitoring during the follow-up period after radical surgery, radiotherapy, chemotherapy, or other systemic treatments to detect tumor recurrence or progression as early as possible. Currently, the core method of clinical follow-up is imaging re-examination (such as CT, MRI, and PET-CT), typically performed every few months. However, there is a monitoring window of several months between two imaging re-examinations: on the one hand, imaging examinations involve radiation exposure, are expensive, and are not suitable for frequent performance; on the other hand, hematological indicators such as tumor markers have limited sensitivity and specificity, making it difficult to provide reliable individualized risk indications during this window period.

[0003] Traditional Chinese medicine believes that the appearance of the tongue can reflect the body's Qi and blood, as well as the functional state of the internal organs. With the development of image recognition technology, the objective acquisition and quantitative analysis of tongue images have become increasingly mature: the Dice coefficient for tongue segmentation has exceeded 95%, and standardized methods for tongue image acquisition have also been publicly reported. In the prior art, patent application CN117173083A discloses a tumor prediction system, method, and application based on tongue images and tumor markers, which integrates tongue images with tumor markers for gastric cancer diagnosis and screening classification. This is a static classification for diagnosis / screening scenarios and does not involve longitudinal continuous monitoring of the same patient. Patent application CN112806964A discloses a tongue image analysis method and system, which uses standardized acquisition and quantification methods for ecchymosis and tongue coating. However, it stops at the quantitative extraction of tongue image features and does not establish a dynamic correlation between quantitative features and tumor prognosis. Patent application CN116530965A discloses a method and system for predicting the prognosis and survival of glioma patients based on multimodal imaging, which integrates multimodal MRI and clinical parameters to predict glioma survival. However, this method relies on low-frequency image modalities and cannot provide a high-frequency, non-invasive, continuous monitoring closed loop.

[0004] In summary, existing technologies have at least the following gaps: First, there is a lack of longitudinal individual dynamic baseline modeling methods for tongue appearance, and tongue appearance research mostly focuses on cross-sectional classification or inter-group trajectory analysis; Second, there is a lack of a complete technical path to transform dynamic changes in tongue appearance into early warning of tumor recurrence and progression risks, especially a lack of systematic gating methods for interfering events such as staining, tongue scraping, fever, and treatment sensitivity window; Third, there is a lack of a mechanism to use imaging review results as an anchor point to perform closed-loop calibration between the early warning model and the individual baseline. Summary of the Invention

[0005] This invention overcomes the shortcomings of existing technologies and provides a method for analyzing the correlation between TCM tongue features and tumor prognosis based on image recognition.

[0006] To achieve the above objectives, the technical solution adopted in this invention is: a method for analyzing the correlation between TCM tongue features and tumor prognosis based on image recognition, comprising the following steps:

[0007] S1. Embedded colorimetric card is used to collect tongue images, and gates and guides re-collection based on tongue posture, highlights, clarity and illumination uniformity.

[0008] S2. Solve the color correction matrix to map to standard viewing conditions and align across devices; frame rejection if residual exceeds threshold.

[0009] S3. Based on the exogenous pigment absorption spectrum library, constrained unmixing of multispectral tongue images is performed to separate the surface staining component and the volume reflectance component. Features are extracted from the volume reflectance component, and the abundance exceeds the threshold for labeling and staining to trigger resampling.

[0010] S4. Segment the tongue body, tongue coating, and ecchymosis, and generate an artifact mask to obtain an effective pixel image;

[0011] S5. Extract feature vectors of tongue color, coating thickness, coating area, and ecchymosis;

[0012] S6. Combine micro-questionnaires, image interference detection and treatment schedule sensitive window to generate dimension-level mask vectors. Masked features do not participate in scoring and are prohibited from being written to the baseline update queue.

[0013] S7. The effective feature subvectors are fused with tumor type, stage, and treatment plan to model and output a recurrence / progression risk score;

[0014] S8. Using the group stratified prior as the baseline prior, Bayesian updates are performed and the prior weights shrink as the number of valid samples shrinks, transitioning to the individual dynamic baseline.

[0015] S9. After deducting seasonal components and regularizing irregular sampling, calculate the deviation from its own baseline, and output yellow, orange and red three-level early warnings by controlling the one-sided cumulative sum and false detection rate. The baseline abnormal occlusion is updated, the early warning suspension period is frozen and the threshold is adjusted according to the false positive rate.

[0016] S10. Using the image review date as a hard anchor point for diverting early warning samples, when the image is negative, partially write back to the baseline and adjust the threshold. When the image is positive, mark the event window data as a precursor mode and enter it into incremental training, and never write it into the baseline.

[0017] In a preferred embodiment of the present invention, in step S2, the color correction matrix is ​​solved by root polynomial regression: the measured color values ​​of each color block of the colorimetric card are expanded into a root polynomial feature vector containing linear terms and radical cross terms, the standard chromaticity value of each color block under standard observation conditions is used as the regression target, the regression coefficient matrix is ​​solved by least squares, and the color values ​​of all pixels of the tongue image are mapped to the standard observation conditions through the root polynomial feature vector and the regression coefficient matrix.

[0018] The cross-device second-order statistical alignment uses the CORAL method to minimize the Frobenius norm difference between the feature covariance matrices of the source device domain and the target device domain.

[0019] The correction residual is the average color difference between the chromaticity values ​​of each color block on the colorimetric card after correction and the standard chromaticity value. When the average color difference exceeds the preset color difference threshold, the frame is rejected and a prompt to re-acquire is made.

[0020] In a preferred embodiment of the present invention, in step S3, the multispectral tongue image includes reflectance images of no less than 6 bands;

[0021] The exogenous pigment absorption spectrum library includes at least the absorption spectra of caffeine-tannin pigments, anthraquinone traditional Chinese medicine pigments, rifampin, and iron or bismuth preparations;

[0022] The constrained spectral unmixing satisfies the abundance non-negativity constraint and the abundance sum of one constraint. The abundance value of each pigment is solved pixel by pixel to obtain the abundance map corresponding to each pigment.

[0023] The separation of surface staining components and tissue volume reflectance components adopts a two-color reflectance model, which decomposes the pixel spectrum into a surface component related to surface reflectance and a volume reflectance component related to pigment absorption.

[0024] When the proportion of pixels with an abundance exceeding the abundance threshold in the abundance map of any pigment exceeds the preset area proportion of the tongue body, the sample is marked as a stained sample, and a resampling process after rinsing is triggered. Samples whose abundance still exceeds the threshold after resampling are removed.

[0025] In a preferred embodiment of the present invention, in step S4, the segmentation adopts a convolutional neural network with an encoder-decoder structure to output three probability channels: tongue body, tongue coating, and ecchymosis.

[0026] The artifact mask includes: a highlight mask generated based on a joint criterion of high brightness and low saturation, a pseudo-membrane mask generated based on a Candida pseudomembrane classifier, and a peeling region mask generated based on a moss peeling detector; the set of pixels with a probability exceeding the segmentation threshold in the tongue probability channel is taken, and the set of pixels corresponding to the highlight mask, pseudo-membrane mask and peeling region mask are subtracted from the set of pixels to obtain an effective pixel mask map. Subsequent feature extraction is only performed within the pixels covered by the effective pixel mask map.

[0027] In a preferred embodiment of the present invention, in step S5, the quantitative features include: the mean chromaticity, standard deviation chromaticity, chromaticity and hue angle of the tongue body region and the tongue coating region in the CIELAB color space, as well as the mean, variance, skewness and kurtosis moment statistics of the lightness histogram.

[0028] The tongue coating thickness distribution field is estimated pixel by pixel based on the tongue coating reflectivity, and the area ratio of the tongue coating to the tongue body area; the number of ecchymosis, the density per unit area, and the aggregation index characterizing the spatial distribution of ecchymosis; the above features are spliced ​​together in a fixed-dimensional order to form a time-point feature vector.

[0029] In a preferred embodiment of the present invention, in step S6, the structured micro-questionnaire includes at least: whether the patient scraped their tongue, whether they had a fever and their body temperature, and whether they had taken any new medications within 24 hours prior to the collection.

[0030] The image-end interference detector includes: a scratch detector based on directional texture features and tongue edge sharpness, a Candida pseudomembrane classifier, and a map tongue coating peeling detector; the treatment schedule sensitive window is from day 3 to day 14 after chemotherapy administration; when any interference path is triggered, a mask vector corresponding one-to-one with the dimension of the time point feature vector is generated, wherein the tongue coating interference only masks the tongue coating thickness feature dimension and the tongue coating area feature dimension while retaining the tongue color feature dimension, and the fever interference marks the entire sample as an interference sample;

[0031] Masked features are not included in the current risk score and are prohibited from being written to the baseline update queue. Re-collection is triggered when the level of interference exceeds the preset severity level.

[0032] In a preferred embodiment of the present invention, in step S7, the hierarchical fusion modeling includes: the deep network layer adopts a shared backbone network and a hybrid expert-gated tumor type branch head, and the gating network assigns weights to each expert network according to the tumor type and treatment plan input;

[0033] The statistical layer uses a mixed-effects model, with tumor type, TNM stage and treatment regimen as fixed effects, and individual intercept and individual time slope as random effects for partial pooling estimation.

[0034] The deep network layer obtains initial parameters through mask reconstruction of tongue image time series data without recurrence events by self-supervised pre-training. Then, it fine-tunes the lightweight prediction head using only labeled data. During the fine-tuning stage, Focal loss is used to alleviate class imbalance caused by the scarcity of recurrence event samples.

[0035] The combined deep network layer and statistical layer output a tumor recurrence or progression risk score.

[0036] In a preferred embodiment of the present invention, in step S8, the offline stage employs a model-independent meta-learning method to learn quickly adaptable model initialization parameters on multiple patient tasks in an existing patient cohort.

[0037] For new patients, the baseline distribution of neighboring patients under the stratification dimension is taken as the individual baseline prior of the new patient, using tumor type, TNM stage, age group and constitution tongue color classification as stratification dimensions.

[0038] The posterior parameters of the individual baseline are updated according to the Bayesian normal-inverse Wishart conjugate, and the prior weights are smoothly reduced by the shrinkage coefficient as the number of effective samples increases, so that the individual baseline smoothly transitions from the population prior to the pure individual baseline as the effective samples accumulate; the confidence interval of the deviation on the first day is explicitly widened according to the prior uncertainty.

[0039] In a preferred embodiment of the present invention, in step S9, the seasonal component is fitted with Fourier harmonic regression with a period of 12 months and pre-subtracted.

[0040] The feature sequences of irregular sampling points are mapped into equally spaced potential trajectories through the Ornstein-Uhlenbeck kernel Gaussian process. The confidence band during the sparse sampling period automatically widens as the distance from the observation point increases.

[0041] The deviation is a standardized channel-wise deviation value, which is converted into a one-sided cumulative sum statistic of the allowable drift parameter and then recursively calculated.

[0042] The p-value for multi-channel alarms is controlled using the Benjamini-Hochberg method to control the false detection rate; the three-level early warning rules are as follows:

[0043] When the false detection rate of a single channel becomes significantly lower after control, a yellow alert is issued and a re-sampling is recommended.

[0044] When the cumulative and statistical values ​​exceed the warning threshold, or when a yellow warning occurs at least twice consecutively, an orange warning is issued and outpatient evaluation is recommended; when the warning threshold is exceeded 2 to 3 times consecutively and confirmed by run-length, a red warning is issued and imaging verification is recommended.

[0045] The dynamic baseline is updated using an abnormal masking index weighted moving average with a half-life of 60 to 120 days. Only samples without interference masks and without pending warnings are given baseline update weights, and the warning threshold is fine-tuned in a closed loop based on the false positive rate confirmed in the past 12 months.

[0046] In a preferred embodiment of the present invention, in step S10, the image review date is a hard anchor point, and baseline update eligibility is granted to samples without warning in the interval before the anchor point.

[0047] The warning samples are triaged according to the image results:

[0048] When the image result is negative, the corresponding deviation segment is marked as a false positive, the baseline is written back with partial weight, and the warning threshold of the corresponding channel is increased.

[0049] When the imaging result is positive, the tongue image time series data of the 90-day window before the event is marked as a precursor pattern sample and stored in the incremental training buffer. It participates in the incremental update of the model through experience playback to prevent catastrophic forgetting, and this data is never written into the baseline.

[0050] Individual baselines are updated using Bayesian conjugate at anchor points, and individual baselines are frozen during periods when there are unconfirmed warnings.

[0051] This invention addresses the shortcomings of the prior art and has the following beneficial effects:

[0052] (1) By using quality four-dimensional gating, root polynomial color correction and device domain alignment, the consistency and comparability of tongue image color data under different devices and different lighting conditions are ensured, laying the data foundation for longitudinal comparison.

[0053] (2) Through multispectral constraint unmixing and dual-color reflectance decomposition, it is possible to effectively identify and remove exogenous staining caused by food pigments, Chinese medicine pigments and drugs, avoid staining events being misjudged as pathological changes in tongue appearance, and reduce the risk of false positives;

[0054] (3) By using feature dimension-level interference gating, interference events such as scraping, fever, and chemotherapy sensitive window only shield the affected feature dimensions instead of simply discarding the entire collection. This ensures data cleanliness while maximizing the retention of usable information and preventing interference samples from contaminating individual baselines.

[0055] (4) By self-supervised pre-training and Focal loss fine-tuning, the class imbalance problem caused by the scarcity of recurrence event samples is alleviated, so that a stable risk scoring model can still be trained under small sample label conditions; the partial pooling mechanism of the mixed effects model takes into account both group patterns and individual differences.

[0056] (5) By using meta-learning and Bayesian conjugate update cold start mechanism, new patients can obtain individual baselines with reasonable uncertainty quantification when there are only a few samples or even the first day of the sample. The baseline transitions smoothly with the accumulation of samples, avoiding over-alarms or under-alarms during the cold start period.

[0057] (6) Through a three-level early warning mechanism of individualized dynamic baseline, one-sided accumulation and recursion and false detection rate control, the comparison with the population norm is transformed into a comparison with its own history. It has high sensitivity to slow, small but continuous trend deviations, and at the same time, it suppresses false alarms caused by occasional noise through multi-level confirmation.

[0058] (7) Through image anchoring closed-loop calibration, the early warning results are fed back to the baseline and threshold after being confirmed by the gold standard of imaging, so that the system can continuously self-calibrate at the individual level and become more and more accurate with use. And through the experience playback mechanism, the group model can achieve incremental learning of the precursor pattern without catastrophic forgetting.

[0059] (8) This invention uses non-invasive, low-cost, and home-based high-frequency tongue images as a monitoring carrier to fill the monitoring gap between two imaging re-examinations. It helps to prompt the clinicians to initiate imaging verification in a timely manner during the intervention window period of tumor recurrence and progression, and has clear clinical application value. Attached Figure Description

[0060] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0061] Figure 1 This is a schematic diagram of the overall process of the method of the present invention. Detailed Implementation

[0062] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0063] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein. Therefore, the scope of protection of the invention is not limited to the specific embodiments disclosed below.

[0064] The technical problem this invention aims to solve is to provide a method for analyzing the correlation between TCM tongue features and tumor prognosis based on image recognition. Utilizing the non-invasive and high-frequency acquisition characteristics of tongue images, this method establishes a correlation between quantitative tongue features and the risk of tumor recurrence and progression. When an individual's tongue image deviates significantly from their dynamic baseline, a graded warning is output, prompting timely clinical intervention with imaging studies, thus filling the monitoring gap of several months between two imaging examinations. Simultaneously, this invention addresses technical problems in longitudinal tongue image monitoring, such as false deviations caused by exogenous staining interference, tongue scraping, fever, and the treatment sensitivity window; difficulties in cold-starting individual baselines for new patients; model imbalance due to scarce recurrence event samples; and the lack of image-anchored closed-loop calibration for warning results.

[0065] like Figure 1 As shown, the method for analyzing the correlation between TCM tongue features and tumor prognosis based on image recognition includes the following steps:

[0066] S1. Embedded colorimetric card is used to collect tongue images, and gates and guides re-collection based on tongue posture, highlights, clarity and illumination uniformity.

[0067] S2. Solve the color correction matrix to map to standard viewing conditions and align across devices; frame rejection if residual exceeds threshold.

[0068] S3. Based on the exogenous pigment absorption spectrum library, constrained unmixing of multispectral tongue images is performed to separate the surface staining component and the volume reflectance component. Features are extracted from the volume reflectance component, and the abundance exceeds the threshold for labeling and staining to trigger resampling.

[0069] S4. Segment the tongue body, tongue coating, and ecchymosis, and generate an artifact mask to obtain an effective pixel image;

[0070] S5. Extract feature vectors of tongue color, coating thickness, coating area, and ecchymosis;

[0071] S6. Combine micro-questionnaires, image interference detection and treatment schedule sensitive window to generate dimension-level mask vectors. Masked features do not participate in scoring and are prohibited from being written to the baseline update queue.

[0072] S7. The effective feature subvectors are fused with tumor type, stage, and treatment plan to model and output a recurrence / progression risk score;

[0073] S8. Using the group stratified prior as the baseline prior, Bayesian updates are performed and the prior weights shrink as the number of valid samples shrinks, transitioning to the individual dynamic baseline.

[0074] S9. After deducting seasonal components and regularizing irregular sampling, calculate the deviation from its own baseline, and output yellow, orange and red three-level early warnings by controlling the one-sided cumulative sum and false detection rate. The baseline abnormal occlusion is updated, the early warning suspension period is frozen and the threshold is adjusted according to the false positive rate.

[0075] S10. Using the image review date as a hard anchor point for diverting early warning samples, when the image is negative, partially write back to the baseline and adjust the threshold. When the image is positive, mark the event window data as a precursor mode and enter it into incremental training, and never write it into the baseline.

[0076] Specifically, it includes a tongue image acquisition frame that conforms to the face, with a miniature colorimetric card embedded in the inner edge of the frame. The colorimetric card contains no fewer than 6 color patches (preferably 6 to 24 color patches; in this embodiment, 12 color patches are used, covering the range of white, gray, black, red, yellow, green, blue, and skin tone). The colorimetric values ​​of the color patches are pre-calibrated under standard observation conditions. The light source can be a simulated D65 LED ring light source with a color temperature of 5500K to 6500K; in a home setting, natural light can also be used, and the colorimetric card calibration in step S2 can be used to eliminate differences in illumination.

[0077] During each data collection, the system calculates the following four quality indicators in parallel:

[0078] (a) Tongue key point pose index: A pre-trained key point detection network (such as a lightweight network based on heatmap regression) is used to locate the tip of the tongue, the bilateral edges of the tongue, and the V-shaped groove. The tongue extension length ratio (the ratio of the distance from the tip of the tongue to the V-shaped groove to the width of the mouth opening, which should be in the range of 0.8 to 1.5) and the tongue extension angle (the angle between the central axis of the tongue and the vertical axis of the image, which should not exceed 15° in absolute value) are calculated.

[0079] (b) Specular highlight index: The image is converted to HSV space, and specular highlights on the tongue surface are detected by a joint threshold of saturation channel S below 0.20~0.30 and lightness channel V above 0.90~0.95. The proportion of the highlight area reflects the degree of saliva coverage (moisture) on the tongue surface, and the proportion should not exceed 2% of the tongue area;

[0080] (c) Image sharpness index: Calculate the Laplacian variance of the grayscale image of the tongue region, which should not be lower than the preset sharpness threshold (100 in this embodiment, which can be adjusted between 50 and 200 according to the camera module calibration).

[0081] (d) Illumination uniformity index: Calculate the brightness variation coefficient within each color patch of the color chart, as well as the dispersion of the average brightness of color patches in different orientations. The variation coefficient is required to be no more than 0.05~0.10.

[0082] After normalizing the above four indicators, a weighted composite data collection quality score is obtained. (The four weights can be 0.30, 0.25, 0.20, and 0.25 respectively), when When the quality threshold is lower than the specified threshold (0.80 in this embodiment, which can be adjusted between 0.70 and 0.90), the system will prompt the user with specific non-compliance items through the interactive interface (such as "Please stick your tongue out a little more", "Please wipe the saliva off your tongue", "Please hold your phone steady"), guiding the user to collect the data again.

[0083] For images that meet quality standards, first locate each color block on the color chart and extract its measured color vector. ( To map colors from any device and under any lighting conditions to standard viewing conditions, this invention employs root polynomial regression to solve the color correction matrix. The color vector... Extended Mapping of Root Polynomials Expanding to an eigenvector containing linear terms and radical cross terms, taking a second-order radical polynomial as an example:

[0084] ;

[0085] The correction mapping is:

[0086] ;

[0087] in: For pixels Corrected color vector; This is the color correction coefficient matrix. The number of terms after the expansion of the root polynomial (in the second order) Third order ); For pixels The root polynomial extension vector of the measured color vector. Solving using least-squares regression of color patches from a color chart: ,in , The first The measured color vector and standard chromaticity vector of each color patch The total number of color blocks. In some embodiments, a Tikhonov regularization term may be introduced to prevent overfitting.

[0088] After completing single-frame correction, the CORAL (Correlation Alignment) method is further used for device domain alignment: taking the feature distribution collected by the calibration equipment in the central laboratory as the target domain, minimizing the Frobenius norm difference between the source device domain feature covariance matrix and the target domain feature covariance matrix, obtaining the whitening-recoloring linear transformation, and aligning the data of different mobile phone models and different acquisition terminals to a unified feature domain.

[0089] Finally, calculate the corrected residuals: ,in For CIEDE 2000 color difference, For the corrected first The chromaticity vector of each color block. When Exceeding the color difference threshold (In this embodiment, 2.5 is used, which can be adjusted between 2.0 and 3.5). When the frame is determined to have failed color correction, the frame is rejected and a prompt is made to check whether the colorimeter is obscured before re-acquiring.

[0090] The tongue surface is easily stained by exogenous pigments such as coffee, tea, traditional Chinese medicine (e.g., rhubarb and senna containing anthraquinones), rifampin, iron supplements, and bismuth preparations. Failure to remove these pigments will cause false deviations in tongue color. This step employs a combination of multispectral imaging and constrained linear spectral unmixing. The acquisition end is configured with no fewer than six narrowband filter bands in the illumination optical path (6-16 bands in this embodiment, with center wavelengths of 450, 500, 530, 570, 620, and 660 nm as examples). In a downgraded embodiment with only a standard RGB camera, a pre-trained staining classifier combined with the S6 micro-questionnaire can be used for approximate discrimination, resulting in slightly reduced accuracy but maintaining workflow compatibility.

[0091] A pre-established exogenous pigment absorption spectrum library was constructed: under a standard light source, the reflectance spectra of dyes such as caffeine-tannin pigments, anthraquinone traditional Chinese medicine pigments, rifampin, and iron / bismuth preparations were measured after being applied to a tongue model. These spectra, along with the background spectrum of the tongue tissue, constituted the spectral library matrix. .

[0092] For each pixel Solve the constrained linear spectral unmixing problem:

[0093] ;

[0094] in: For pixels The multispectral reflectance vector; For the spectral library matrix; Let be the endmember abundance vector to be determined. For the first The abundance of each endmember (pigment or background tissue) at that pixel; The solution involves unmixing the residual vectors; the constraints are a non-negativity abundance constraint and a one-to-one abundance sum constraint. This problem can be efficiently solved using the Fully Constrained Least Squares (FCLS) algorithm to obtain the abundance maps of each pigment.

[0095] Furthermore, to distinguish between staining, salivary membrane, and the natural color of the tissue attached to the tongue surface, a two-color reflectance model was used to decompose the pixel spectrum into:

[0096] ;

[0097] in: , which is the surface component coefficient (corresponding to the reflection of interfaces such as dyed films and salivary films); This is a spectral shape function of surface reflection (approximately independent of wavelength); For volume reflection component coefficients; This is the spectral shape function of volume reflectance (carrying tissue and pigment absorption information). Only the volume reflectance component is considered. The surface component is discarded and sent to subsequent feature extraction.

[0098] Staining determination rule: When the abundance diagram of any exogenous pigment satisfies... (In this embodiment, 0.25 is used, but it can be adjusted between 0.15 and 0.40.) The pixel area accounts for more than [a certain percentage] of the tongue area. (In this embodiment, 10% is used, which can be adjusted between 5% and 20%). When the sample is stained, the system prompts the patient to rinse their mouth with water and then re-collect the sample after 10 to 20 minutes. Samples that still exceed the threshold after re-collection are removed and do not proceed to feature extraction and baseline update.

[0099] A convolutional neural network with an encoder-decoder structure (such as U-Net, DeepLabv3+, or the lightweight MobileUNet) is used to segment the corrected image. The network outputs three probability channels: tongue body, tongue coating, and ecchymosis. On a standard tongue image dataset, the Dice coefficient for tongue body segmentation can reach over 95%. The segmentation threshold is set to 0.5, which can be adjusted between 0.4 and 0.6.

[0100] Simultaneously generate three types of artifact masks:

[0101] (a) Specular mask: Generated using the specular detection criterion of S1;

[0102] (b) Candida pseudomembrane mask: The white patch area on the tongue is determined by a pre-trained pseudomembrane binary classifier, and the pixels in the pseudomembrane area are set as masks;

[0103] (c) Cotton Skin Peeling Mask: The tongue-like peeling area on the map is identified by the tongue skin peeling detector. The set of pixels with a tongue body probability exceeding the segmentation threshold is subtracted from the set of pixels of the three types of artifact masks to obtain the effective pixel mask map. Subsequent feature statistics for S5 will only be available on [date / time]. The process is carried out internally to avoid contamination characteristics in non-physiological areas such as highlights, false films, and peeling.

[0104] In effective pixel mask The following quantitative features are extracted and concatenated in a fixed-dimensional order to form a time-point feature vector. (This embodiment) (Choose from 32 to 128 dimensions, example is 64 dimensions)

[0105] (a) Tongue color characteristics: The tongue region in the CIELAB color space Mean and standard deviation, chroma Color angle;

[0106] (b) Tongue coating color characteristics: Tongue coating area is the same as the above statistical measure;

[0107] (c) Brightness distribution characteristics: four moment statistics of the tongue body brightness histogram: mean, variance, skewness, and kurtosis;

[0108] (d) Moss thickness characteristics: Based on the spectral and color reflectance characteristics of the moss area, the moss thickness distribution field is estimated pixel by pixel by a regression model, and its mean, standard deviation and spatial gradient statistics are extracted.

[0109] (e) Area ratio of tongue coating: The ratio of the area of ​​the tongue coating region to the area of ​​the tongue body region;

[0110] (f) Characteristics of ecchymosis: the number of connected domains in the ecchymosis channel, the density per unit area (number / cm²), and the Moran's I index or nearest neighbor distance statistic, which characterizes the degree of spatial clustering.

[0111] The above features are all continuous numerical features designed for comparison with the individual's own historical data, forming a point-in-time feature vector. Along with the collection timestamp and quality score The device identifier is written into the individual tongue image time series database.

[0112] Tongue images are sensitive to physiological, behavioral, and therapeutic interference. This step uses a three-pronged approach—questionnaire, image, and calendar data—to generate a feature-dimensional mask vector, enabling fine-grained interference control.

[0113] (a) Structured micro-questionnaire: Each collection session includes a structured micro-questionnaire with no more than 5 questions, including at least: whether the patient scraped or brushed their skin in the 24 hours prior to collection; whether they had a fever and their current body temperature; whether they had taken any new medications (select commonly used medications from the drop-down menu and support OCR recognition of the pillbox); and whether they had eaten foods that are easily stained (for S3 verification).

[0114] (b) Image-end interference detectors: including scratch detectors (extracting directional texture features, such as the directional consistency of Gabor filter response, and combining the moss edge sharpness mutation criterion to identify mechanical scraping marks), Candida pseudomembrane classifier (shared with S4), and map tongue coating peeling detector;

[0115] (c) Treatment schedule sensitive window: It is connected with the hospital chemotherapy calendar and automatically marks the period from day 3 to day 14 after chemotherapy administration (D+3~D+14) as the sensitive window for bone marrow suppression and mucosal reaction. Samples within the window have increased interference weight by default.

[0116] The gated output is a feature-dimensional mask vector, and its operation is defined as follows:

[0117] ;

[0118] in: For time point The original time-point feature vector; Let be the mask vector, and its th... Each component Indicates the first One feature dimension is masked. Indicates reservation; This represents element-wise multiplication; This is the effective feature vector after masking. The masked dimension (positions with a value of 0) does not participate in the current risk score and is treated as a missing value in serialized storage, and is prohibited from being written to the baseline update queue.

[0119] The preferred masking rules are as follows: When the tongue scraping or scratch detector is triggered, only the tongue thickness feature dimension and the tongue area feature dimension are set to 0, while dimensions such as tongue color and ecchymosis are retained; when fever (body temperature ≥37.5℃) is triggered, the entire sample is marked as an interfering sample, and the mask vector is set to 0; when a new drug in the spectral library is taken, the S3 spectral verification is triggered first, and if the verification confirms staining, the S3 process is followed; the tongue texture dimension of samples within the chemotherapy sensitivity window is downweighted or set to 0. When multiple paths are triggered simultaneously, or when the image quality is severely impaired due to interference, it is judged as severe interference, and a re-acquisition process is triggered.

[0120] This step fuses effective feature vectors with clinical information to output a tumor recurrence and progression risk score. The model consists of a deep network layer and a statistical layer connected in parallel. The weighted fusion of the outputs of the two layers (the fusion weights are calibrated on the validation set; in the example, it is 0.6:0.4) is taken as the final risk score.

[0121] Deep network layers: Employ a "shared backbone + hybrid expert (MoE) gated tumor seeding head" structure. The shared backbone is a temporal masked autoencoder (temporal MAE), with input being temporal segments of the patient's tongue image features; the gated network takes the embedding vectors of tumor type and treatment plan as input and outputs pairs of... A network of experts (in this embodiment) We use a weighting of 4-8, employing top-2 sparse activation, with each expert network corresponding to a prediction head for different tumor types. Training is divided into two phases: the first phase is self-supervised pre-training, utilizing massive amounts of event-free tongue image time series, randomly masking features from several time steps and reconstructing them to obtain initial parameters; this phase requires no labels. The second phase only fine-tunes the lightweight prediction head using labeled data, employing Focal loss to address the imbalance problem caused by the scarcity of recurrence events.

[0122] ;

[0123] in: This represents the model's predicted probability of the true class of the sample. The balancing factor is set to 0.25 in this embodiment, but can be adjusted between 0.1 and 0.75. The focus factor is set to 2 in this embodiment, but can be adjusted between 1 and 5. It is the natural logarithm.

[0124] Statistical layer: Using a mixed-effects model to characterize group patterns and individual differences:

[0125] ;

[0126] in: For the first Individuals at a given time point The risk-related response value (which may be a latent variable that deviates from the characteristic); For the first Individual clinical covariate vectors (tumor type, TNM stage, and treatment regimen coding); This is a vector of fixed effects coefficients; , The first The random intercepts and random time slopes of each individual follow a joint normal distribution with a mean of zero, thus achieving partial pooling across individuals; This represents the residual term. Partial pooling automatically shrinks individuals with small sample sizes towards the population mean, improving robustness in small sample phases.

[0127] In the initial stage of new patient enrollment, the sample size is small, making it difficult to establish individual baselines directly. This step employs a two-stage approach: offline meta-learning combined with online Bayesian shrinkage.

[0128] In the existing patient cohort, each patient is treated as a task, and model-independent meta-learning (MAML) is used to train the model initialization parameters that can be quickly adapted, so that the model can be adapted to new patients with a small number of gradient steps.

[0129] Online phase: The new patient was retrieved from the existing cohort based on tumor type, TNM stage, age group, and tongue color classification (e.g., pale red tongue, red tongue, dark purple tongue, pale white tongue) as stratified dimensions. Neighboring patients ( (Taking values ​​from 20 to 50), the mean and covariance of the stable-period baseline distribution of neighboring patients are used as the individual baseline prior for the new patient. Individual feature channel sub-vectors The baseline distribution modeling (channel-by-channel or block-by-block modeling) is a normal-inverse Wishart (NIW) model, with the prior being... Incorporation 100 valid samples (mean vector) Scatter matrix After that, the posterior parameters are updated according to the conjugate relation:

[0130] ;

[0131] ;

[0132] in: The prior mean vector; This represents the number of prior equivalent samples. The prior scale matrix; These are the a priori degrees of freedom; The number of valid samples; for The mean vector of each valid sample; It is a scatter matrix; These are the corresponding posterior parameters.

[0133] As can be seen from the above formula, the prior weights are calculated according to... With the number of valid samples Smooth contraction is equivalent to... The weighting transitions towards a purely individual baseline, and the contraction coefficient... Take 8~20 (10 in this example). On the first day, due to... If the variance of the posterior predicted distribution is extremely small, the variance will be significantly larger. The deviation confidence interval will be explicitly widened according to the prior uncertainty (for example, the confidence band width on the first day will be 2 to 3 times that of the stable period) to avoid false alarms during the cold start period.

[0134] This step is the core of early warning decision-making and includes six stages: seasonal correction, irregular sampling normalization, deviation calculation, accumulation and recursion, multiple verification control and baseline update.

[0135] (a) Seasonal correction: Tongue appearance exhibits seasonal physiological fluctuations (e.g., yellowish and greasy tongue coating in summer, pale tongue color in winter). Fourier harmonic regression is established with a 12-month cycle and pre-subtracted:

[0136] ;

[0137] in: For time point Seasonal component estimates; The seasonal cycle is 365 days. The harmonic order is 1 to 2; For the first The cosine and sine coefficients of the first harmonic are obtained by hierarchical least squares fitting of group data and individual historical data; Based on consecutive days within the year, each characteristic channel, after deducting the corresponding seasonal component, proceeds to the next stage.

[0138] (b) Regularization of Irregular Sampling: Patients collected samples at home at irregular intervals (ranging from 1 to 7 days). An Ornstein-Uhlenbeck (OU) kernel Gaussian process was used to map the unequal-interval observations to equally-interval latent trajectories, with the following covariance function:

[0139] ;

[0140] in: For a moment and Covariance of latent eigenvalues; The standard deviation of the signal; The time scale is 14-45 days (in this embodiment, it is 14-45 days, and hyperparameters are optimized separately for each channel). The standard deviation of the observed noise; For the Kronecker function, The value is 1 if the condition is met, and 0 otherwise. The OU kernel corresponds to a first-order autoregressive continuous process with Markov properties, which is consistent with the physiological assumption of slow regression of tongue image features. During the sampling sparsity period, the posterior variance of the Gaussian process automatically increases with the distance from the nearest observation point, and the confidence band widens accordingly, avoiding overconfidence during the sparsity period.

[0141] (c) Deviation calculation: for each feature channel Standardized deviation is ,in The first effective feature after masking Quantity, , These are the dynamic baselines. The mean and standard deviation of the channel.

[0142] (d) Cumulative summation: To sensitively capture persistent, small-amplitude unidirectional drifts, the deviation is converted into a one-sided cumulative sum statistic with an allowable drift parameter:

[0143] ;

[0144] in: For time point CUSUM statistic; For time point Standardized deviation of a certain channel (recursively calculated independently for each channel, channel subscripts omitted here); The allowable drift parameter is set to 0.5~1.0 (corresponding to a tolerance of 0.5~1.0 times the standard deviation of physiological fluctuation). The warning threshold is initially set to 4-8 and fine-tuned using the closed-loop adjustment method S10. When... The channel is determined to have accumulated deviations exceeding the threshold.

[0145] (e) Multi-channel false detection rate control: simultaneously controlling the false detection rate of multiple channels. Analyzing alarms across multiple channels introduces multiple comparison problems. The Benjamini-Hochberg (BH) method is used to control the false detection rate: the p-values ​​of each channel are sorted in ascending order. Find the maximum rank that satisfies the conditions.

[0146] ;

[0147] in: The total number of characteristic channels participating in alarm determination; After sorting in ascending order, the first The p-value of the bit; The false detection rate control level is set at 0.05~0.10; The maximum rank required to satisfy the BH critical condition. The rank must not exceed [a certain value]. All channels were determined to be statistically significant and were included in the early warning classification.

[0148] (f) Three-level early warning rules: Yellow warning, a single channel shows significant changes after BH control, only prompting the patient to repeat the test once according to the S1 standard to confirm; Orange warning, any channel's CUSUM statistic exceeds the threshold ( If a yellow alert is triggered (e.g., 2 consecutive yellow alerts or 3 consecutive yellow alerts), outpatient evaluation is recommended. If a red alert is triggered (e.g., 2-3 consecutive red alerts exceeding the threshold and confirmed by run-length), imaging verification recommendations will be sent to the patient and follow-up physician. The grading thresholds can be adjusted according to the tumor type and risk level during deployment.

[0149] (g) Baseline update and feedback threshold adjustment: The dynamic baseline is updated using an exponentially weighted moving average (EWMA) with anomalous occlusion.

[0150] ;

[0151] in: For time point The dynamic baseline mean vector; The effective feature vector after masking; Update the coefficients for EWMA; The time interval (in days) between the last valid update; The half-life is set at 60-120 days to ensure the baseline can slowly track physiological drift without being swayed by short-term fluctuations. The anomaly masking rule is: only when the sample has no interference mask ( 1) No suspension warning and all channels ( Only when the baseline value is between 2.5 and 3.5 is taken will the sample be entitled to baseline update; the baseline will be frozen during the warning suspension period.

[0152] In addition, the system sets the warning threshold based on the false positive rate confirmed by image anchors over the past 12 months. Perform closed-loop fine-tuning (adjust upwards when the false positive rate is higher than the upper limit of the target interval). Conversely, if the sensitivity is lowered within the allowable range, the target false positive rate range is 5%~15%.

[0153] The imaging re-examination result is the sole hard gold standard for this system. This step uses the imaging re-examination date as the hard anchor point to achieve closed-loop calibration of the monitoring system. The process includes the following:

[0154] (a) Anchor point supplementation: For samples that did not receive an early warning within the interval before the anchor point and did not obtain the right to update at that time due to other rules, the baseline update qualification is supplemented after verification to confirm that there is no interference, and the baseline sample pool is retrospectively enriched.

[0155] (b) Early warning sample diversion: When the image result is negative, the corresponding deviation segment is marked as a false positive, and the baseline is written back with a partial weight (0.3~0.5 in this embodiment), and the early warning threshold of the corresponding channel is increased. (Example upward adjustment range is 10%~20%) to reduce the probability of false alarms due to similar deviations; when the imaging result is positive (diagnosed recurrence or progression), the tongue image time series data of the 90-day anterior window of the event is marked as a precursor pattern sample and stored in the incremental training buffer. After being mixed with existing representative samples through experience replay, it participates in the incremental training of the S7 model to prevent catastrophic amnesia; this precursor segment data is never written into the individual baseline to avoid pathological deviations contaminating the baseline;

[0156] (c) Anchor Bayesian update: The individual baseline is updated once at the anchor point, and the valid samples confirmed before the anchor point are formally included in the posterior.

[0157] (d) Freeze rule: During the period when there is an unconfirmed warning (the warning has been issued but has not yet been confirmed by imaging or outpatient clinic), the individual baseline will be frozen and will be unfrozen or diverted according to the above rules after the anchor point is confirmed.

[0158] Through the closed loop of S10, the system achieves a self-calibration cycle of early warning, verification, feedback and threshold adjustment at the individual level, and achieves continuous accumulation of precursor patterns and model iteration at the group level.

[0159] Example: Full-process monitoring of patients after gastric adenocarcinoma surgery.

[0160] Patient Information and Record Establishment (D0): The patient is a 62-year-old male who underwent radical subtotal gastrectomy for stage IIIA gastric adenocarcinoma and received adjuvant chemotherapy (SOX regimen, 21 days per cycle). D0 Record Establishment: Tumor type (gastric adenocarcinoma), pathological grade (poorly differentiated), TNM stage (stage IIIA), and treatment regimen were entered; the patient's constitution and tongue color were classified as pale red with a slightly darker tinge. The system retrieved 30 neighboring patients with the same tumor type × stage × age 60-65 × similar tongue color classification from the existing cohort using S8, constructing an individual baseline prior. ,Pick Simultaneously, load and quickly adaptable model initialization trained with MAML. The confidence interval for the first day is widened by 2.5 times the width of the stable period.

[0161] Baseline accumulation period (D1~D60): Tongue images were collected from patients at home daily or every other day, resulting in a total of 28 qualified samples (quality score). During chemotherapy cycle 1, five samples taken within the sensitive window (D+3 to D+14) had their tongue coating thickness and area dimensions automatically masked by S6, with only tongue color contributing to the scoring and not included in the baseline. Four other samples were not included in the baseline valid samples due to interference from fever (one sample was masked) and failure to meet the criteria after resampling with highlight / posture gating (three samples). The remaining samples underwent S2 color correction (residual correction of each frame). All samples were less than 2.5. After S3 unmixing and verification (no pigment abundance exceeding the threshold), the samples were normally entered into the database. Up to D60, the number of valid samples... Prior weights Individual baselines are now primarily dominated by individual data.

[0162] Staining Rejection Event (D47): The patient self-medicated with a traditional Chinese medicine preparation containing rhubarb (an anthraquinone component) due to constipation. The tongue sample collected on D47 showed a noticeably yellowish tinge. S3 multispectral unmixing revealed an anthraquinone pigment abundance of 0.42 in the central region of the tongue (exceeding the abundance threshold of 0.25), and the over-threshold pixels accounted for approximately 31% of the tongue area (exceeding the area proportion threshold of 10%). The system marked this as a stained sample and suggested rinsing the mouth with water for 15 minutes before re-collecting. The re-collected sample still had an abundance of 0.35, and this sample was rejected, not included in the scoring, and not entered into the baseline. On D48, medication was discontinued and a second sample was collected; the abundance dropped to 0.08, and the sample returned to the normal procedure. Without the S3 step, the tongue color of this sample would have been... Value will be generated A false deviation is enough to trigger a false alarm.

[0163] Orange alert and negative imaging correction (D85~D120): From D85 onwards, the patient's tongue color... The channel (yellowness) showed a continuous increase: the standardized deviations of the three samples D85, D87, and D89 were respectively .Pick Recursion: , , D89 The initial threshold has not yet been exceeded. However, in the same sampling, the moss thickness channel was controlled by BH ( If the p-value is significant and there was a previous yellow alert for D87, and there are two consecutive yellow alerts, the alert will be upgraded to an orange alert according to the S9(f) rule. Outpatient evaluation is recommended.

[0164] On day 93, outpatient physical examination and blood tests revealed no clear signs of recurrence; the alert remained suspended, and the baseline was frozen according to rule S9(g). On day 120, a follow-up imaging examination (hard anchor point) was performed as planned. Enhanced CT scans of the chest, abdomen, and pelvis showed no signs of recurrence or metastasis; the alert was confirmed as a false positive. The system marked this deviation segment as a false positive, rewrote the baseline with a weight of 0.4, and... Warning thresholds for channels and moss-covered channels The baseline update value was increased by 15% from 5.0 to 5.75; samples that were denied update rights twice within the interval before the anchor point due to the sensitive window were granted baseline update eligibility after review; the baseline was unfrozen after Bayesian conjugate update was performed at the anchor point.

[0165] Stable monitoring period (D121~D195): Patients resumed collection every other day, with a total of 31 valid samples collected from various channels. The CUSUM statistic repeatedly returned to zero, and the EWMA baseline was updated smoothly with a half-life of 90 days, without triggering any level of warning.

[0166] Red Alert and Confirmed Cases (D196~D210): From D196 onwards, ecchymosis density channels and tongue brightness are monitored. The channel showed a continuous deterioration in the same direction: ecchymosis density from four sampling times (D196, D201, D205, and D208). The values ​​are 2.2, 2.6, 3.0, and 3.3, respectively, corresponding to CUSUM recursive values ​​of 1.7, 3.8, 6.3, and 9.1. ).

[0167] D205 First time exceeding the threshold; D208 The threshold was exceeded for the second consecutive time; D210 was sampled once more. , The cumulative value continued to reach 12.2, exceeding the threshold for the third consecutive time. The run-length was confirmed, and the system issued a red alert and immediately pushed out imaging verification suggestions. An enhanced CT scan was performed on D213, which confirmed liver metastasis (a single nodule in the S6 segment of the liver, approximately 2.1 cm in diameter). The patient was subsequently transferred to second-line treatment.

[0168] According to rule S10: Data from the event window D196 to D210 (within the 90-day window) are marked as "precursor patterns" and enter the incremental training buffer pool. Data from D123 onwards within the 90-day event window are included in the precursor pattern buffer pool after channel significance screening and participate in the incremental model update through empirical replay. This data segment is never written into the baseline. The patient's baseline is reconstructed according to the new protocol in the new treatment cycle.

[0169] This embodiment demonstrates that the present invention can capture persistent tongue image deviation and issue a red warning at D210 (approximately 30 days earlier than the routine image re-examination) during the window period between two image re-examinations (D120 and the planned D240), thus providing a window of time for clinical intervention. At the same time, the elimination of the D47 staining event by S3 and the threshold reversal of the D89 false positive warning by S10 respectively verify the effectiveness of the interference gating and closed-loop calibration mechanisms.

[0170] In some embodiments, the following equivalent substitutions can be made without departing from the concept of the invention:

[0171] (1) The multispectral imaging of S3 can be downgraded to an RGB three-channel scheme with a pre-trained staining classifier, which is suitable for pure mobile phone scenarios without multispectral hardware;

[0172] (2) The root polynomial regression of S2 can be replaced by third-order polynomial regression or the lookup table method (3DLUT) for color correction;

[0173] (3) The temporal MAE of S7 can be replaced by contrastive learning or autoregressive self-supervised pre-training;

[0174] (4) The OU kernel Gaussian process of S9 can be replaced with a Matérn-1 / 2 kernel or a state-space model (such as a local level model); the EWMA baseline can be replaced with a Kalman filter baseline;

[0175] (5) S8’s MAML can be replaced by first-order meta-learning algorithms such as Reptile;

[0176] (6) The three-level early warning can be expanded to four levels or simplified to two levels. The threshold and confirmation rules can be configured differently according to the tumor type and follow-up plan.

[0177] This invention can be deployed on smartphones, portable tongue image acquisition terminals, or tongue diagnosis instruments in medical institutions. It can be used in conjunction with cloud or edge inference services and is suitable for home-based continuous monitoring of cancer patients during the follow-up period. It can be seamlessly integrated into existing cancer follow-up management systems and has good industrial applicability and promotional value.

[0178] Finally, to maintain consistency in mathematical expression throughout the entire text, the following notation is agreed upon, and the same symbol shall have a unique meaning in all formulas throughout the text:

[0179] Pixel coordinates in the image. The set of pixels covered by the effective pixel mask. : pixel The measured RGB color vectors These are the values ​​for the red, green, and blue channels, respectively. : Root polynomial extension mapping. : Root polynomial color correction coefficient matrix. : The corrected color vector. Color chart The measured color vectors of each color block. : No. The standard chromaticity vector of each color patch under the standard viewing conditions of D65 / 2°. Total number of color blocks on the colorimeter card. Color correction residual (mean color difference). Color difference threshold. Spectral band index , This represents the total number of bands. : pixel The multispectral reflectance vector. : Spectral library matrix, whose column vectors are composed of The spectrum consists of end-member spectra (exogenous pigment spectra and tongue tissue background spectra). : pixel endmember abundance vector, For its first Each component. : Unmixed residual vector. , Pixels in a two-color reflection model Surface component coefficients and volume reflection component coefficients. : Spectral shape function of surface (interface) reflection. : Spectral shape function of volume reflectance. Pigment abundance threshold. : Threshold for the proportion of staining area. : Index of discrete acquisition time points. : The time interval between two consecutive data collections (unit: days). Time point The original time-point feature vector, For feature dimensions. Time point The feature dimension level mask vector. : The effective feature vector after masking. Element-wise multiplication. , Time point The mean vector and standard deviation vector of an individual dynamic baseline. Time point No. Standardized deviation of each feature channel. EWMA update coefficients. EWMA half-life (unit: days). Abnormal occlusion deviates from the threshold. The mean, equivalent sample size, scaling matrix, and degrees of freedom hyperparameters of the normal-inverse Wishart prior. (Subscript) Indicates inclusion Posterior parameters after a number of valid samples. : The mean vector of each valid sample. : The scatter matrix of the valid samples. Prior contraction coefficient (with the same dimensions as the prior equivalent sample size, taking the same value as the prior equivalent sample size, i.e.) ). : Any two sampling times. : Covariance function of Gaussian process. Signal standard deviation. Time length scale. Standard deviation of observation noise. : Kronecker function. : One-sided cumulative sum (CUSUM) statistic. Allowed drift parameters. : Cumulative and warning thresholds. The total number of feature channels involved in alarm determination. : after ascending order The p-value of the bit. False detection rate control level. Seasonal cycle (365 days). Fourier harmonic order. : No. First harmonic coefficient. : The model's predicted probability of the true class. Focal loss balance factor. Focal loss focusing factor. : No. The vector of clinical covariates for each individual. : Fixed effects coefficient vector. : No. The random intercept and random time slope of each individual. : Residual term. : No. Individual time point Risk-related response values.

[0180] Note: Letters In the shape function of two-color reflection Fourier harmonic coefficients Mixed effects and random effects The three locations are distinguished by their respective subscripts. After correcting the residual The two entries, "expert network number" and "number of experts", are distinguished by their context. Different subscripts indicate different residual terms.

[0181] Based on the preferred embodiments of the present invention described above, those skilled in the art can make various changes and modifications without departing from the inventive concept. The technical scope of this invention is not limited to the contents of the specification, but must be determined according to the scope of the claims.

Claims

1. A method for analyzing the correlation between TCM tongue features and tumor prognosis based on image recognition, characterized in that, Includes the following steps: S1. Embedded colorimetric card is used to collect tongue images, and gates and guides re-collection based on tongue posture, highlights, clarity and illumination uniformity. S2. Solve the color correction matrix to map to standard viewing conditions and align across devices; frame rejection if residual exceeds threshold. S3. Based on the exogenous pigment absorption spectrum library, constrained unmixing of multispectral tongue images is performed to separate the surface staining component and the volume reflectance component. Features are extracted from the volume reflectance component, and the abundance exceeds the threshold for labeling and staining to trigger resampling. S4. Segment the tongue body, tongue coating, and ecchymosis, and generate an artifact mask to obtain an effective pixel image; S5. Extract feature vectors of tongue color, coating thickness, coating area, and ecchymosis; S6. Combine micro-questionnaires, image interference detection and treatment schedule sensitive window to generate dimension-level mask vectors. Masked features do not participate in scoring and are prohibited from being written to the baseline update queue. S7. The effective feature subvectors are fused with tumor type, stage, and treatment plan to model and output a recurrence / progression risk score; S8. Using the group stratified prior as the baseline prior, Bayesian updates are performed and the prior weights shrink as the number of valid samples shrinks, transitioning to the individual dynamic baseline. S9. After deducting seasonal components and regularizing irregular sampling, calculate the deviation from its own baseline, and output yellow, orange and red three-level early warnings by controlling the one-sided cumulative sum and false detection rate. The baseline abnormal occlusion is updated, the early warning suspension period is frozen and the threshold is adjusted according to the false positive rate. S10. Using the image review date as a hard anchor point for diverting early warning samples, when the image is negative, partially write back to the baseline and adjust the threshold. When the image is positive, mark the event window data as a precursor mode and enter it into incremental training, and never write it into the baseline.

2. The method for analyzing the correlation between TCM tongue features and tumor prognosis based on image recognition according to claim 1, characterized in that: In step S2, the color correction matrix is ​​solved by root polynomial regression: the measured color values ​​of each color block of the color chart are expanded into a root polynomial feature vector containing linear terms and radical cross terms. The standard chromaticity value of each color block under standard observation conditions is used as the regression target. The regression coefficient matrix is ​​solved by least squares. The color values ​​of all pixels in the tongue image are mapped to the standard observation conditions through the root polynomial feature vector and the regression coefficient matrix. The cross-device second-order statistical alignment uses the CORAL method to minimize the Frobenius norm difference between the feature covariance matrices of the source device domain and the target device domain. The correction residual is the average color difference between the chromaticity values ​​of each color block on the colorimetric card after correction and the standard chromaticity value. When the average color difference exceeds the preset color difference threshold, the frame is rejected and a prompt to re-acquire is made.

3. The method for analyzing the correlation between TCM tongue features and tumor prognosis based on image recognition according to claim 1, characterized in that: In step S3, the multispectral tongue image includes reflectance images of no less than 6 bands; The exogenous pigment absorption spectrum library includes at least the absorption spectra of caffeine-tannin pigments, anthraquinone traditional Chinese medicine pigments, rifampin, and iron or bismuth preparations; The constrained spectral unmixing satisfies the abundance non-negativity constraint and the abundance sum of one constraint. The abundance value of each pigment is solved pixel by pixel to obtain the abundance map corresponding to each pigment. The separation of surface staining components and tissue volume reflectance components adopts a two-color reflectance model, which decomposes the pixel spectrum into a surface component related to surface reflectance and a volume reflectance component related to pigment absorption. When the proportion of pixels with an abundance exceeding the abundance threshold in the abundance map of any pigment exceeds the preset area proportion of the tongue body, the sample is marked as a stained sample, and a resampling process after rinsing is triggered. Samples whose abundance still exceeds the threshold after resampling are removed.

4. The method for analyzing the correlation between TCM tongue features and tumor prognosis based on image recognition according to claim 1, characterized in that: In step S4, the segmentation uses a convolutional neural network with an encoder-decoder structure to output three probability channels: tongue body, tongue coating, and ecchymosis. The artifact mask includes: a highlight mask generated based on a joint criterion of high brightness and low saturation, a pseudo-membrane mask generated based on a Candida pseudomembrane classifier, and a peeling region mask generated based on a moss peeling detector; the set of pixels with a probability exceeding the segmentation threshold in the tongue probability channel is taken, and the set of pixels corresponding to the highlight mask, pseudo-membrane mask and peeling region mask are subtracted from the set of pixels to obtain an effective pixel mask map. Subsequent feature extraction is only performed within the pixels covered by the effective pixel mask map.

5. The method for analyzing the correlation between TCM tongue features and tumor prognosis based on image recognition according to claim 1, characterized in that: In step S5, the quantitative features include: the mean chromaticity, standard deviation chromaticity, chromaticity and hue angle of the tongue body region and the tongue coating region in the CIELAB color space, as well as the mean, variance, skewness and kurtosis moment statistics of the lightness histogram. The tongue coating thickness distribution field is estimated pixel by pixel based on the tongue coating reflectivity, and the area ratio of the tongue coating to the tongue body area; the number of ecchymosis, the density per unit area, and the aggregation index characterizing the spatial distribution of ecchymosis; the above features are spliced ​​together in a fixed-dimensional order to form a time-point feature vector.

6. The method for analyzing the correlation between TCM tongue features and tumor prognosis based on image recognition according to claim 1, characterized in that: In step S6, the structured micro-questionnaire includes at least: whether the patient scraped their tongue, had a fever and body temperature, and whether they had taken any new medications within 24 hours prior to the data collection. The image-end interference detector includes: a scratch detector based on directional texture features and tongue edge sharpness, a Candida pseudomembrane classifier, and a map tongue coating peeling detector; the treatment schedule sensitive window is from day 3 to day 14 after chemotherapy administration; when any interference path is triggered, a mask vector corresponding one-to-one with the dimension of the time point feature vector is generated, wherein the tongue coating interference only masks the tongue coating thickness feature dimension and the tongue coating area feature dimension while retaining the tongue color feature dimension, and the fever interference marks the entire sample as an interference sample; Masked features are not included in the current risk score and are prohibited from being written to the baseline update queue. Re-collection is triggered when the interference level exceeds the preset severity level.

7. The method for analyzing the correlation between TCM tongue features and tumor prognosis based on image recognition according to claim 1, characterized in that: In step S7, the hierarchical fusion modeling includes: the deep network layer adopts a shared backbone network and a hybrid expert-gated tumor type branch head, and the gating network assigns weights to each expert network according to the tumor type and treatment plan input; The statistical layer uses a mixed-effects model, with tumor type, TNM stage and treatment regimen as fixed effects, and individual intercept and individual time slope as random effects for partial pooling estimation. The deep network layer obtains initial parameters through mask reconstruction of tongue image time series data without recurrence events by self-supervised pre-training. Then, it fine-tunes the lightweight prediction head using only labeled data. During the fine-tuning stage, Focal loss is used to alleviate class imbalance caused by the scarcity of recurrence event samples. The combined deep network layer and statistical layer output a tumor recurrence or progression risk score.

8. The method for analyzing the correlation between TCM tongue features and tumor prognosis based on image recognition according to claim 1, characterized in that: In step S8, the offline phase employs a model-independent meta-learning method to learn quickly adaptable model initialization parameters on multiple patient tasks in the existing patient cohort. For new patients, the baseline distribution of neighboring patients under the stratification dimension is taken as the individual baseline prior of the new patient, using tumor type, TNM stage, age group and constitution tongue color classification as stratification dimensions. The posterior parameters of the individual baseline are updated according to the Bayesian normal-inverse Wishart conjugate, and the prior weights are smoothly reduced by the shrinkage coefficient as the number of effective samples increases, so that the individual baseline smoothly transitions from the population prior to the pure individual baseline as the effective samples accumulate. The confidence interval for the deviation on the first day is explicitly widened according to the prior uncertainty.

9. The method for analyzing the correlation between TCM tongue features and tumor prognosis based on image recognition according to claim 1, characterized in that: In step S9, the seasonal components are fitted with Fourier harmonic regression with a 12-month period and pre-subtracted. The feature sequences of irregular sampling points are mapped into equally spaced potential trajectories through the Ornstein-Uhlenbeck kernel Gaussian process. The confidence band during the sparse sampling period automatically widens as the distance from the observation point increases. The deviation is a standardized channel-wise deviation value, which is converted into a one-sided cumulative sum statistic of the allowable drift parameter and then recursively calculated. The p-value for multi-channel alarms is controlled using the Benjamini-Hochberg method to control the false detection rate; the three-level early warning rules are as follows: When the false detection rate of a single channel becomes significantly lower after control, a yellow alert is issued and a re-sampling is recommended. When the cumulative and statistical values ​​exceed the warning threshold, or when a yellow warning occurs at least twice consecutively, an orange warning is issued and outpatient evaluation is recommended; after exceeding the warning threshold 2 to 3 times consecutively and confirmed by run-length, a red warning is issued and imaging verification is recommended. The dynamic baseline is updated using an abnormal masking index weighted moving average with a half-life of 60 to 120 days. Only samples without interference masks and without suspended warnings are given baseline update weights, and the warning threshold is fine-tuned in a closed loop based on the confirmed false positive rate over the past 12 months.

10. The method for analyzing the correlation between TCM tongue features and tumor prognosis based on image recognition according to claim 9, characterized in that: In step S10, the image review date is a hard anchor point, and baseline update eligibility is granted to samples without warning in the interval before the anchor point. The warning samples are triaged according to the image results: When the image result is negative, the corresponding deviation segment is marked as a false positive, the baseline is written back with partial weight, and the warning threshold of the corresponding channel is increased. When the imaging result is positive, the tongue image time series data of the 90-day window before the event is marked as a precursor pattern sample and stored in the incremental training buffer. It participates in the incremental update of the model through experience playback to prevent catastrophic forgetting, and this data is never written into the baseline. Individual baselines are updated using Bayesian conjugate at anchor points, and individual baselines are frozen during periods when there are unconfirmed warnings.

Images