Chinese medicine tongue diagnosis auxiliary method and system based on image reconstruction and tongue image detection

Abstract

The application provides a tongue image detection method and system based on a double-task feedback learning framework, uses super resolution as a pre-step of tongue image detection, and adopts alternating training, so that the tongue image detection network can process high-quality images, thereby significantly improving detection accuracy. The specific steps include: performing preprocessing such as downsampling on tongue image data to construct super resolution model training data; performing super resolution reconstruction; performing image-level data enhancement based on the reconstructed image to obtain a fusion image; extracting tongue image features and performing feature-level data enhancement; decoding the fusion features to realize tongue image detection; and performing alternating training according to a double-task feedback mode. According to the prior art, the application first applies super resolution technology to tongue image detection, and the super resolution image generated by the method significantly improves the tongue image detection performance.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of medical image processing and artificial intelligence, and specifically designs a traditional Chinese medicine tongue diagnosis auxiliary method and system based on image reconstruction and tongue image detection. BACKGROUND

[0002] Traditional Chinese medicine tongue diagnosis is an important part of the diagnosis system. By observing the color, shape and tongue fur of the tongue, the state of the patient's zang-fu function, blood and body fluid can be reflected. With the advantages of non-invasiveness and convenience, tongue diagnosis is widely used in daily health monitoring and chronic disease management. However, traditional tongue diagnosis relies on the subjective experience of doctors, and there are individual differences, which affects the objectivity and accuracy of diagnosis. Therefore, the development of tongue image detection system based on image analysis is crucial to improve the scientificity and consistency of tongue diagnosis. Tongue image detection can automatically identify the thickness, color change and tongue shape abnormalities of tongue fur, providing a basis for doctors' diagnosis. Thick and greasy tongue fur may indicate internal dampness, and pale tongue color may indicate blood deficiency. Accurate tongue image detection can help doctors detect health problems early and achieve preventive medicine.

[0003] In recent years, the application of artificial intelligence (AI) in tongue image detection has promoted the modernization development of traditional Chinese medicine tongue diagnosis. As an important basis for reflecting the internal health status of the body, tongue diagnosis traditionally relies on the experience of doctors, and lacks standardized and quantitative basis. The introduction of AI can automatically analyze the color, shape and texture of the tongue through image processing and deep learning algorithms, greatly improving the accuracy and efficiency of diagnosis. Studies have shown that tongue image detection systems based on convolutional neural networks (CNN) and transformer models have achieved accurate classification of diseases. Combined with big data and multi-modal analysis, AI can integrate tongue image and other physiological indicators to significantly improve the accuracy of disease prediction and promote the standardization and intelligent development of traditional Chinese medicine diagnosis and treatment.

[0004] At the same time, the development and application of image reconstruction technology in tongue diagnosis have greatly improved the accuracy and clarity of tongue image analysis. Traditional tongue images may be affected by factors such as light and angle, affecting the diagnosis results. In recent years, deep learning-based image reconstruction methods, especially super-resolution reconstruction (SR), have made significant progress in restoring and enhancing the details of tongue images. These technologies restore the high-frequency details of tongue images, making the texture and color of the tongue more clear, providing a more reliable data basis for subsequent AI model analysis. In addition, combined with advanced methods such as generative adversarial networks (GAN), researchers can further improve the quality of tongue image, making it closer to the real clinical observation results. These developments not only improve the accuracy of tongue diagnosis, but also promote the application of AI in traditional Chinese medicine tongue diagnosis.

[0005] Although AI technology has shown great potential in tongue image detection and image reconstruction, promoting the development of intelligent and modernization of traditional Chinese medicine, there are still many challenges in this field. First, the tongue image detection data is scarce and the annotation quality is not high, which limits the generalization ability of the model. Second, the image acquisition quality is unstable and is easily affected by factors such as device performance and lighting conditions, resulting in inaccurate detection results. In addition, current research usually carries out tongue image detection and image reconstruction tasks independently, which fails to fully utilize the feature information extracted in detection, resulting in poor reconstruction effect and affecting the accuracy and effectiveness of diagnosis. SUMMARY

[0006] In view of the deficiencies of the prior art, the present application proposes a traditional Chinese medicine tongue diagnosis auxiliary method and system based on image reconstruction and tongue image detection. The tongue image detection accuracy is improved by super-resolution reconstruction, and the image reconstruction quality is optimized by tongue image detection, to solve the problem of image blur caused by poor imaging conditions during tongue image detection in traditional Chinese medicine tongue diagnosis. The system uses a dual-task feedback learning framework to improve the objectivity and accuracy of traditional Chinese medicine tongue diagnosis.

[0007] The traditional Chinese medicine tongue diagnosis auxiliary method based on image reconstruction and tongue image detection disclosed in the present application comprises the following steps:

[0008] Collecting tongue image true value images and processing to obtain tongue image low resolution images;

[0009] Constructing a tongue image detection learning model: performing super-resolution reconstruction on the tongue image low resolution images; performing image-level enhancement processing on the reconstructed images; combining the tongue image true value images, the reconstructed images and the enhanced images, performing feature-level enhancement processing to obtain fused high-dimensional tongue image feature maps; performing tongue image detection on the high-dimensional tongue image feature maps to obtain the type of tongue image;

[0010] Training the tongue image detection learning model, introducing feature alignment loss between super-resolution reconstruction and tongue image detection, and transmitting specific task features extracted by tongue image detection back to the super-resolution reconstruction process to adjust the super-resolution reconstruction network parameters.

[0011] Further, the tongue image is down-sampled by bicubic interpolation to obtain the corresponding low resolution tongue image, and the tongue image and the corresponding low resolution tongue image are used as training data pairs.

[0012] Further, the low resolution tongue image is input into a convolutional neural network and a Transformer network architecture to gradually extract spatial and contextual texture and fine structure information of the image and represent it in a high-dimensional feature space; a multi-layer neural perception network decoder is used for up-sampling restoration to obtain the reconstructed super-resolution image.

[0013] Further, the image-level enhancement processing specifically includes taking the super-resolution reconstructed image and the tongue image ground truth image as inputs, introducing a binary matrix with the same size as the image as a random mask to control the image fusion manner, then the super-resolution reconstructed image and the corresponding pixels of the tongue image ground truth image are element-wise added according to the binary mask to obtain the fusion image of the image-level enhancement processing .

[0014] Further, the feature-level data enhancement processing includes taking the input tongue image ground truth image , the super-resolution reconstructed image obtained in S2 , and the corresponding fusion image obtained in S3 are extracted by the same feature extraction encoder in the tongue image detection model to obtain the high-dimensional features of the ground truth image , the super-resolution image , and the fusion image , and the high-dimensional features are directly element-wise combined to obtain the fused high-dimensional tongue image features.

[0015] Preferably, the tongue image detection model is composed of the RPN network structure in FasterR-CNN, the ROI pooling layer, and the fully connected layer for classification and regression.

[0016] Further, the loss function in the super-resolution reconstruction is:

[0017]

[0018] wherein N represents the total number of pixels in the image, i.e. , and respectively represent the value of the i-th pixel in the tongue image low-resolution image and the ground truth image.

[0019] Further, the loss of the tongue image detection learning model is:

[0020]

[0021] wherein represents the network model connection, represents the image splicing. represents the super-resolution reconstruction loss, respectively represents the input low-resolution tongue image and tongue image ground truth image, and the corresponding task label pairing, refers to the high-dimensional tongue image feature map generated after the image-level enhancement, represent trainable parameters of a super-resolution reconstruction network, in obtaining a generated super-resolution image After that, a tongue appearance detection model is performed, wherein is a feature extractor, indicates that the tongue appearance detection is performed to obtain the type of tongue appearance.

[0022] Based on the same inventive concept, the present application also designs an electronic device, comprising:

[0023] one or more processors;

[0024] a storage device for storing one or more programs;

[0025] When the one or more programs are executed by the one or more processors, the one or more processors implement the traditional Chinese tongue diagnosis auxiliary method based on image reconstruction and tongue appearance detection.

[0026] Based on the same inventive concept, the present application also designs a computer readable medium having a computer program stored thereon, which is executed by a processor to implement the traditional Chinese tongue diagnosis auxiliary method based on image reconstruction and tongue appearance detection.

[0027] The present application has the advantages of:

[0028] In the present application, a super-resolution reconstruction module is used to reconstruct low-resolution tongue image into high-resolution image. This module receives low-definition tongue image from input device such as ordinary camera, and performs reconstruction processing on the image through pre-trained super-resolution neural network. The super-resolution network generates tongue image with higher resolution through multiple convolution layers, up-sampling layers and optimization of fusion loss function. The super-resolution reconstruction module can solve the problem of image blur caused by insufficient light and poor performance of imaging device. For the tongue appearance detection part, after the super-resolution module generates high-quality image, the image is input into the tongue appearance detection module. The tongue appearance detection module analyzes and identifies the features of tongue appearance in combination with pre-trained classifier. The module identifies the main area of tongue appearance through multiple feature extractors, and outputs the detection result through feature restoration decoder. Unlike existing research, in order to improve the interactivity of super-resolution reconstruction and tongue appearance detection module, the present application introduces a double-task feedback learning framework. Through feature alignment loss, super-resolution reconstruction not only depends on its own loss function for training, but also obtains task information related to tongue appearance detection from tongue appearance detection through feedback mechanism. The feature alignment loss ensures that the super-resolution reconstruction can better adapt to the requirements of the tongue appearance detection task when generating images, so that the super-resolution reconstruction can more accurately optimize the tongue appearance features during the training process. Attached Figure Description

[0029] Figure 1 This is a flowchart of the traditional Chinese medicine tongue diagnosis auxiliary method based on image reconstruction and tongue image detection according to the present invention.

[0030] Figure 2 This is a network structure diagram of image reconstruction and tongue image detection based on tongue diagnosis assistance in this invention.

[0031] Figure 3 This is a framework diagram of data augmentation and random mask generation in this invention.

[0032] Figure 4 This is a flowchart of the dual-task feedback learning framework proposed in this invention.

[0033] Figure 5 This is a schematic diagram of the tongue coating detection and classification results in the tongue diagnosis of this invention. Detailed Implementation

[0034] To make the objectives, features, and advantages of this invention more apparent and understandable, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, this invention is not limited to the following embodiments, and specific implementation methods can be determined according to the technical solutions of this invention and actual circumstances. To avoid obscuring the essence of this invention, well-known methods, processes, and procedures are not described in detail.

[0035] Example 1

[0036] This invention proposes a tongue image detection method based on a dual-task feedback learning framework to improve the quality of tongue image images and optimize the accuracy of tongue image detection. The system comprises two main task modules: a super-resolution reconstruction (SR) module and a tongue image detection (TD) module. The method operates as follows: first, the input tongue image is subjected to super-resolution reconstruction; then, the reconstructed high-quality image is processed by a tongue image detection network. These alternating training steps aim to achieve accurate tongue image detection.

[0037] For the tongue image examples of this invention, the TCMID-Tongue tongue image dataset published on the Traditional Chinese Medicine Comprehensive Database (http: / / www.megabionet.org / tcmid / ) was used. This dataset contains tongue images with different resolutions and five types of tongue coating: mirror-like, thin white coating, white greasy coating, yellow greasy coating, and gray-black coating, as shown below. Figure 5 As shown. Each image is annotated in XML format, including the bounding box of the tongue image and category labels.

[0038] The main steps and flowchart of the TCM tongue diagnosis auxiliary method based on image reconstruction and tongue image detection disclosed in the embodiment are shown in Figure 1 , and specifically include

[0039] S1: Preprocessing the image of the tongue image dataset. Input the tongue image true value image , and perform bicubic interpolation downsampling to obtain the corresponding low-resolution tongue image , and construct training data for the super-resolution model.

[0040] The specific implementation is as follows:

[0041] In the specific embodiment of the application, first, a high-quality true value tongue image is obtained, and a bicubic interpolation method is used to perform downsampling at a fixed downsampling ratio , for example, ×4, ×8 in the embodiment, to generate a corresponding low-resolution tongue image , simulating a low-quality image generated by an imaging device. Bicubic interpolation reduces the resolution by weighted average of surrounding pixels, retains details and reduces distortion. Then, the low-resolution image is taken as input data, and the true value high-resolution image is taken as output label to construct a paired training dataset for training the super-resolution model. The specific definition is as follows:

[0042]

[0043] wherein the dimension of the high-quality true value tongue image is , and the dimension of the low-resolution tongue image is . The downsampling scaling scale is {4, 8}. Bicubic is a bicubic interpolation downsampling method.

[0044] S2: Super-resolution reconstruction of the tongue image. The corresponding low-resolution tongue image obtained in S1 is input into the super-resolution reconstruction model , and feature extraction is performed by an encoder: the low-resolution tongue image downsampled in S1 is input into a network architecture based on convolutional neural network (CNN) and Transformer, and key information such as spatial and contextual texture, fine structure, etc. of the image is gradually extracted. And represented in a high-dimensional feature space. After feature extraction, the decoder learns the mapping relationship between the input high-bit feature and the high-resolution image, and uses a multi-layer neural perception network decoder to perform upsampling restoration to obtain the reconstructed super-resolution image .

[0045] The specific implementation method is as follows:

[0046] like Figure 2 As shown, the low-resolution tongue image generated by S1 Input to reconstruction model Super-resolution reconstruction is performed on the model. Composed of a network architecture based on convolutional neural networks (CNN) and Transformers, this embodiment selects existing super-resolution reconstruction network architectures such as EDSR-baseline, SwinIR, and SRFomer as the reconstruction model of this invention. During the encoding stage, a low-resolution tongue image is input. ,Model By progressively extracting key information such as spatial features and local texture details of the tongue image through multi-layer convolutional operations, and embedding these into pixels, a high-dimensional feature encoding of the input tongue image is obtained. Next, the multi-head attention mechanism of the Transformer is used to further model the contextual information of the encoded high-dimensional features and capture global dependencies, ensuring that the information of the input low-resolution tongue image is fully represented in the high-dimensional feature space. In the decoding stage, the model maps the high-dimensional features back to the high-resolution image space, i.e., it uses a multi-layer neural perceptual network for upsampling, progressively restoring and reconstructing the super-resolution tongue image. The specific definition is as follows:

[0047]

[0048] Among them, the low-resolution image is input for super-resolution reconstruction. Dimensions , It is a super-resolution reconstruction network. It is the reconstructed super-resolution image mapped back to the high-resolution image space, with dimensions and the true value of the tongue image. Same, that is .

[0049] S3: Implement image-level data augmentation strategies. Reconstruct the super-resolution image obtained in S2. As a basis for data augmentation, this preserves more details in the model reconstruction. The true value image of the tongue... A baseline image is used to compare the reconstruction results. A binary matrix of the same size as the original image is introduced as a random mask to control the image fusion method. After applying the random mask, the reconstructed image... and truth image The corresponding pixels are added element-wise according to the binary mask, and the data is fused to obtain the corresponding fused image. This data is then used as input for subsequent segmentation training to help the super-resolution reconstruction model. Learn more robust features at different pixel levels.

[0050] The implementation is as follows:

[0051] As shown in Figure 3 , based on the super-resolution reconstructed image obtained in S2 , the super-resolution model is retained Capture the learned information in the reconstruction process, and take the tongue image true value image , i.e. the original high-resolution image, as a comparison standard to enrich and verify the reconstruction effect. In order to realize effective image-level data fusion, this method uses a binary matrix with the same size as the image as a random mask to control the fusion mode of the reconstructed image and the true value image at each pixel position. After applying the random mask, the super-resolution image and the true value image are added element by element according to the pixel, which is defined as follows:

[0052]

[0053] where the random mask is a binary matrix with the same size as the image, and its dimension is , and are the pixel values of the super-resolution image and the tongue image true value image at the corresponding coordinates . Therefore, , .

[0054] Through the above element-by-element addition operation, the data augmentation fusion image retains the details in the super-resolution reconstructed image while combining the original information of the true value image. According to the value of the random mask, it is randomly decided to use the pixel of the true value image or the generated super-resolution image, specifically:

[0055]

[0056] The fused image will be used as input in the subsequent tongue image detection training, and the dimension represents . It can help the model learn more robust representations at different pixel levels.

[0057] S4: Implement a feature-level data augmentation strategy. Take the input tongue image true value image , the super-resolution reconstructed image obtained in S2 , and the corresponding fusion image obtained in S3 , respectively, using the same tongue image feature extraction encoder to extract high-dimensional features, obtaining high-dimensional features of the ground truth image , high-dimensional features of the super-resolution image , and high-dimensional features of the fusion image . Directly element-wise weighted combination of high-dimensional features creates diversified complex features for subsequent tongue image detection tasks, improving the analysis ability of subsequent tongue image detection models for different tongue image features.

[0058] The specific implementation is as follows:

[0059] As shown in Figure 2 , by extracting high-dimensional features from the ground truth image , the super-resolution reconstruction image obtained in S2 , the corresponding data enhanced fusion image obtained in S3 , respectively, using the same tongue image feature extraction model , the network architecture of FasterR-CNN is adopted in the present application for encoding, obtaining high-dimensional features of the ground truth image , high-dimensional features of the super-resolution image , and high-dimensional features of the fusion image . The specific representation is:

[0060] ( ),

[0061] ( ),

[0062] ( )

[0063] wherein the dimension of the generated feature is according to the network in order to generate more diversified and complex feature representation, we element-wise weighted combination of the above high-dimensional features. And manually set the combination coefficient corresponding to the feature fusion weight of the super-resolution image, the real image, and the fusion image generated by fusion, respectively, for data enhancement. The specific representation is:

[0064]

[0065] wherein is the fusion high-dimensional feature input into the subsequent tongue image detection model decoder, and needs to satisfy to ensure the rationality of the weighted combination.

[0066] S5: Tongue image feature decoding and tongue image detection. The high-dimensional tongue image feature obtained in S4 contains rich detailed information about the tongue image. Through the decoder , the fusion feature is restored to a visual output, realizing automatic tongue image detection.

[0067] The specific implementation is as follows:

[0068] The fusion high-dimensional tongue image feature obtained in S4 contains rich detailed information about the tongue image. These information includes the context global relationship and local content of the color, texture, and morphology of the tongue image. Therefore, this method takes it as the input of the decoder in the tongue image detection model, and gradually restores its feature to the tongue image detection result in the image space, where the decoder retains the RPN network structure, ROI pooling layer, and fully connected layer of classification and regression in FasterR-CNN. The output result includes the detection frame and classification probability, and the tongue image classification is as shown in Figure 5 . The specific representation is as follows:

[0069]

[0070] where B is the output result detection frame of the tongue image detection model, and its dimension is: 4, N is the number of detected tongue images, and 4 is the parameter (x, y, w, h). of each bounding box. The classification output is P， with a dimension of N x C, C = 5 , representing the number of categories, containing the category probability of each tongue image, and being restored to a visual output.

[0071] S6: Training according to the dual-task feedback learning framework. The tongue image feature extraction model in S4 and the tongue image detection model in S5 are parameter frozen, and the super-resolution model is trained. In this stage, the super-resolution model focuses on optimizing the tongue image fusion feature obtained in the tongue image detection. And ensure that the output features of the feature extraction encoder in the detection model and the decoder in the detection model are consistent. Finally, the tongue image feature extraction model , the decoder in the detection model, and the super-resolution reconstruction model are alternately trained to improve the accuracy and reliability of the overall tongue image detection.

[0072] The specific implementation is as follows:

[0073] Unlike the previous open-loop framework, the present application uses a double-task feedback learning closed-loop framework, aiming to improve the tongue diagnosis accuracy while improving the tongue image quality. As shown in Figure 4 , unlike the traditional open-loop structure, the method of the present application uses a feedback-connected feature alignment loss function Combining the super-resolution task (Super Resolution, SR) and the tongue image detection (Tongue Detection, TD) task module, the feature alignment loss is a specific implementation of the feedback connection. As shown in Figure 4 , during the training process, the forward connection transmits the image generated by SR to the TD module, so that TD can use higher quality images for detection; the feedback connection transmits the specific task features extracted by TD back to the SR module, and through back propagation, adjusts the SR network parameters to generate super-resolution images optimized for the detection task. This design promotes effective cooperation between the two modules, maximizes the use of feature information by passing feature priors, and thus improves detection accuracy.

[0074] The feature alignment loss function is specifically defined as:

[0075]

[0076] As shown in Figure 4 , according to the double-task feedback learning closed-loop framework in the present application, in addition to the feature-level loss function , which is used to guide the SR network to obtain specific task-driven features required in the downstream task TD network, the loss function also retains the original pixel-level loss in the SR task to ensure that the model can effectively retain the structure and texture details of the image during training. During training, the combined loss function of the SR task is specifically defined as:

[0077]

[0078] where N represents the total number of pixels in the image, i.e. . and represent the value of the i-th pixel in the low-resolution image and the high-resolution ground truth image, respectively.

[0079] When training the network for the tongue image detection (TD) task, the present application uses the loss function introduced in Faster R-CNN, which includes a classification loss and a bounding box regression loss . The classification loss usually uses cross-entropy to evaluate the accuracy of class prediction, while the bounding box regression loss uses smooth The loss function measures the difference between the predicted bounding box and the ground truth bounding box. Optimizing these two loss functions improves object detection performance. Therefore, the combined loss function for the TD task during training is specifically defined as follows:

[0080]

[0081] In summary, the goal of the dual-task feedback learning framework in this invention is to learn from low-resolution tongue image images. China uses super-resolution reconstruction model Generating super-resolution images To improve the target detection model The performance of can be formally expressed as:

[0082]

[0083] in, These represent the input low-resolution tongue image and the ground truth high-resolution tongue image, respectively, along with their corresponding task labels. pair. The fused image refers to the image generated after employing image-level quality data augmentation strategies. This indicates that there are trainable parameters. The super-resolution reconstruction network, in acquiring Generated super-resolution image Afterwards, the tongue image detection model passed... To proceed. Among them, It is a feature extractor. Including the subsequent RPN, ROL pooling, and fully connected layers in Faster R-CNN, the tongue image classification in the output results is shown in Figure 5.

[0084] Overall, the dual-task feedback learning framework in this invention consists of two main components: a super-resolution reconstruction (SR) network and a tongue detection (TD) network. In traditional open-loop dual-task training, the SR module receives low-resolution input... Generating super-resolution images Then, through the TD module... Tongue diagnosis is performed. However, this open-loop design restricts the information flow between the two modules, resulting in unsatisfactory detection results due to a lack of task-specific details. To address this issue, this invention proposes a closed-loop structure by introducing a feedback connection between TD and SR. This is achieved by introducing a feature alignment loss (…). ), we implement the feature-driven prior information transfer from TD to SR, establishing a strong dependency between tasks. This approach ensures that SR receives high-frequency information related to the task, thus improving image reconstruction, while TD benefits from the improved image quality, achieving more accurate detection. In addition, the quality fusion enhancement and alternating training strategy further enhance the effect of feature alignment loss and overall performance.

[0085] Embodiment Two

[0086] Based on the same inventive concept, the present application also provides an electronic device comprising one or more processors; a storage device for storing one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in Embodiment One.

[0087] Since the device introduced in Embodiment Two of the present application is the electronic device used to implement the Chinese tongue diagnosis auxiliary method based on image reconstruction and tongue appearance detection in Embodiment One of the present application, the specific structure and modifications of the electronic device can be understood by those skilled in the art based on the method introduced in Embodiment One of the present application, and therefore will not be described here. Any electronic device used in the method of Embodiment One of the present application belongs to the scope of protection of the present application.

[0088] Embodiment Three

[0089] Based on the same inventive concept, the present application also provides a computer readable medium having a computer program stored thereon, the program being executed by a processor to implement the method described in Embodiment One.

[0090] Since the device introduced in Embodiment Three of the present application is the computer readable medium used to implement the Chinese tongue diagnosis auxiliary method based on image reconstruction and tongue appearance detection in Embodiment One of the present application, the specific structure and modifications of the electronic device can be understood by those skilled in the art based on the method introduced in Embodiment One of the present application, and therefore will not be described here. Any electronic device used in the method of Embodiment One of the present application belongs to the scope of protection of the present application.

[0091] The above description of preferred embodiments of the present application in conjunction with the accompanying drawings does not limit the scope of the present application. Any modifications, equivalent replacements or improvements made by those skilled in the art without departing from the scope and core idea of the present application shall be included in the scope of protection of the present application.

Claims

1. A traditional Chinese medicine tongue diagnosis auxiliary method based on image reconstruction and tongue image detection, characterized in that, The method comprises the following steps: Collecting a tongue image true value image and processing to obtain a tongue image low resolution image; Constructing a tongue image detection learning model: performing super-resolution reconstruction on the tongue image low resolution image; performing image level enhancement processing on the reconstructed image; combining the tongue image true value image, the reconstructed image and the enhanced image to perform feature level enhancement processing, to obtain a fused high-dimensional tongue image feature map; performing tongue image detection on the high-dimensional tongue image feature map to obtain a tongue image type; Training the tongue image detection learning model, introducing a feature alignment loss between the super-resolution reconstruction and the tongue image detection, and feeding back specific task features extracted by the tongue image detection to the super-resolution reconstruction process to adjust the super-resolution reconstruction network parameters.

2. The image reconstruction and tongue pattern detection based traditional Chinese tongue diagnosis auxiliary method according to claim 1, characterized in that: Performing bicubic interpolation down-sampling on the tongue image to obtain a corresponding low resolution tongue image, and taking the tongue image and the corresponding low resolution tongue image as training data pairs.

3. The image reconstruction and tongue pattern detection based TCM tongue diagnosis auxiliary method according to claim 2, characterized in that: Inputting the low resolution tongue image into a convolutional neural network and a Transformer network architecture, gradually extracting spatial and contextual texture and subtle structure information of the image, and representing the information in a high-dimensional feature space; performing up-sampling restoration by using a multi-layer neural perception network decoder to obtain a reconstructed super-resolution image.

4. The image reconstruction and tongue pattern detection based traditional Chinese tongue diagnosis auxiliary method according to claim 1, characterized in that: The image-level enhancement processing specifically includes: taking the super-resolution reconstructed image and the tongue image true value image as inputs, introducing a binary matrix with the same size as the image as a random mask to control the image fusion, then performing element-by-element addition on corresponding pixels of the super-resolution reconstructed image and the tongue image true value image according to the binary mask to obtain a fusion image of the image-level enhancement processing . . .

5. The image reconstruction and tongue pattern detection based traditional Chinese tongue diagnosis auxiliary method according to claim 1, characterized in that: The feature level data augmentation processing includes, obtaining high-dimensional features of the true value image of the input tongue image , the super-resolution reconstruction image obtained in S2 , the corresponding fusion image obtained in S3 , respectively using the same feature extraction encoder in the tongue image detection model , obtaining high-dimensional features of the true value image , high-dimensional features of the super-resolution image , and high-dimensional features of the fusion image , directly performing element-by-element weighted combination on the high-dimensional features to obtain the fused high-dimensional tongue features.

6. The image reconstruction and tongue pattern detection based traditional Chinese tongue diagnosis auxiliary method according to claim 1, characterized in that: The tongue image detection model is composed of an RPN network structure in a FasterR-CNN, an ROI pooling layer and a full connection layer for classification and regression.

7. The image reconstruction and tongue pattern detection based traditional Chinese tongue diagnosis auxiliary method according to claim 1, characterized in that: The loss function in the super-resolution reconstruction is: where N denotes the total number of pixels in the image, i.e. , respectively denote the value of the i-th pixel in the low-resolution tongue image and the ground truth image, is the feature alignment loss function, represents a super-resolution reconstruction network with trainable parameters , is the feature extractor.

8. The image reconstruction and tongue pattern detection based traditional Chinese tongue diagnosis auxiliary method according to claim 1, characterized in that: The loss of the tongue image detection learning model is: wherein, is the combined loss function of the TD task during training, represents the network model connection, represents image stitching, represents the super-resolution reconstruction loss, respectively represent the input low-resolution tongue image and the tongue image ground truth, and the task label corresponding thereto pairing, refers to the high-dimensional tongue feature map generated after image-level enhancement, represents a super-resolution reconstruction network with trainable parameters after obtaining the generated super-resolution image , the tongue detection model is performed by , wherein, is a feature extractor, represents performing tongue detection to obtain the type of tongue.

9. An electronic device, comprising: The method comprises the following steps: One or more processors; A storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the image reconstruction and tongue image detection based traditional Chinese tongue diagnosis auxiliary method according to any one of claims 1-8.

10. A computer readable medium having stored thereon a computer program, characterized in that: The program is executed by the processor to implement the image reconstruction and tongue image detection based traditional Chinese tongue diagnosis auxiliary method according to any one of claims 1-8.

Images