Tongue image automatic detection method

Abstract

The invention discloses a tongue image automatic detection method, which belongs to the field of image processing, and comprises the following steps: obtaining a tongue image; inputting the tongue image into a shared trunk convolutional neural network, and extracting a multi-scale feature map; inputting the multi-scale feature map into a tooth mark detection branch, a tongue coating detection branch and a tongue crack detection branch at the same time; wherein the tooth mark detection branch outputs a tooth mark position detection frame, the tongue coating detection branch outputs a tongue coating area detection frame and a tongue coating color classification result and a tongue coating thickness classification result based on a central area of the detection frame, and the tongue crack detection branch outputs a tongue crack position detection frame. According to the method, multiple tasks are executed by sharing the backbone network and branches, multiple diagnosis results can be output at the same time, the detection efficiency is greatly improved, and the method is suitable for large-scale application scenes.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of image processing, and particularly relates to a tongue image automatic detection method. BACKGROUND

[0002] Traditional Chinese medicine tongue diagnosis mainly relies on the naked eye observation and experience judgment of doctors on tongue features, and disease diagnosis is performed by analyzing features such as tooth marks, thickness and color of tongue fur, and tongue cracks. With the development of computer vision technology, some studies attempt to use image processing technology (such as edge detection, morphological processing, color segmentation, etc.) or single deep learning model (such as YOLO series, Faster R-CNN) to realize automatic detection of tongue diagnosis features, and generate diagnosis results by processing each tongue feature.

[0003] The prior art has significant defects: traditional manual diagnosis is affected by the experience level of doctors, and different doctors are prone to differences in judging the same tongue, resulting in poor consistency of diagnosis results; the existing automatic method needs to detect tooth marks, tongue fur, cracks and other features one by one, repeatedly performs feature extraction and calculation, and the processing speed cannot meet the large-scale tongue diagnosis demand; when detecting each feature in isolation, the correlation between different features (such as the cooperative relationship between tongue fur region segmentation and color classification) is not considered, which is prone to error accumulation due to feature interference, and reduces the overall detection robustness; the existing model architecture does not use modular design, and the addition of tongue diagnosis feature detection tasks needs to reconstruct the entire network, which is difficult to flexibly adapt to diversified diagnosis demand. SUMMARY

[0004] To solve the above technical problems, the present application provides a tongue image automatic detection method to solve the problems existing in the prior art.

[0005] To achieve the above purpose, in a first aspect, the present application provides a tongue image automatic detection method, comprising:

[0006] obtaining a tongue image;

[0007] inputting the tongue image into a shared backbone convolutional neural network to extract a multi-scale feature map;

[0008] simultaneously inputting the multi-scale feature map into a tooth mark detection branch, a tongue fur detection branch and a tongue crack detection branch;

[0009] wherein the tooth mark detection branch outputs a tooth mark position detection frame, the tongue fur detection branch outputs a tongue fur region detection frame, and a tongue fur color classification result and a tongue fur thickness classification result based on the center region of the detection frame, and the tongue crack detection branch outputs a tongue crack position detection frame.

[0010] Preferably, the step of extracting a multi-scale feature map comprises:

[0011] The first scale feature map, the second scale feature map and the third scale feature map are generated by sharing a backbone convolutional neural network;

[0012] The resolution of the first scale feature map is 80x80x256, the resolution of the second scale feature map is 40x40x256, and the resolution of the third scale feature map is 20x20x256.

[0013] Preferably, the tooth mark detection branch processing process comprises:

[0014] The multi-scale feature map is subjected to deep separable convolution dimension reduction;

[0015] The dimension-reduced feature map is subjected to cross-scale splicing;

[0016] A 1x1 convolution operation is performed on the splicing result to generate a tooth mark position regression result and a tooth mark existence classification result;

[0017] The position regression result and the classification result are integrated based on a non-maximum suppression algorithm to output a final tooth mark detection frame.

[0018] Preferably, the tongue fur detection branch processing process comprises:

[0019] A tongue fur region detection frame is generated by a tongue fur region detection subnetwork;

[0020] A tongue fur center region image of a preset size is intercepted according to the center coordinates of the tongue fur region detection frame;

[0021] The tongue fur center region image is input into an independent classification subnetwork to output a tongue fur color classification result and a tongue fur thickness classification result in parallel.

[0022] Preferably, the process of intercepting the tongue fur center region image of the preset size comprises:

[0023] When the length of the detection frame is greater than or equal to 64 pixels and the width is greater than or equal to 64 pixels, the region is intercepted by expanding the center point by 32 pixels in all directions;

[0024] When the length of the detection frame is greater than or equal to 64 pixels and the width is less than 64 pixels, the width is scaled to 64 pixels after expanding to 64 pixels in the height direction;

[0025] When the length of the detection frame is less than 64 pixels and the width is greater than or equal to 64 pixels, the height is scaled to 64 pixels after expanding to 64 pixels in the width direction;

[0026] When the length of the detection frame is less than 64 pixels and the width is less than 64 pixels, it is directly scaled to 64x64 pixels.

[0027] Preferably, the tongue fur crack detection branch processing process comprises:

[0028] performing deep separable convolution dimension reduction on the multi-scale feature maps;

[0029] performing cross-scale splicing on the dimension-reduced feature maps;

[0030] performing 1x1 convolution operation on the splicing result to generate tongue crack position regression result and tongue crack existence classification result;

[0031] integrating the output final tongue crack detection frame based on a non-maximum suppression algorithm.

[0032] Preferably, the training process of generating the tongue crack position regression result and the tongue crack existence classification result comprises:

[0033] adopting a binary cross-entropy loss function to optimize the classification result;

[0034] adopting a Smooth L1 loss function to optimize the position regression result;

[0035] weighting and summing the two types of loss functions as the total loss function of the tongue crack detection branch.

[0036] Preferably, the model weights of the tooth mark detection branch, the tongue fur detection branch and the tongue crack detection branch are independent of each other.

[0037] In a second aspect, the present application further discloses a computer readable storage medium, which has a computer program stored thereon, and the computer program is executed by a processor to realize the steps of the method in the first aspect.

[0038] In a third aspect, the present application further discloses a computer program product comprising a computer program, and the computer program is executed by a processor to realize the steps of the method in the first aspect.

[0039] Compared with the prior art, the present application has the following advantages and technical effects:

[0040] The present application provides an automatic tongue image detection method, which comprises the following steps: firstly, acquiring a tongue image; secondly, inputting the tongue image into a shared backbone convolutional neural network to extract multi-scale feature maps; and finally, simultaneously inputting the multi-scale feature maps into a tooth mark detection branch, a tongue fur detection branch and a tongue crack detection branch; wherein the tooth mark detection branch outputs a tooth mark position detection frame, the tongue fur detection branch outputs a tongue fur region detection frame and a tongue fur color classification result and a tongue fur thickness classification result based on the center region of the detection frame, and the tongue crack detection branch outputs a tongue crack position detection frame.

[0041] The application can output multiple diagnostic results at the same time by sharing the backbone network and performing multiple tasks (such as notch detection, tongue fur detection, crack detection, etc.), greatly improving the detection efficiency, being suitable for large-scale application scenarios, and solving the technical problem of low efficiency of the traditional tongue detection method which usually processes each tongue feature one by one.

[0042] The application uses the complementary information of each detection task (such as the tongue fur region segmentation result to provide support for tongue color and thickness classification) through the multi-task learning framework, ensures the information sharing and collaborative learning between tasks, reduces the error caused by isolated detection, improves the consistency and robustness of the result, and solves the technical problem that each feature is easily disturbed by other features when detected separately. BRIEF DESCRIPTION OF DRAWINGS

[0043] The accompanying drawings, which form a part of this application, are included to provide a further understanding of the application and are incorporated in and constitute a part of this application. The embodiments of this application and their description together with the drawings serve to explain the application. In the drawings:

[0044] Figure 1 The tongue image automatic detection method of the embodiment of the application is shown in the figure;

[0045] Figure 2 The tongue fur thickness and color classification network architecture of the embodiment of the application is shown in the figure. DETAILED DESCRIPTION

[0046] It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0047] It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a group of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described herein can be executed in a different order.

[0048] Embodiment one

[0049] As shown in Figure 1 The tongue image automatic detection method provided in the embodiment includes:

[0050] S1, obtaining a tongue image;

[0051] S2, inputting the tongue image into a shared backbone convolutional neural network to extract a multi-scale feature map;

[0052] Further, the step of extracting the multi-scale feature map includes:

[0053] generate a first scale feature map, a second scale feature map and a third scale feature map through a shared backbone convolutional neural network;

[0054] The resolution of the first scale feature map is 80x80x256, the resolution of the second scale feature map is 40x40x256, and the resolution of the third scale feature map is 20x20x256.

[0055] Specifically, the backbone network is responsible for extracting high-quality global features from the tongue image. In order to ensure that the extracted features have good expression ability, the embodiment selects YOLOv8 as the backbone network. The YOLOv8 backbone network extracts features. This network has been pre-trained on a large-scale dataset, which can accelerate the convergence of the model and improve performance.

[0056] Since the embodiment faces a multi-task detection problem, the global features extracted by the backbone network can be shared by different tasks. This not only reduces the parameter amount, but also enables the model to learn more robust features through joint training. After the output of the last layer of the backbone network, these shared features are passed to each task branch.

[0057] S3, inputting the multi-scale feature map into a tooth mark detection branch, a tongue fur detection branch and a tongue crack detection branch at the same time;

[0058] The tooth mark detection branch outputs a tooth mark position detection box, the tongue fur detection branch outputs a tongue fur region detection box and a tongue fur color classification result and a tongue fur thickness classification result based on the center region of the detection box, and the tongue crack detection branch outputs a tongue crack position detection box.

[0059] Further, the tooth mark detection branch processing process includes:

[0060] Performing depth separable convolution dimension reduction on the multi-scale feature map;

[0061] Performing cross-scale concatenation on the dimension-reduced feature map;

[0062] Performing a 1x1 convolution operation on the concatenation result to generate a tooth mark position regression result and a tooth mark existence classification result;

[0063] Integrating the position regression result and the classification result based on a non-maximum suppression algorithm to output a final tooth mark detection box.

[0064] Further, the tongue fur detection branch processing process includes:

[0065] Generating a tongue fur region detection box through a tongue fur region detection subnetwork;

[0066] According to the center coordinates of the tongue fur region detection box, a tongue fur center region image of a preset size is intercepted.

[0067] The tongue center area image is input into an independent classification subnetwork, and the tongue color classification result and the tongue thickness classification result are output in parallel.

[0068] Further, the tongue crack detection branch processing process includes:

[0069] Depth separable convolution dimension reduction is performed on the multi-scale feature map;

[0070] The dimension-reduced feature map is cross-scale spliced;

[0071] 1x1 convolution operation is performed on the spliced result to generate tongue crack position regression result and tongue crack existence classification result;

[0072] The non-maximum suppression algorithm is used to integrate the output final tongue crack detection frame.

[0073] Specifically, in order to process multiple feature detection tasks of tongue image, a multi-task learning model is used. This model can simultaneously learn different detection tasks (tooth mark detection, tongue fur color and thickness classification, and tongue crack), thereby improving the generalization ability and overall performance of the model.

[0074] The multi-task branch is responsible for processing different features (tooth marks, tongue fur thickness and color, and tongue cracks) in the tongue image. Each task has a dedicated subnetwork for processing the respective task.

[0075] (1) Tooth mark detection branch: The task of this branch is to detect the position of tooth marks on the tongue.

[0076] a. Tooth mark detection branch network architecture:

[0077] Input: The image resolution is 256x256x3, and the input is a multi-scale 80x80x256, 40x40x256, and 20x20x256 feature map extracted by the YOLOv8 backbone network.

[0078] Output: The output is a rectangular frame containing tongue tooth marks.

[0079] Network architecture: The network structure of the tongue tooth mark detection head is mainly based on the feature maps extracted by the YOLOv8 backbone network. The input is three scales of feature maps: 80x80x256, 40x40x256, and 20x20x256. The 80x80x256 scale feature map is reduced in channel number by 3x3 depth separable convolution, and the output dimension is 80x80x128. At the same time, it is changed to 40x40x256 by 2x2 max pooling and spliced with the 40x40x256 scale feature map. For the 40x40x256 scale feature map, the dimension after splicing is 40x40x512, and it is processed by two layers of 3x3 depth separable convolution, and the dimension is changed to 40x40x256 and 40x40x128 in turn. Among them, the 40x40x256 feature is processed by 2x2 max pooling to obtain 20x20x256 feature and spliced with the 20x20x256 scale feature map. For the 20x20x256 scale feature map, two layers of 3x3 depth separable convolution are used to finally obtain a feature with a dimension of 20x20x128. For the feature maps of different scales (80x80x128, 40x40x128, 20x20x128), the network is divided into two parts: the first part generates n*n*4(n=20, 40, 80) bounding box regression results (detects the position and size of the tongue tooth mark) through 1x1 convolution. The second part generates n*n*1(n=20, 40, 80) classification results (whether there is a tooth mark) through 1x1 convolution. Finally, non-maximum suppression (NMS) is used to remove the repeated detection boxes under the three scales to obtain the final detection box. This structure effectively judges whether there is a tongue tooth mark and locates the tongue tooth mark range to generate a detection box.

[0080] b、Tooth mark detection branch loss function:

[0081] The loss function of the tooth mark detection branch uses a combination of binary cross-entropy loss and smooth L1 loss.

[0082]

[0083] In tongue tooth mark detection, binary cross-entropy loss is used for classification tasks, such as determining whether there is a tooth mark on the tongue. It can effectively guide the model to learn classification features by comparing the difference between the predicted probability and the true label. The binary cross-entropy loss is shown in formula (1), where yi is the true label (with or without tooth marks) of the i-th sample, pi is the predicted probability of the i-th sample, and N is the number of samples.

[0084] Smooth L1 loss is used for bounding box regression to regress the coordinates of the dental mark position. Since bounding box regression involves subtle position information, Smooth L1 loss can better handle small errors while suppressing the influence of outliers. Smooth L1 loss is shown in equations (2) and (3), where xi is the true value, x^i is the predicted value, and N is the number of samples. It uses the form of quadratic loss when the error is small, which helps to improve the accuracy of the model in small target detection, and uses the form of linear loss when the error is large, which can avoid gradient explosion caused by outliers, thereby improving the stability of training.

[0085] Finally, the combined loss function of the dental mark detection branch of the embodiment is shown in equation (4), where Lcls and Lreg are binary cross-entropy loss and Smooth L1 loss, respectively, and λcls and λreg are weight coefficients for balancing the classification loss and bounding box regression loss.

[0086] (2) Tongue fur thickness and color detection branch: The task of this branch is to detect the tongue fur area and classify the color and thickness of the tongue fur. This branch will first regress the tongue fur area detection box through the tongue fur area detection network, and then sample the image of the center area (64x64) of the detection box to input into the tongue fur thickness and color classification network to obtain the thickness (thin fur, thick fur) and color (white fur, yellow fur) results of the tongue fur. When sampling the center area of the detection box, the following principles are followed according to the different situations of the length and width (h x w) of the detection box and 64x64:

[0087] When the length h of the detection box > = 64 and the width w of the detection box > = 64: Calculate the center point of the detection box, expand 32 pixels in the up, down, left, and right directions from the center point of the detection box, and then cut out a 64x64 image as the sampling result.

[0088] When the length h of the detection box > = 64 and the width w of the detection box < 64: Calculate the center point of the detection box, expand 32 pixels in the up and down directions from the center point of the detection box, then cut out a 64xw image, and resize the image size to 64x64 as the sampling result.

[0089] When the length h of the detection box < 64 and the width w of the detection box > = 64: Calculate the center point of the detection box, expand 32 pixels in the left and right directions from the center point of the detection box, then cut out an h x 64 image, and resize the image size to 64x64 as the sampling result.

[0090] When the length h of the detection box < 64 and the width w of the detection box < 64: directly cut out an h x w image, and resize the image size to 64x64 as the sampling result.

[0091] a, Tongue fur area detection network architecture:

[0092] Input: The image resolution is 256x256x3, and the input is a multi-scale 80x80x256, 40x40x256, and 20x20x256 feature map extracted by the YOLOv8 backbone network.

[0093] Output: The output is a rectangular box containing the tongue fur area.

[0094] Network architecture: The network structure of the tongue fur area detection head is the same as that of the tooth mark detection head, but only the backbone network shares weights, and the detection head does not share weights. The input is a feature map of three scales: 80x80x256, 40x40x256, and 20x20x256. The 80x80x256 scale feature map is reduced in channel number by a 3x3 depth separable convolution, and the output dimension is 80x80x128. At the same time, it is changed to 40x40x256 by 2x2 max pooling and spliced with the 40x40x256 scale feature map. For the 40x40x256 scale feature map, the dimension after splicing is 40x40x512, and two layers of 3x3 depth separable convolution are used to change the dimension to 40x40x256 and 40x40x128, respectively. Among them, the 40x40x256 feature is processed by 2x2 max pooling to obtain a 20x20x256 feature, which is spliced with the 20x20x256 scale feature map. For the 20x20x256 scale feature map, two layers of 3x3 depth separable convolution are used to finally obtain a 20x20x128 dimension feature. For the feature maps of different scales (80x80x128, 40x40x128, 20x20x128), the network is divided into two parts: the first part generates n*n*4 (n=20, 40, 80) bounding box regression results (detects the position and size of the tongue fur area) through 1x1 convolution. The second part generates n*n*1 (n=20, 40, 80) classification results (whether there is tongue fur) through 1x1 convolution. Finally, non-maximum suppression (NMS) is used to remove the repeated detection boxes under the three scales to obtain the final detection box.

[0095] b, Tongue fur area detection loss function:

[0096] The loss function of the tongue fur area detection branch in this embodiment is the same as the combined loss function of the binary cross-entropy loss and the smooth L1 loss of the tooth mark detection branch.

[0097] c, Tongue fur thickness and color classification network architecture:

[0098] Input: 64x64x3 tongue center area image.

[0099] Output: The color (white or yellow) and thickness (thin or thick) classification results of tongue fur.

[0100] Network architecture: The network architecture is as shown in Figure 2 The tongue fur thickness and color classification network structure is composed of an input layer, multiple convolutional layers, a max pooling layer, and a fully connected layer. The input is a 64x64x3 image, which is extracted through four convolutional layers, compressed through two max pooling layers, and then flattened into a vector through a fully connected layer and Softmax to obtain the classification result. The tongue fur thickness and color share this network architecture but do not share weights. The same network trains two sets of weights, which are used to output the color (white or yellow) and thickness (thin or thick) classification results of tongue fur.

[0101]

[0102] d. Loss function for tongue fur thickness and color classification:

[0103] Since tongue fur thickness and color classification is a binary classification task, and binary cross-entropy is specifically designed to handle cases with only two classes (such as white and yellow fur, thin and thick fur), the output probability can be directly compared with the true label. Therefore, binary cross-entropy is used as the loss function here, as shown in Equations 5 and 6, which are the loss functions for tongue fur thickness classification and tongue fur color classification, respectively. Here, yi is the true label (white or yellow fur, thin or thick fur), and y^i is the predicted tongue fur thickness and color probability.

[0104] (3) Tongue crack detection branch: The task of this branch is to detect cracks on the tongue.

[0105] a. Network architecture of tongue crack detection branch:

[0106] Input: The image resolution is 256x256x3, and the input is a multi-scale 80x80x256, 40x40x256, 20x20x256 feature map extracted by the YOLOv8 backbone network.

[0107] Output: The output is a rectangular box containing the tongue fur area.

[0108] Network architecture: the network architecture of the tongue crack detection branch is the same as that of the tooth mark detection head, but only the main network shares weights, and the detection head does not share weights. The input is a feature map of three scales of 80x80x256, 40x40x256 and 20x20x256. The 80x80x256 scale feature map is reduced in channel number by 3x3 depth separable convolution, and the output dimension is 80x80x128. At the same time, it is changed into 40x40x256 by 2x2 max pooling and spliced with the 40x40x256 scale feature map. For the 40x40x256 scale feature map, the dimension after splicing is 40x40x512, and two layers of 3x3 depth separable convolution are performed, and the dimensions are changed to 40x40x256 and 40x40x128 in turn. Among them, the 40x40x256 features are subjected to 2x2 max pooling to obtain 20x20x256 features, which are spliced with the 20x20x256 scale feature map. For the 20x20x256 scale feature map, two layers of 3x3 depth separable convolution are performed, and finally a feature of 20x20x128 is obtained. For the feature maps of different scales (80x80x128, 40x40x128, 20x20x128), the network is divided into two parts: the first part generates n x n x 4 (n = 20, 40, 80) bounding box regression results (detects the position and size of the tongue crack) through 1x1 convolution. The second part generates n x n x 1 (n = 20, 40, 80) classification results (whether there is a tongue crack) through 1x1 convolution. Finally, non-maximum suppression (NMS) is used to remove the repeated detection boxes under the three scales to obtain the final detection box.

[0109] b. Loss function of tongue crack detection branch:

[0110] The loss function of the tongue crack detection branch in this embodiment is the same as the combined loss function of the tooth mark detection branch, which is a combination of binary cross-entropy loss and smooth L1 loss.

[0111] In this embodiment, the detection branches of each task are trained at the same time, and the weighted sum of the loss functions can be used to optimize the entire network, so that the network can learn the detection of multiple tongue features at the same time.

[0112] Further, the model weights of the tooth mark detection branch, the tongue fur detection branch and the tongue crack detection branch are independent of each other.

[0113] The beneficial effects of the embodiment are as follows:

[0114] High detection efficiency: benefits from the shared backbone network YOLOv8 and multi-task branch structure. Unlike traditional methods that usually rely on single feature extraction and sequential analysis, this method uses YOLOv8 as the backbone network to extract the global features of the tongue image. These features are shared by multiple detection branches, avoiding the computational burden of repeated feature extraction and greatly improving the overall processing efficiency. At the same time, the network structure can output multiple results simultaneously by parallel processing of multiple tasks (dental mark, tongue fur, and tongue crack detection), reducing the inference time of the model. This design enables the system to process a large number of tongue diagnosis images in a short time, suitable for large-scale tongue diagnosis automation detection scenarios.

[0115] High detection accuracy: benefits from the multi-task learning framework. By sharing the backbone network, the network can simultaneously learn features related to multiple tasks, promoting information complementarity between different tasks. For example, tongue fur region detection can provide additional information support for tongue fur color and thickness classification, making the classification more robust. Collaborative learning between tasks can improve the detection accuracy of the overall model and reduce error accumulation caused by single tasks.

[0116] Good model scalability: benefits from modular design. This method is based on a shared backbone network and implements different detection tasks through multi-task branches. This modular design makes it easy to extend other tongue diagnosis feature detection tasks in the future. For example, more branches can be added to the existing network structure to detect other tongue image features, such as tongue surface texture and tongue fur morphology, without the need for large-scale modifications to the backbone network. Therefore, the overall system maintains efficiency and accuracy while having high scalability to adapt to future diverse tongue diagnosis needs.

[0117] Embodiment Two

[0118] The embodiment also discloses a computer readable storage medium, which stores a computer program. The computer program is executed by a processor to implement the steps of the method in Embodiment One.

[0119] Embodiment Three

[0120] The embodiment also discloses a computer program product, which includes a computer program. The computer program is executed by a processor to implement the steps of the method in Embodiment One.

[0121] The above is only the preferred specific implementation of the present application, but the protection scope of the present application is not limited to this. Any changes or replacements that can be easily thought of by those skilled in the art within the technical range disclosed in the present application should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A tongue image automatic detection method, characterized in that: The following steps are involved: Obtain tongue images; Inputting the tongue image into a shared backbone convolutional neural network to extract a multi-scale feature map; Inputting the multi-scale feature map into the tooth mark detection branch, the tongue coating detection branch and the tongue crack detection branch simultaneously; Among them, the tooth mark detection branch outputs a tooth mark position detection frame, the tongue coating detection branch outputs a tongue coating area detection frame and the tongue coating color classification result and tongue coating thickness classification result based on the center area of ​​the detection frame, and the tongue crack detection branch outputs a tongue crack position detection frame.

2. The method according to claim 1, characterized in that The steps of extracting multi-scale feature maps include: Generate a first-scale feature map, a second-scale feature map, and a third-scale feature map through a shared backbone convolutional neural network; The resolution of the first-scale feature map is 80×80×256, the resolution of the second-scale feature map is 40×40×256, and the resolution of the third-scale feature map is 20×20×256.

3. The method according to claim 1, characterized in that The tooth mark detection branch processing process includes: Performing depth-wise separable convolution dimensionality reduction on the multi-scale feature map; Cross-scale splicing of the feature maps after dimensionality reduction; Perform a 1×1 convolution operation on the spliced ​​results to generate the tooth mark position regression result and the tooth mark presence classification result; The position regression result and the classification result are integrated based on the non-maximum suppression algorithm to output the final tooth mark detection frame.

4. The method according to claim 1, wherein The tongue coating detection branch processing process includes: Generate a tongue coating area detection frame through the tongue coating area detection sub-network; According to the center coordinates of the tongue coating area detection frame, intercepting the tongue coating center area image of a preset size; The tongue coating center area image is input into an independent classification sub-network, and the tongue coating color classification result and the tongue coating thickness classification result are output in parallel.

5. The method according to claim 4, characterized in that The process of capturing an image of the tongue coating center area of ​​a preset size includes: When the detection frame is greater than or equal to 64 pixels in length and greater than or equal to 64 pixels in width, the intercepted area is expanded 32 pixels in all directions from the center point; When the detection frame is longer than or equal to 64 pixels and less than 64 pixels wide, it is expanded to 64 pixels in height and then scaled back to 64 pixels in width. When the detection frame is less than 64 pixels in length and greater than or equal to 64 pixels in width, it is expanded to 64 pixels in width and then scaled to 64 pixels in height. When the detection box is less than 64 pixels in length and width, it is directly scaled to 64×64 pixels.

6. The method according to claim 1, characterized in that The tongue crack detection branch processing process includes: Performing depth-wise separable convolution dimensionality reduction on the multi-scale feature map; Cross-scale splicing of the feature maps after dimensionality reduction; Perform a 1×1 convolution operation on the splicing results to generate tongue crack position regression results and tongue crack presence classification results; The final tongue crack detection frame is integrated and output based on the non-maximum suppression algorithm.

7. The method according to claim 3 or 6, characterized in that The training process for generating tongue crack location regression results and tongue crack presence classification results includes: The binary cross entropy loss function is used to optimize the classification results; The Smooth L1 loss function is used to optimize the position regression results; The weighted sum of the two types of loss functions is used as the total loss function of the tongue crack detection branch.

8. The method according to claim 1, characterized in that The model weights of the tooth mark detection branch, the tongue coating detection branch, and the tongue crack detection branch are independent of each other.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images