Tongue picture image segmentation method based on edge gating mechanism

Abstract

The invention discloses a tongue picture image segmentation method based on an edge gating mechanism. According to the method, the preprocessing operation of tongue body positioning and color correction is adopted for the original tongue picture data collected by the mobile equipment. Aiming at the problem of rough tongue edge extraction in a tongue segmentation task, the invention designs a tongue segmentation model based on an edge gating mechanism, the model is improved by adding an edge branch on the basis of a Deeplabv3 + model, and a loss function of weighted fusion of segmentation loss and edge loss is designed to train the improved model. Therefore, the prediction precision of the model segmentation mask is improved. According to the invention, a jump connection method is used in the decoder, feature fusion is carried out on the feature vector obtained by the encoder and the feature vector of the decoder, and the image is restored to the original size by adopting a double-layer convolution and up-sampling mechanism, so that the feature capture capability is enhanced, and the decoder performance and the segmentation precision are improved.

Description

Technical Field

[0001] The present invention relates to a semantic segmentation method for a tongue body in the field of automated tongue image diagnosis in traditional Chinese medicine diagnosis, and in particular to a tongue image segmentation method based on an edge gating mechanism. Background Art

[0002] Traditional TCM tongue diagnosis methods are too dependent on subjective conditions such as the doctor's personal judgment and clinical experience, and are also easily affected by external factors such as the environment, light, and weather, resulting in the non-repeatability and ambiguity of the traditional tongue diagnosis process. In addition, the characteristics of tongue diagnosis are difficult to quantify, and the lack of standardized diagnostic results also makes it difficult for the TCM system to develop and promote further. Therefore, promoting the objective research of TCM tongue diagnosis methods has become one of the hot research directions of TCM automated diagnosis and treatment systems. The necessary prerequisite for the objectivity of tongue diagnosis is to separate the main body of the tongue from the face and lips of the person, that is, to achieve the task of tongue segmentation. When collecting tongue images, due to the limitations of hardware equipment, the collected images usually contain additional facial information in addition to the tongue. Although the tongue area can be captured to the greatest extent by optimizing the hardware equipment and shooting angle, it is impossible to completely eliminate redundant information other than the tongue, which brings certain challenges to the subsequent tongue classification task. Therefore, tongue segmentation must be performed before tongue feature classification.

[0003] Tongue segmentation can be regarded as an image segmentation task. Traditional image segmentation methods include threshold-based methods, watershed methods, edge detection-based methods, graph-cut-based methods, Snakes active contour model, etc. However, the application of traditional image processing methods to tongue segmentation and extraction is usually subject to great limitations. For example, the graph-cut-based method requires the bounding box to be set as the target segmentation position, while the Snakes active contour model uses the HSV color space method to obtain the initial contour of the tongue, and converges the contour to the edge of the tongue by minimizing the energy function, but a suitable energy function needs to be designed, so the segmentation results are relatively rough and cannot achieve the intended goal. In recent years, with the development of deep learning, many models with excellent segmentation effects have emerged in image segmentation and semantic segmentation tasks, such as the U-Net series, Deeplab series, R-CNN series and other networks, which have also been applied to the field of tongue segmentation. However, tongue image data has problems such as different shooting angles, different tongue shapes and sizes, tongue pathological characteristics affecting tongue color and texture, and the similar colors of lips and tongue easily lead to blurred edges, making it impossible to fully transplant classic deep learning models to the tongue segmentation task, especially difficult to directly solve the problem of tongue edge segmentation. Summary of the invention

[0004] Tongue image data faces multiple challenges in analysis, including differences in shooting angles, different tongue shapes and sizes, the impact of tongue pathological characteristics on tongue color and material, etc., and the similarity of lip and tongue color often leads to unclear tongue edges. These complex factors make it difficult to directly apply classic deep learning models to tongue segmentation tasks, especially in accurately segmenting the tongue edge. In order to overcome the limitations of the above-mentioned prior art, the present invention provides a tongue image segmentation method based on an edge gating mechanism to solve the problem of fuzzy tongue edge segmentation.

[0005] In order to achieve the above purpose, the technical solution adopted by the present invention is as follows:

[0006] 1. A tongue image segmentation method based on edge gating mechanism

[0007] Step 1: Collect the patient's tongue-extending image;

[0008] Step 2: After preprocessing the collected tongue-extending image, a processed standard tongue image is obtained;

[0009] Step 3: Input the processed standard tongue image into the trained tongue segmentation model based on the edge gating mechanism, and the model outputs the corresponding tongue segmentation result.

[0010] The step 2 is specifically as follows:

[0011] After tongue positioning and color cast correction are performed on the collected tongue-extending image, a processed standard tongue image is obtained.

[0012] The step 2 is specifically as follows:

[0013] Step 21: Use the YOLOv5-Lite-based target detection model to locate the tongue body in the collected tongue-extending image to obtain the tongue body image;

[0014] Step 22: After color cast check of the tongue body image, a processed standard tongue image is obtained.

[0015] The step 22 is specifically as follows:

[0016] Firstly, the color cast detection method of tongue image based on equivalent circle detection method is used to calculate the color cast degree of each tongue body image. If the color cast degree of each tongue body image is greater than the preset threshold, the tongue body image is subjected to color cast verification to obtain the processed standard tongue image, otherwise it is not processed.

[0017] The tongue segmentation model based on the edge gating mechanism includes a ResNeXt model embedded with an SE attention mechanism, an ASPP module, a decoder and an edge prediction module, wherein the ResNeXt model embedded with an SE attention mechanism serves as an encoder for feature extraction, and the processed standard tongue image serves as the input of the ResNeXt model embedded with the SE attention mechanism. The ResNeXt model embedded with the SE attention mechanism extracts feature vectors X at 6 different depths. 0 , X 1 , X 2 , X 3 , X 4 , X 5 , the first eigenvector X 0 , the third eigenvector X 2 , the fourth eigenvector X 3 and the sixth eigenvector X 5 are input into the edge prediction module, the sixth eigenvector X 5 After being spliced ​​with the output of the edge prediction module, it is input into the ASPP module, and the output of the ASPP module serves as the input of the decoder.

[0018] The edge prediction module includes a Canny operator edge extraction module, a fusion layer, a convolution layer, a residual module and a gated attention layer. The first feature vector X 0 After passing through the first convolutional layer and the first residual module, it is connected to the first gated attention layer. The third feature vector X 2 After the second convolutional layer, it is connected to the first gated attention layer. After the first gated attention layer passes through the second residual module, it is connected to the second gated attention layer. The fourth eigenvector X 3 After the third convolutional layer, it is connected to the second gated attention layer. After the second gated attention layer passes through the third residual module, it is connected to the third gated attention layer. The sixth eigenvector X 5 After passing through the fourth convolutional layer, it is connected to the third gated attention layer. The output of the third gated attention layer is recorded as the edge output. The processed standard tongue image is also used as the input of the Canny operator edge extraction module. The output of the third gated attention layer and the output of the Canny operator edge extraction module are added and then input into the fusion layer. The output of the fusion layer is used as the output of the edge prediction module.

[0019] The gated attention layer includes a batch normalization layer, a convolution layer, a Relu activation layer and a Sigmoid activation layer. Among the two inputs of each gated attention layer, the output of the corresponding residual module is recorded as the first input feature, and the other input is recorded as the second input feature. After element-by-element addition, they are input into the batch normalization layer. The batch normalization layer is sequentially connected to the Sigmoid activation layer after passing through the fifth convolution layer, the first Relu activation layer, the sixth convolution layer and the second Relu activation layer. The result of multiplying the second input feature by α times the output of the Sigmoid activation layer is recorded as the intermediate feature. The intermediate feature is added to the second input feature and then input into the seventh convolution layer. The output of the seventh convolution layer is used as the output of the current gated attention layer.

[0020] The decoder contains a double-layer convolution block and an upsampling layer. The output of the ASPP module is combined with the fourth feature vector X 3 After concatenation, it is input into the first double-layer convolution block, which is connected to the first upsampling layer. The output of the first upsampling layer is connected to the third feature vector X 2 After concatenation, it is input into the second double-layer convolution block, which is connected to the second upsampling layer. The output of the second upsampling layer is combined with the first feature vector X 0 The concatenated result is used as the output of the decoder.

[0021] During the training process of the tongue segmentation model based on the edge gating mechanism, the loss function used is the weighted fusion loss of the predicted segmentation mask loss and the edge loss.

[0022] The edge loss is a binary cross entropy loss function BCE calculated based on the edge output of the edge prediction module.

[0023] 2. A computer device

[0024] The computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the steps of the tongue image segmentation method based on the edge gating mechanism when executing the computer program.

[0025] 3. A computer-readable storage medium

[0026] The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the tongue image segmentation method based on the edge gating mechanism are implemented.

[0027] IV. A COMPUTER PROGRAM PRODUCT

[0028] The computer program product comprises a computer program / instruction, which, when executed by a processor, implements the steps of the tongue image segmentation method based on an edge gating mechanism.

[0029] The beneficial effects of the present invention are:

[0030] (1) The method of the present invention uses color correction and tongue positioning preprocessing operations for the original tongue image data collected by the mobile device to establish a standard tongue image data set.

[0031] (2) To address the problem of rough tongue edge extraction in tongue segmentation tasks, the method of the present invention designs a tongue segmentation method based on the edge gating mechanism, adds an edge branch to the Deeplabv3+ model as an improvement, and designs a loss function that is a weighted fusion of segmentation loss and edge loss to train the improved model, thereby improving the prediction accuracy of the model segmentation mask.

[0032] (3) The method of the present invention uses a skip connection method in the decoder to fuse the feature vector obtained by the encoder with the feature vector of the decoder, and uses a double-layer convolution and upsampling mechanism to restore the image to its original size, thereby enhancing the feature capture capability and improving the decoder performance and segmentation accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 It is a flow chart of the method of the present invention.

[0034] Figure 2 It is a schematic diagram of the structure of the tongue segmentation model based on the edge gating mechanism.

[0035] Figure 3 It is a flowchart of the preprocessing operation for the tongue sticking out image.

[0036] Figure 4 It is a schematic diagram of the structure of the gated attention layer in the edge prediction module. DETAILED DESCRIPTION

[0037] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments, but the protection scope of the present invention is not limited thereto.

[0038] like Figure 1 and Figure 3 As shown, the tongue image segmentation method based on edge gating mechanism proposed by the present invention comprises the following steps:

[0039] Step 1: Use a mobile device such as a mobile phone or camera to capture the patient's tongue-extending image under natural lighting conditions. The captured image includes partial facial information and tongue information of the patient;

[0040] Step 2: After preprocessing the collected tongue-extending image, a processed standard tongue image is obtained; the preprocessing specifically includes tongue positioning and color cast correction.

[0041] Step 2 is as follows:

[0042] Step 21: Use the YOLOv5-Lite-based target detection model to locate the tongue part in the collected tongue-protruding image, filter out the human facial information, select the center area of ​​the tongue, and obtain the tongue body image; wherein, the tongue part of the patient in the original image information is annotated, and the annotation information includes the marker category, the maximum and minimum X, Y pixel coordinates of the two-dimensional bounding box, so as to obtain the tongue-protruding image data set. The YOLOv5-Lite-based target detection model for predictive segmentation is obtained after pre-training with the tongue-protruding image data set, and the YOLOv5-Lite-based target detection model uses a rectangular box to select the tongue part.

[0043] Step 22: After color cast check of the tongue body image, a processed standard tongue image is obtained.

[0044] Step 22 is specifically as follows:

[0045] Firstly, the color cast detection method of tongue image based on equivalent circle detection method is used to calculate the color cast degree of each tongue body image. If the color cast degree of each tongue body image is greater than the preset threshold, the gray world method based on standard deviation weighting is used to perform color cast correction on the tongue body image to obtain the processed standard tongue image. Otherwise, no processing is performed.

[0046] The tongue color deviation test method based on the equivalent circle detection method is as follows:

[0047] First, the processed standard tongue image in RGB color space is converted to Lab color space via XYZ space. The conversion formula is:

[0048]

[0049] The three components X, Y, and Z of the converted XYZ space are normalized using the white point value of the D65 standard light source. The formula is as follows:

[0050]

[0051] For the normalized X, Y, and Z components, apply the nonlinear transformation f(t) and calculate the Lab component:

[0052]

[0053] t=x,y,z

[0054] L * =116·f(y)-16,

[0055] a * =500·(f(x)-f(y))

[0056] b* =200·(f(y)-f(z))

[0057] Among them, X, Y, and Z are the values ​​of the three basis vectors representing the brightness component, relative brightness value, and relative blue component value in the XYZ color space. R, G, and B are the values ​​of the three basis vectors representing the red intensity, green intensity, and blue intensity in the RGB color space. L * 、a * , b * The values ​​of the three basis vectors representing the brightness component, the color range from green to magenta, and the color range from blue to yellow in the Lab color space. x, y, and z are the normalized basis vector values.

[0058] After conversion, the distance D from the center of the equivalent circle to the origin of the ab chromaticity plane is calculated, and the radius of the equivalent circle is M′.

[0059] The calculation method is as follows

[0060]

[0061]

[0062] Among them, M and N are the width and height of the image, and the center coordinates of the equivalent circle are (d a ,d b ), the center of the equivalent circle is to the origin of the neutral axis of the ab chromaticity plane (a=0, b=0). a and M b The first radius parameter and the second radius parameter are respectively used. The tongue color deviation error K is calculated according to the distance D and the equivalent circle radius M', and K=D / M' is satisfied. Whether the tongue image is color deviated is determined according to a preset threshold.

[0063] The gray-scale world method based on standard deviation weighting is as follows: divide the image into n small blocks of 16×16, calculate the pixel standard deviation of each block, and assign weights to the image blocks according to the size of the standard deviation. The calculation formula is as follows:

[0064]

[0065] Among them, R ave (k), G ave (k) and B ave (k) are the mean values ​​of the color components of the kth image block in the RGB color space, R std (i) G std (i) and B std (i) are the standard deviations of the color components of the i-th image block, Y SDW-R , Y SDW-G and Y SDW-Bare the weighted means of the calculated R, G, and B color components respectively.

[0066] After obtaining the weighted mean of the color components, the relationship equation between the weighted mean of the color components in different RGB channels and the average RGB value is established, the channel secondary transformation coefficients u, v are obtained, and the final correction result is calculated.

[0067]

[0068]

[0069] Among them, R max , G max , B max are the maximum values ​​of the three color channels R, G, and B of the original image in the RGB color space, R ave , G ave , B ave are the average values ​​of the three color channels R, G, and B, respectively, and u R 、u G 、u B 、v R 、v G and v B These are the two secondary transform coefficients corresponding to the R, G, and B channels that need to be obtained.

[0070] Step 3: Input the processed standard tongue image into the trained tongue segmentation model based on the edge gating mechanism, and the model outputs the corresponding tongue segmentation result.

[0071] Among them, the tongue segmentation model based on the edge gating mechanism is based on the Deeplabv3+ model structure, and uses the ResNeXt model embedded with the SE attention mechanism as the encoder for feature extraction. The encoding vector of the input tongue image after being encoded by the encoder is taken as the input of the edge branch, and the gated attention layer is used to learn the weights, and the image edge information extracted by the Canny operator is integrated. Figure 2 As shown in FIG. 1 , the tongue segmentation model based on the edge gating mechanism includes a ResNeXt model embedded with an SE attention mechanism, an ASPP module, a decoder, and an edge prediction module. The ResNeXt model embedded with an SE attention mechanism is used as an encoder for feature extraction. The processed standard tongue image is used as the input of the ResNeXt model embedded with an SE attention mechanism. The ResNeXt model embedded with an SE attention mechanism extracts feature vectors X at 6 different depths. 0 , X 1 , X 2 , X 3 , X 4 , X 5 , the first eigenvector X0 , the third eigenvector X 2 , the fourth eigenvector X 3 and the sixth eigenvector X 5 are input into the edge prediction module, the sixth eigenvector X 5 After being spliced ​​with the output of the edge prediction module, it is input into the ASPP module, and the output of the ASPP module serves as the input of the decoder.

[0072] In the ResNeXt model embedded with the SE attention mechanism, when the processed standard tongue image is input, a convolution layer with a convolution kernel of 3×3 is first used to increase the number of channels from 3 to 64, and then a convolution layer Layer0 with a convolution kernel of 7×7 and four convolution layers Layer1, Layer2, Layer3, and Layer4 consisting of BottleNeck layers with numbers of 3, 4, 6, and 3 respectively are passed through to obtain the feature vectors X at different depths. 0 , X 1 , X 2 , X 3 , X 4 , X 5 .

[0073] The edge prediction module includes the Canny operator edge extraction module, fusion layer, convolution layer, residual module and gated attention layer. The first eigenvector X 0 After passing through the first convolutional layer and the first residual module, it is connected to the first gated attention layer. The third feature vector X 2 After the second convolutional layer, it is connected to the first gated attention layer. After the first gated attention layer passes through the second residual module, it is connected to the second gated attention layer. The fourth eigenvector X 3 After the third convolutional layer, it is connected to the second gated attention layer. After the second gated attention layer passes through the third residual module, it is connected to the third gated attention layer. The sixth eigenvector X 5 After the fourth convolutional layer, it is connected to the third gated attention layer. The output of the third gated attention layer is recorded as the edge output. The processed standard tongue image is also used as the input of the Canny operator edge extraction module. The output of the third gated attention layer and the output of the Canny operator edge extraction module are added and then input into the fusion layer. The output of the fusion layer is used as the output of the edge prediction module, that is, the edge prediction result is combined with the low-dimensional edge information extracted by the Canny operator, and together they are used as the final output vector edge_cat of the edge branch.

[0074] like Figure 4As shown in the figure, the gated attention layer includes a batch normalization layer, a convolution layer, a Relu activation layer, and a Sigmoid activation layer. In the two inputs of each gated attention layer, the output of the corresponding residual module is recorded as the first input feature, and the other input is recorded as the second input feature. After element-by-element addition, they are input into the batch normalization layer. The batch normalization layer is sequentially connected to the Sigmoid activation layer after passing through the fifth convolution layer, the first Relu activation layer, the sixth convolution layer, and the second Relu activation layer. The result of multiplying the second input feature by α times the output of the Sigmoid activation layer is recorded as the intermediate feature. The intermediate feature is added to the second input feature and then input into the seventh convolution layer. The output of the seventh convolution layer is used as the output of the current gated attention layer. The details are as follows:

[0075] X after 1×1 convolution layer processing 0 With X 2 After concatenation, convolution and normalization operations are performed to generate the gated attention weight α that focuses on the edge information. 1 , weights and input X 0 Perform weighted and residual connection operations to generate The formula is as follows:

[0076] α 1 =σ(C 1×1 (X 0 )||C 1×1 (X 2 ))

[0077]

[0078] Among them, || represents the concatenation of two vectors, ⊙ represents the element-by-element multiplication of two vectors, (i, j) represents the position of each pixel in the image, and w 1 represents the convolution kernel function, Represents the output features weighted by the attention score at each pixel position. Output As the input of the next gated attention layer, it is combined with X after being processed by the 1×1 convolution layer. 3 Calculate the gated attention weight α after splicing 2 , and is calculated by weighting the attention score and the residual connection Output and X after 1×1 convolution layer processing 5 After splicing, repeat the previous step and calculate the result as the edge prediction result Edge_out.

[0079] The decoder contains a double-layer convolution block and an upsampling layer. The output of the ASPP module is combined with the fourth eigenvector X 3After concatenation, it is input into the first double-layer convolution block, which is connected to the first upsampling layer. The output of the first upsampling layer is connected to the third feature vector X 2 After concatenation, it is input into the second double-layer convolution block, which is connected to the second upsampling layer. The output of the second upsampling layer is combined with the first feature vector X 0 The concatenated result is used as the output of the decoder.

[0080] In the training process of the tongue segmentation model based on the edge gating mechanism, the loss function used is the weighted fusion loss of the predicted segmentation mask loss and the edge loss. The predicted segmentation mask loss is the multi-classification cross entropy loss function CE calculated based on the output of the decoder (i.e., the segmentation mask output mask_out). The edge loss is the binary cross entropy loss function BCE calculated based on the edge output of the edge prediction module. Loss function The formula is as follows:

[0081]

[0082] in, Indicates the actual edge information of the tongue. represents the actual segmentation mask of the tongue, e represents the edge information predicted by the model, and m represents the segmentation mask predicted by the model. Represents the binary cross entropy loss value, represents the multi-classification cross entropy loss value, λ 1 With λ 2 Calculate weights for loss.

[0083] The evaluation parameters of the model are the average intersection-over-union ratio MioU and Dice coefficient of the tongue prediction mask and the true mask, which can be expressed as:

[0084]

[0085] The number of model training times is set to 300. The learning rate is warmed up in the first 5 training times, and then decayed using a learning rate strategy based on polynomial decay. The formula for decaying lr is:

[0086]

[0087] Among them, lr init is the initial learning rate, set to 0.01, iter is the current number of iterations, maxiter is the maximum number of iterations (i.e. batch_size × number of training times), power is a hyperparameter that controls the learning rate decay rate, and is set to 0.9.

[0088] In this embodiment, a total of 1063 patients' tongue-extending pictures were collected, of which 850 pictures were selected as training sets and 213 pictures were selected as validation sets. The data set was preprocessed and tongue mask was predicted, and the classic segmentation network U-Net and DeeplabV3+ were used for comparative experiments. The evaluation parameters calculated by the final tongue prediction mask and the real mask are shown in Table 1:

[0089] Table 1 is the evaluation parameter table calculated by tongue prediction mask and true mask

[0090]

[0091]

[0092] The present invention proposes a tongue image analysis method combining preprocessing, improved segmentation method and feature fusion for the original tongue image data collected by mobile devices. First, the original tongue image data is preprocessed by color correction and tongue positioning, and a standardized tongue image data set is established. In view of the problem of inaccurate tongue edge extraction in the tongue segmentation task, the present invention introduces an edge gating mechanism on the basis of the Deeplabv3+ model, adds edge branches, and designs a weighted loss function that fuses segmentation loss and edge loss, thereby improving the segmentation mask prediction accuracy of the model. In addition, the present invention uses jump connection technology to fuse the feature vectors obtained by the encoder and decoder, thereby enhancing the feature capture capability of the model and improving the segmentation accuracy.

[0093] Finally, it should be noted that the above embodiments and explanations are only used to illustrate the technical solution of the present invention rather than to limit it. Those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the spirit and scope disclosed in the technical solution of the present invention, which should be included in the scope of protection of the claims of the present invention.

Claims

1. A tongue image segmentation method based on edge gating mechanism, characterized in that: The following steps are involved: Step 1: Collect the patient's tongue-extending image; Step 2: After preprocessing the collected tongue-extending image, a processed standard tongue image is obtained; Step 3: Input the processed standard tongue image into the trained tongue segmentation model based on the edge gating mechanism, and the model outputs the corresponding tongue segmentation result.

2. The tongue image segmentation method based on edge gating mechanism according to claim 1, characterized in that: The step 2 is specifically as follows: After tongue positioning and color cast correction are performed on the collected tongue-extending image, a processed standard tongue image is obtained.

3. The tongue image segmentation method based on edge gating mechanism according to claim 1, characterized in that: The step 2 is specifically as follows: Step 21: Use the YOLOv5-Lite-based target detection model to locate the tongue body in the collected tongue-extending image to obtain the tongue body image; Step 22: After color cast check of the tongue body image, a processed standard tongue image is obtained.

4. The tongue image segmentation method based on edge gating mechanism according to claim 3, characterized in that: The step 22 is specifically as follows: Firstly, the color cast detection method of tongue image based on equivalent circle detection method is used to calculate the color cast degree of each tongue body image. If the color cast degree of each tongue body image is greater than the preset threshold, the tongue body image is subjected to color cast verification to obtain the processed standard tongue image, otherwise it is not processed.

5. The tongue image segmentation method based on edge gating mechanism according to claim 1, characterized in that: The tongue segmentation model based on the edge gating mechanism includes a ResNeXt model embedded with an SE attention mechanism, an ASPP module, a decoder and an edge prediction module, wherein the ResNeXt model embedded with the SE attention mechanism serves as an encoder for feature extraction, and the processed standard tongue image serves as the input of the ResNeXt model embedded with the SE attention mechanism. The ResNeXt model embedded with the SE attention mechanism extracts and obtains feature vectors X0, X1, X2, X3, X4, and X5 at 6 different depths. The first feature vector X0, the third feature vector X2, the fourth feature vector X3, and the sixth feature vector X5 are all input into the edge prediction module, and the sixth feature vector X5 is spliced ​​with the output of the edge prediction module and then input into the ASPP module, and the output of the ASPP module serves as the input of the decoder.

6. The tongue image segmentation method based on edge gating mechanism according to claim 5, characterized in that: The edge prediction module includes a Canny operator edge extraction module, a fusion layer, a convolution layer, a residual module and a gated attention layer. The first eigenvector X0 is connected to the first gated attention layer after passing through the first convolution layer and the first residual module in sequence. The third eigenvector X2 is connected to the first gated attention layer after passing through the second convolution layer. The first gated attention layer is connected to the second gated attention layer after passing through the second residual module. The fourth eigenvector X3 is connected to the second gated attention layer after passing through the third convolution layer. The second gated attention layer is connected to the third gated attention layer after passing through the third residual module. The sixth eigenvector X5 is connected to the third gated attention layer after passing through the fourth convolution layer. The output of the third gated attention layer is recorded as the edge output. The processed standard tongue image is also used as the input of the Canny operator edge extraction module. The output of the third gated attention layer and the output of the Canny operator edge extraction module are added and then input into the fusion layer. The output of the fusion layer is used as the output of the edge prediction module.

7. The tongue image segmentation method based on edge gating mechanism according to claim 6, characterized in that: The gated attention layer includes a batch normalization layer, a convolution layer, a Relu activation layer and a Sigmoid activation layer. Among the two inputs of each gated attention layer, the output of the corresponding residual module is recorded as the first input feature, and the other input is recorded as the second input feature. After element-by-element addition, they are input into the batch normalization layer. The batch normalization layer is sequentially connected to the Sigmoid activation layer after passing through the fifth convolution layer, the first Relu activation layer, the sixth convolution layer and the second Relu activation layer. The result of multiplying the second input feature by α times the output of the Sigmoid activation layer is recorded as the intermediate feature. The intermediate feature is added to the second input feature and then input into the seventh convolution layer. The output of the seventh convolution layer is used as the output of the current gated attention layer.

8. The tongue image segmentation method based on edge gating mechanism according to claim 5, characterized in that: The decoder includes a double-layer convolution block and an upsampling layer. The output of the ASPP module is spliced ​​with the fourth eigenvector X3 and then input into the first double-layer convolution block. The first double-layer convolution block is connected to the first upsampling layer. The output of the first upsampling layer is spliced ​​with the third eigenvector X2 and then input into the second double-layer convolution block. The second double-layer convolution block is connected to the second upsampling layer. The output of the second upsampling layer is spliced ​​with the first eigenvector X0 as the output of the decoder.

9. The tongue image segmentation method based on edge gating mechanism according to claim 5, characterized in that: During the training process of the tongue segmentation model based on the edge gating mechanism, the loss function used is the weighted fusion loss of the predicted segmentation mask loss and the edge loss.

10. The tongue image segmentation method based on edge gating mechanism according to claim 9, characterized in that: The edge loss is a binary cross entropy loss function BCE calculated based on the edge output of the edge prediction module.

Images