A traditional chinese medicine tongue image segmentation method for mobile terminal
By employing feature extraction, edge enhancement, and global enhancement techniques from a tongue image segmentation network, the problem of inaccurate tongue boundary segmentation was solved, achieving accurate segmentation and stability on mobile terminals while reducing computational complexity.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- BEIJING UNIV OF TECH
- Filing Date
- 2026-02-13
- Publication Date
- 2026-06-05
AI Technical Summary
Existing tongue image segmentation methods cannot accurately determine the boundaries of the tongue body, and deep learning models have high computational complexity on mobile terminals, making them difficult to adapt to devices with limited computing power and storage resources.
A method based on tongue image segmentation network is adopted, which uses feature extraction, edge enhancement and global enhancement techniques, combined with dilated asymmetric convolution and spatial shifting techniques to achieve accurate segmentation of the tongue region.
Precise segmentation of the tongue boundary was achieved on mobile terminals, improving segmentation accuracy and stability, reducing the number of model parameters, and adapting to resource-constrained devices.
Smart Images
Figure CN122156229A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of image processing technology, and in particular to a method for segmenting traditional Chinese medicine tongue images for mobile terminals. Background Technology
[0002] Tongue diagnosis in Traditional Chinese Medicine (TCM) is an important component of TCM clinical diagnosis. Tongue image segmentation, as a fundamental step in the intelligent analysis of tongue diagnosis information, directly affects the accuracy of subsequent analysis of tongue color, tongue coating, and tongue shape.
[0003] In recent years, with the rapid development of deep learning, tongue image segmentation methods based on deep network models have made significant progress, with segmentation accuracy far exceeding that of traditional methods. However, due to the high similarity in color and texture features between the tongue edge region and the lips, face, etc., existing segmentation methods cannot accurately determine the tongue boundary when segmenting tongue images. Summary of the Invention
[0004] In view of this, the present invention proposes a traditional Chinese medicine tongue image segmentation method for mobile terminals, which solves the problem that traditional tongue image segmentation methods cannot accurately determine the boundaries of the tongue body when segmenting tongue images.
[0005] On one hand, embodiments of the present invention provide a method for segmenting tongue images in traditional Chinese medicine for mobile terminals, including performing the following steps based on a tongue image segmentation network: Feature extraction was performed on the tongue image to obtain the features of the first tongue region and the second tongue region at different scales; Based on the features of the first tongue region, edge enhancement is performed on the features of the second tongue region to obtain edge-enhanced features of the second tongue region. The second tongue region features are globally enhanced by spatial shifting to obtain globally enhanced second tongue region features. The edge-enhanced second tongue region features and the globally enhanced second tongue region features are fused to obtain the tongue region segmentation result of the tongue image.
[0006] In some implementations, feature extraction is performed on the tongue image to obtain first tongue region features and second tongue region features at different scales, including: Feature extraction is performed on the tongue image to obtain the features of the first tongue body region; Based on different expansion rates, feature extraction and splicing of the first tongue region features are performed to obtain the second tongue region features.
[0007] In some implementations, feature extraction and splicing of the first tongue region features based on different expansion rates are performed to obtain the second tongue region features, including: Based on the first dilation rate, dilation convolution is performed on the features of the first tongue region to obtain the global semantic features of the first tongue region. Based on the second dilation rate, dilation convolution is performed on the features of the first tongue region to obtain the edge detail features of the first tongue region. The global semantic features and edge detail features of the first tongue region are concatenated to obtain the features of the second tongue region.
[0008] In some implementations, extracting and stitching features of the first tongue region based on different expansion rates to obtain features of the second tongue region includes: extracting and stitching features of the first tongue region layer by layer based on different expansion rates according to a preset number of layers to obtain multiple features of the second tongue region. The step of globally enhancing the second tongue region features through spatial shifting to obtain globally enhanced second tongue region features includes: globally enhancing the second tongue region features obtained from the last layer of feature extraction through spatial shifting to obtain globally enhanced second tongue region features.
[0009] In some implementations, edge enhancement is performed on the second tongue region features based on the first tongue region features to obtain edge-enhanced second tongue region features, including: Edge feature enhancement is performed on the first tongue region features to obtain an edge-enhanced tongue region feature map; Based on the edge-enhanced tongue region feature map and the second tongue region feature, the edge-enhanced second tongue region feature is obtained.
[0010] In some embodiments, edge feature enhancement is performed on the first tongue region features to obtain an edge-enhanced tongue region feature map, including: Different edge detection operators are used to perform edge detection on the features of the first tongue region to obtain different tongue contour features; Different tongue contour features are spliced together to obtain the spliced tongue contour features; Using a target region attention mechanism, the edge-enhanced tongue region feature map is generated based on the spliced tongue contour features.
[0011] In some implementations, the second tongue region features are globally enhanced by spatial shifting to obtain globally enhanced second tongue region features, including: The second tongue region features are uniformly divided into multiple groups of features along the channel dimension, and each group of features is cyclically shifted to obtain globally enhanced second tongue region features.
[0012] In some implementations, cyclically shifting each set of features to obtain globally enhanced second tongue region features includes: Each set of features is cyclically shifted along the height and width directions respectively to obtain the shifted features in the height direction and the shifted features in the width direction; By using a gated activation function, the globally enhanced second tongue region features are obtained based on the displacement features in the height direction and the displacement features in the width direction.
[0013] In some implementations, the method further includes the following tongue image segmentation network training steps: Construct a labeled tongue image segmentation dataset; The tongue image segmentation network is trained using the tongue image segmentation dataset and a composite loss function to obtain a trained tongue image segmentation network.
[0014] In some implementations, training the tongue image segmentation network based on the tongue image segmentation dataset and the composite loss function includes: The tongue images in the tongue image segmentation dataset are input into the tongue image segmentation network to obtain edge-enhanced second tongue region features, globally enhanced second tongue region features, and tongue region segmentation results. Based on the composite loss function, the composite loss is determined according to the edge-enhanced second tongue region features, the globally enhanced second tongue region features, and the tongue region segmentation results; The parameters of the tongue image segmentation network are updated based on the composite loss to train the tongue image segmentation network.
[0015] In some implementations, the composite loss is determined based on edge-enhanced second tongue region features, globally enhanced second tongue region features, and tongue region segmentation results, including: Based on the edge-enhanced features of the second tongue region, the target attention loss is determined. Based on the globally enhanced features of the second tongue region, a layer-by-layer deep supervision loss is determined; Based on the tongue region segmentation results, the main segmentation loss is determined; Based on the main segmentation loss, the target attention loss, and the layer-by-layer deep supervision loss, a composite loss is determined.
[0016] The present invention has at least the following beneficial effects: This invention provides a method for segmenting tongue images in Traditional Chinese Medicine (TCM) for mobile terminals. The method involves extracting features from a tongue image to obtain first and second tongue region features at different scales. Based on the first tongue region features, edge enhancement is applied to the second tongue region features to obtain edge-enhanced second tongue region features. Then, global enhancement is performed on the second tongue region features through spatial shifting to obtain globally enhanced second tongue region features. Finally, feature fusion is performed on the edge-enhanced and globally enhanced second tongue region features to obtain the tongue region segmentation result. This technical solution achieves accurate segmentation of the tongue region, improves the accuracy of tongue boundary segmentation, and maintains good segmentation stability under complex background conditions. Attached Figure Description
[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other embodiments can be obtained based on these drawings without creative effort.
[0018] Figure 1 A flowchart illustrating a traditional Chinese medicine tongue image segmentation method for mobile terminals provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of the structure of a tongue image segmentation network provided in an embodiment of the present invention; Figure 3 This is a flowchart of generating second tongue region features in a traditional Chinese medicine tongue image segmentation method for mobile terminals provided in an embodiment of the present invention. Figure 4 A schematic diagram of a dilated asymmetric convolutional block in a tongue image segmentation network provided in an embodiment of the present invention; Figure 5 This is a schematic diagram of the structure of another tongue image segmentation network provided in an embodiment of the present invention; Figure 6 A flowchart illustrating the generation of edge-enhanced tongue region features in a traditional Chinese medicine tongue image segmentation method for mobile terminals provided in this embodiment of the invention; Figure 7 This is a schematic diagram of the structure of an edge feature enhancement unit in a tongue image segmentation network provided in an embodiment of the present invention; Figure 8 This is an iterative diagram of the target attention map during the training of a tongue image segmentation network in a traditional Chinese medicine tongue image segmentation method for mobile terminals provided in an embodiment of the present invention. Figure 9This is a flowchart illustrating a method for global context enhancement of tongue region features in a traditional Chinese medicine tongue image segmentation method for mobile terminals provided in an embodiment of the present invention. Figure 10 for Figure 9 The diagram shows a process for spatially shifting tongue region features in a global context enhancement process. Figure 11 for Figure 9 The diagram shows the process of adaptively filtering the shifted tongue region features in the global context enhancement process for tongue region features. Figure 12 A flowchart of another TCM tongue image segmentation method for mobile terminals provided in an embodiment of the present invention; Figure 13 This is a schematic diagram of a tongue image segmentation dataset in a traditional Chinese medicine tongue image segmentation method for mobile terminals provided in an embodiment of the present invention; Figure 14 This is a schematic diagram of a tongue region segmentation result obtained by the TCM tongue image segmentation method for mobile terminals provided in this embodiment of the invention. Detailed Implementation
[0019] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be further described in detail below with reference to specific examples and the accompanying drawings.
[0020] It should be noted that all uses of "first" and "second" in the embodiments of the present invention are for the purpose of distinguishing two entities or parameters with the same name but different names. It is clear that "first" and "second" are only for the convenience of expression and should not be construed as limiting the embodiments of the present invention. Subsequent embodiments will not explain this in detail.
[0021] In recent years, with the rapid development of deep learning, significant progress has been made in tongue image segmentation methods based on deep network models. However, these methods still face the following challenges when segmenting tongue images: 1) Because the edge region of the tongue is highly similar to the lips and face in terms of color and texture features, problems such as the inability to accurately determine the tongue boundary and background intrusion occur during the segmentation process of the tongue region in the tongue image. 2) The U-shaped symmetrical encoder-decoder is used as the basic framework. The model structure is complex, the number of parameters is large, and the computational complexity is high. It is difficult to adapt to mobile terminals or edge devices with limited computing power and storage resources.
[0022] In view of this, in order to solve at least one of the above technical problems, this invention proposes a TCM tongue image segmentation method for mobile terminals, which can achieve high segmentation accuracy and accurate tongue segmentation results with extremely low model parameter count (around 0.035M), meeting the usage needs of various mobile terminal scenarios such as mobile medical scenarios.
[0023] The first aspect of this invention provides a method for segmenting TCM tongue images for mobile terminals, which is implemented based on a tongue image segmentation network.
[0024] like Figure 1 and Figure 2 As shown, the TCM tongue image segmentation method for mobile terminals provided by the present invention includes steps S10 to S40. The tongue image segmentation network includes: an encoder 10, an edge feature enhancement unit 20, a global context enhancement unit 30, and a feature fusion and image segmentation unit 40. The encoder 10 in the tongue image segmentation network is configured to perform step 10, the edge feature enhancement unit 20 in the tongue image segmentation network is configured to perform step 20, the global context enhancement unit 30 in the tongue image segmentation network is configured to perform step 30, and the feature fusion and image segmentation unit 40 in the tongue image segmentation network is configured to perform step 40.
[0025] S10. Extract features from the tongue image to obtain the first tongue region features and the second tongue region features at different scales.
[0026] Specifically, a tongue image can be acquired, and a convolution operation can be performed on the acquired tongue image to obtain the first tongue region feature. Then, the first tongue region feature is downsampled to obtain the tongue region feature with a reduced scale. Subsequently, the tongue region feature with a reduced scale is subjected to a dilated asymmetric convolution operation to obtain the second tongue region feature.
[0027] In this embodiment of the invention, during the process of obtaining the second tongue region feature from the first tongue region feature, at least one downsampling and dilated asymmetric convolution operation can be performed. For example, in some embodiments, the first tongue region feature can be downsampled and dilated convolutioned once to obtain one second tongue region feature. For example, in some embodiments, the first tongue region feature can be downsampled and dilated asymmetric convolutioned multiple times in a progressive manner across multiple layers to obtain multiple second tongue region features.
[0028] In this embodiment of the invention, by downsampling the features of the first tongue region and performing dilated asymmetric convolution, global and local tongue feature information in the tongue image can be extracted better. The effect is even better when the features of the first tongue region are downsampled and dilated asymmetricly convolved multiple times.
[0029] S20. Based on the features of the first tongue region, the features of the second tongue region are edge-enhanced to obtain the edge-enhanced features of the second tongue region.
[0030] Specifically, based on the attention mechanism, edge feature enhancement can be performed on the features of the first tongue region to obtain an edge-enhanced tongue region feature map. This edge-enhanced tongue region feature map has fine tongue features. Based on this edge-enhanced tongue region feature map, the features of the second tongue region can be enhanced to obtain an edge-enhanced second tongue region feature map. This can effectively suppress background interference and enhance the edge information in the features.
[0031] It should be noted that, in the embodiments of the present invention, in the process of obtaining the second tongue region feature through the first tongue region feature, whether one second tongue region feature or multiple second tongue region features are obtained, edge enhancement is required for each second tongue region feature.
[0032] S30. Globally enhance the features of the second tongue region by spatial shifting to obtain the globally enhanced features of the second tongue region.
[0033] Specifically, the features of the second tongue region can be evenly divided into multiple groups of features along the channel dimension, and each group of features can be cyclically shifted in the height and width directions. This results in globally enhanced features of the second tongue region. Based on this, information interaction between features at different spatial locations can be achieved without introducing additional convolutional computation, establishing multi-directional contextual dependencies for the tongue region features, and enhancing the global contextual information of the tongue region features.
[0034] It should be noted that when multiple second tongue region features are obtained through the first tongue region features, the deepest second tongue region features can be globally enhanced by spatial shifting to obtain globally enhanced second tongue region features. This allows for feature fusion of each edge-enhanced second tongue region feature based on the globally enhanced second tongue region features.
[0035] S40. Perform feature fusion on the edge-enhanced second tongue region features and the globally enhanced second tongue region features to obtain the tongue region segmentation result of the tongue image.
[0036] Specifically, by fusing and upsampling the edge-enhanced and globally enhanced features of the second tongue region, the tongue features can be restored to their original scale, resulting in the tongue region segmentation at the original size.
[0037] This invention extracts features from a tongue image to obtain first tongue region features and second tongue region features of different scales. Based on the first tongue region features, edge enhancement is performed on the second tongue region features to obtain edge-enhanced second tongue region features. Feature fusion is performed on the edge-enhanced second tongue region features and the globally enhanced second tongue region features to obtain the tongue region segmentation result of the tongue image. Finally, global enhancement is performed on the second tongue region features through spatial shifting to obtain globally enhanced second tongue region features. This technical solution achieves accurate segmentation of the tongue region, improves the accuracy of tongue boundary segmentation, and maintains good segmentation stability under complex background conditions.
[0038] In some embodiments, feature extraction of a tongue image to obtain first tongue region features and second tongue region features at different scales includes: extracting features from the tongue image to obtain first tongue region features; and extracting and stitching features from the first tongue region features based on different dilation rates to obtain second tongue region features.
[0039] In some embodiments, extracting and stitching features from the first tongue region based on different expansion rates to obtain second tongue region features includes: extracting and stitching features from the first tongue region layer by layer based on different expansion rates according to a preset number of layers to obtain multiple second tongue region features. When multiple layers of features are extracted and stitched from the first tongue region to obtain multiple second tongue region features, globally enhancing the second tongue region features through spatial shift to obtain globally enhanced second tongue region features includes: globally enhancing the second tongue region features obtained from the last layer of feature extraction through spatial shift to obtain globally enhanced second tongue region features.
[0040] In some embodiments, such as Figure 2 and Figure 3 As shown, the specific implementation of extracting and stitching features of the first tongue region based on different dilation rates to obtain the features of the second tongue region includes steps S310 to S330. The encoder 10 may include a dilated asymmetric convolution block (DACB) 11, which is configured to perform steps S310 to S330.
[0041] S310. Perform dilation convolution on the features of the first tongue region based on the first dilation rate to obtain the global semantic features of the first tongue region.
[0042] S320. Based on the second dilation rate, dilate and convolve the features of the first tongue region to obtain the edge detail features of the first tongue region.
[0043] S330. The global semantic features and edge detail features of the first tongue region are concatenated to obtain the features of the second tongue region.
[0044] In this embodiment of the invention, the first dilation rate is greater than the second dilation rate. The second dilation rate is used to extract the edge detail features of the tongue region, and the first dilation rate is used to extract the global semantic features of the tongue region. By extracting the edge detail features and the global semantic features, accurate segmentation of the tongue region is achieved, and background interference can also be suppressed, further improving the accuracy of tongue region segmentation.
[0045] In some embodiments, such as Figure 4 As shown, the dilated asymmetric convolution block 11 includes a first dilated convolution branch 111 and a second dilated convolution branch 112. The first dilated convolution branch 111 is a global context-aware branch, and the second dilated convolution branch 112 is a local detail extraction branch. Through the dual-branch design of "low dilation rate to capture details + high dilation rate to obtain the global context," the network can simultaneously extract local detail features and global context information. Furthermore, using this block as the basic unit, an ultra-lightweight segmentation network is constructed, which can maintain good local-global feature extraction and expression capabilities with extremely low model parameter count. Specifically, the local detail extraction branch captures local detail information such as tongue edges by setting a low dilation rate asymmetric convolution operation; the global context-aware branch expands the receptive field by setting a high dilation rate asymmetric convolution operation, thereby obtaining global semantic information of the tongue region, achieving the goal of "low dilation rate to capture details and high dilation rate to obtain the global context." This enables accurate segmentation of the tongue region, suppresses background interference, and improves the accuracy of tongue region segmentation.
[0046] In some embodiments, a schematic diagram of the tongue image segmentation network is shown below. Figure 5 As shown, in Figure 5 The tongue image segmentation network shown has an encoder consisting of four stages (E1 to E4). In stage E1, basic convolutional blocks are used to extract features from the input tongue image to obtain tongue region features. The obtained tongue region features are then downsampled to obtain the first tongue region features (F1). In stages E2 to E4, dilated asymmetric convolutional blocks 11 are used as basic units to sequentially perform dilated asymmetric convolution and downsampling on the first tongue region features to obtain the second tongue region features (F2 to F4) corresponding to each encoding stage. This achieves the goal of "low dilation rate to capture details and high dilation rate to obtain the global picture", thereby realizing accurate segmentation of the tongue region, suppressing background interference, and improving the accuracy of tongue region segmentation.
[0047] In some embodiments, the process of enhancing the features of a second tongue region based on the features of a first tongue region to obtain an edge-enhanced second tongue region feature may include: enhancing the features of the first tongue region to obtain an edge-enhanced tongue region feature map; and obtaining the edge-enhanced second tongue region feature based on the edge-enhanced tongue region feature map and the second tongue region feature.
[0048] In this embodiment, edge detection is performed on the features of the first tongue region using different edge detection operators to obtain the edge features of the first tongue region in different dimensions. Then, deconvolution-convolution-activation enhancement operations are performed on the edge features of the first tongue region in different dimensions to obtain an edge-enhanced tongue region feature map. Subsequently, the edge-enhanced tongue region feature map is used to enhance the features of the second tongue region to obtain an edge-enhanced second tongue region feature.
[0049] It should be noted that when there are multiple features in the second tongue region, each feature in the second tongue region needs to be enhanced based on the edge-enhanced tongue region feature map to obtain the corresponding edge-enhanced second tongue region feature.
[0050] This embodiment enhances the edge information in the tongue features and effectively suppresses background interference through the above scheme, thereby achieving accurate segmentation of the tongue region and improving the accuracy of tongue region segmentation.
[0051] In some embodiments, such as Figure 6 As shown, edge feature enhancement of the first tongue region features to obtain an edge-enhanced tongue region feature map may include steps S610 to S630.
[0052] S610. By using different edge detection operators to perform edge detection on the features of the first tongue region, different tongue contour features are obtained.
[0053] S620. Different tongue contour features are spliced together to obtain the spliced tongue contour features.
[0054] S630. Through the target region attention mechanism, an edge-enhanced tongue region feature map is generated based on the spliced tongue contour features.
[0055] Edge detection operators can include Laplacian, Sobelx, and Sobely edge detection operators. These operators are used to perform edge detection on the first tongue region features, resulting in different tongue contour features. These different tongue contour features are then concatenated to obtain a coarse tongue contour feature map. Subsequently, a target region attention mechanism is used to generate a refined target attention map (i.e., an edge-enhanced tongue region feature map) under the constraint of Ground Truth (real information).
[0056] This embodiment enhances the edge information in the tongue features and effectively suppresses background interference through the above scheme, thereby achieving accurate segmentation of the tongue region and improving the accuracy of tongue region segmentation.
[0057] In some embodiments, such as Figure 2 The edge feature enhancement unit 20 shown is configured to perform edge enhancement on the features of the second tongue region based on the features of the first tongue region, so as to obtain the edge-enhanced features of the second tongue region.
[0058] Specifically, such as Figure 2 The edge feature enhancement unit 20 shown may include a tongue contour generation module and a target attention map generation module. The tongue contour generation module is configured to perform steps S610-S620 to obtain a coarse tongue contour feature map based on the first tongue region features. The target attention map generation module is configured to perform step S630 to obtain a fine target attention map (i.e., an edge-enhanced tongue region feature map) based on the coarse tongue contour feature map (i.e., the stitched tongue contour features).
[0059] More specifically, the specific structure of the edge feature enhancement unit can be as follows: Figure 7 As shown below, in conjunction with Figure 7 The process of generating the edge-enhanced tongue region feature map is explained.
[0060] like Figure 7 As shown, a rough outline feature map of the tongue can be generated using Laplacian, Sobelx, and Sobely edge detection operators. The calculation process can be represented as follows:
[0061]
[0062]
[0063] and These represent the horizontal and vertical kernels of the Sobel operator, respectively. The Laplace nucleus is represented in the form:
[0064] Subsequently, using a target region attention mechanism, the previously obtained tongue contour feature map was processed. Under the constraint of Ground Truth (real information), a refined target attention map is generated. The specific generation process is as follows: First, a deconvolution operation is performed on the tongue contour feature map to obtain a size of... of Next, proceed Convolution, which performs inter-channel interactions without changing the size and dimensions of the feature map, yields... Then, a target attention map is generated using Sigmoid activation. It can be expressed by the formula:
[0065]
[0066]
[0067] During the training of the tongue image segmentation network, the generated target attention map It will be refined step by step under the constraints of Ground Truth, and the iterative process will be as follows: Figure 8 As shown.
[0068] In embodiments of the present invention, such as Figure 7 The edge feature enhancement unit shown can be applied to, for example... Figure 2 The tongue image segmentation network shown can also be applied to, for example... Figure 5 The tongue image segmentation network shown can also be applied to tongue image segmentation networks with other structures, without specific limitations here.
[0069] In some embodiments, such as Figure 5 The tongue image segmentation network shown has an encoder consisting of four stages (E1 to E4). The first tongue region image features obtained in stage E1 can be processed as follows: Figure 7 The edge feature enhancement unit shown obtains the target attention map through the edge detection process and the target attention map generation process under the target region attention mechanism described above. Through this detailed target attention map It can segment features at different scales in tongue image networks. (i.e., F2~F4 obtained from E2~E4 respectively) are weighted and enhanced to obtain the edge-enhanced features of the second tongue region. The specific expression for the weighted enhancement operation is:
[0070] in, This indicates a pixel-by-pixel multiplication operation.
[0071] In this embodiment, the above-mentioned scheme can effectively suppress background interference and enhance the edge information in the features, thereby achieving accurate segmentation of the tongue region and further improving the accuracy of tongue region segmentation.
[0072] In some embodiments, such as Figure 2 The global context enhancement unit 30 in the tongue image segmentation network shown is configured to globally enhance the features of the second tongue region through spatial shifting, thereby obtaining globally enhanced features of the second tongue region. A specific implementation of globally enhancing the features of the second tongue region through spatial shifting may include: uniformly dividing the features of the second tongue region into multiple groups along the channel dimension, and cyclically shifting each group of features to obtain globally enhanced features of the second tongue region.
[0073] Specifically, assuming the number of channels in the second tongue feature region is C, it can be divided into K groups. After division, each group of tongue region features has C / K channels. The tongue region features of each group are cyclically shifted along different directions. After the shift is completed, the shifted features are adaptively filtered, thereby further improving the accuracy of tongue image segmentation.
[0074] It should be noted that, in this embodiment of the invention, the global context enhancement unit 30 is embedded in the deepest layer of the tongue image segmentation network, and is used to perform global context enhancement on the high-level features output by the encoder. That is, when multiple layers of feature extraction and concatenation are performed on the first tongue region features to obtain multiple second tongue region features, the global context enhancement unit 30 is configured to perform global enhancement on the second tongue region features obtained from the last layer of feature extraction through spatial shifting, thereby obtaining globally enhanced second tongue region features. For example, in... Figure 5 In the tongue image segmentation network shown, the second tongue region feature F4 obtained in stage E4 can be globally enhanced to obtain the globally enhanced second tongue region feature. The globally enhanced second tongue region feature is then fused and upsampled with each of the second tongue region features F2 to F4 obtained in stages E2 to E4 to obtain the tongue region segmentation result.
[0075] In some embodiments, a specific implementation of cyclically shifting each group of features to obtain globally enhanced second tongue region features may include: cyclically shifting each group of features along the height and width directions respectively to obtain shifted features in the height direction and shifted features in the width direction; and obtaining globally enhanced second tongue region features based on the shifted features in the height direction and the shifted features in the width direction using a gated activation function.
[0076] In this embodiment, spatial shifting operations are used to achieve information interaction between features at different spatial locations without introducing additional convolutional computations. This establishes multi-directional contextual dependencies, enhances the model's ability to capture global semantic information of the tongue, and improves the segmentation accuracy of the tongue region. By using a gated activation function to adaptively filter shifted features in different directions, the response of non-tongue regions and background noise is suppressed. While preserving spatial detail information, the network's ability to focus on features of the tongue target region is enhanced.
[0077] In some embodiments, reference may also be made to Figure 9-11 Each set of features is cyclically shifted to obtain the globally enhanced second tongue region features.
[0078] First, a channel segmentation strategy is adopted to divide the feature tensor of the second tongue region. (B is the batch size,) (where C is the number of spatial locations and C is the number of channels) is uniformly divided along the channel dimension into K Groups, each group contains the following number of channels: dim :
[0079] Next, perform a cyclic shift operation on each set of features:
[0080] in In dimension d Displacement operator on, These correspond to the height and width directions, respectively. This operation is equivalent to a convolution operation with a shift operator as its kernel in discrete space, specifically achieving multi-directional context modeling through two-step shifting: first along the height direction (… d=h Displacement is performed along the width direction to establish contextual dependencies in the vertical direction; then along the width direction ( d= w The displacement is performed to establish horizontal correlation, and finally, the spatial artifacts generated during the displacement process are eliminated through boundary clipping operations to ensure the spatial structural integrity of the features.
[0081] Finally, the SiLU gated activation function is used for adaptive feature selection. The linear transformation part preserves the spatial structure information of the features, while the gate weights suppress the responses of non-tongue regions and background noise. This enhances the network's ability to focus on tongue target region features while retaining spatial detail. Since both the spatial shifting and gated fusion operations are lightweight, this module can model long-distance dependencies in the image with extremely low computational overhead. Embedding it into the highest layer of the network enhances the model's ability to capture global semantic information of the tongue, thereby improving segmentation accuracy.
[0082] In some embodiments, such as Figure 12 As shown, the TCM tongue image segmentation method for mobile terminals provided in this embodiment of the invention may include, in addition to the tongue image segmentation steps described in steps S10 to S40, the tongue image segmentation network training steps described in steps S6 and S8.
[0083] S6. Construct a tongue image segmentation dataset.
[0084] Specifically, the tongue image segmentation dataset used in this embodiment of the invention can be formed by integrating public datasets and self-built datasets, covering different lighting conditions, environments and devices, providing diverse and high-quality data support for network training, and improving the generalization ability of the segmentation model.
[0085] The acquisition environment can include both closed and open environments. In a closed environment, multiple models of TCM tongue image acquisition and analysis instruments are used to acquire images under controlled conditions, ensuring the standardization and clarity of the acquired tongue images and avoiding image quality fluctuations due to equipment differences. In an open environment, tongue images are acquired using various digital acquisition devices (such as digital cameras and smartphones) under different natural lighting conditions, such as indoors or outdoors.
[0086] The acquired tongue images were manually annotated by professional physicians with extensive clinical experience in TCM tongue diagnosis, clearly defining the boundaries between the tongue and background regions such as the face, lips, and teeth, generating a tongue mask. This tongue mask can be used as ground truth information to supervise the input tongue images during the subsequent training of the tongue image segmentation network. The annotation process strictly followed unified standards to ensure the accuracy and consistency of the annotation results, providing reliable supervisory information for model training.
[0087] Partial samples from the tongue image segmentation dataset, such as Figure 13 As shown, the tongue image segmentation dataset can be divided into a training set and a test set in a 7:3 ratio for training and testing the tongue image segmentation network.
[0088] S8. Train the tongue image segmentation network based on the tongue image segmentation dataset and the composite loss function to obtain a trained tongue image segmentation network.
[0089] Specifically, tongue images from the tongue image segmentation dataset can be input into a tongue image segmentation network. This network segments the tongue image based on the segmentation scheme described in steps S10 to S40, obtaining segmentation information such as first tongue region features and second tongue region features at different scales, edge-enhanced second tongue region features, globally enhanced second tongue region features, and tongue region segmentation results.
[0090] Subsequently, based on the composite loss function, the loss can be calculated on the edge-enhanced second tongue region features, the globally enhanced second tongue region features, and the tongue region segmentation results to obtain the composite loss. This ensures that the tongue region segmentation accuracy, boundary details, and background suppression are synergistically optimized while maintaining the model's lightweight nature.
[0091] In some embodiments, the specific implementation of training the tongue image segmentation network based on the tongue image segmentation dataset and the composite loss function includes: inputting tongue images from the tongue image segmentation dataset into the tongue image segmentation network to obtain edge-enhanced second tongue region features, globally enhanced second tongue region features, and tongue region segmentation results; determining the composite loss based on the composite loss function, according to the edge-enhanced second tongue region features, the globally enhanced second tongue region features, and the tongue region segmentation results; and updating the parameters of the tongue image segmentation network based on the composite loss to train the tongue image segmentation network.
[0092] In this embodiment, the composite loss function can include three parts: target attention loss, layer-by-layer deep supervision loss, and main segmentation loss. The target attention loss can be obtained based on the edge-enhanced second tongue region features and the tongue mask corresponding to the input tongue image. The layer-by-layer deep supervision loss can be obtained based on the second tongue region features and the tongue mask corresponding to the input tongue image. The main segmentation loss can be obtained based on the tongue region segmentation result and the tongue mask corresponding to the input tongue image. The composite loss function is the sum of the target attention loss, layer-by-layer deep supervision loss, and main segmentation loss. The parameters of the tongue image segmentation network are updated based on this composite loss to continuously optimize the tongue image segmentation network and achieve synergistic optimization of tongue region segmentation accuracy, boundary details, and background suppression.
[0093] In some embodiments, a schematic diagram of the tongue image segmentation network is shown below. Figure 5 As shown, the training and inference process of the tongue image segmentation network is as follows.
[0094] Before training, all training samples were preprocessed uniformly (size adjusted to 256×256, pixel values standardized). Data augmentation operations such as flipping, rotating, and scaling were used to expand the training sample size. Training was performed using the AdamW optimizer with a learning rate of 0.003, Epoch=120, and Batch Size=10.
[0095] During training, a layer-by-layer supervision strategy is adopted for network training. That is, each layer uses a composite loss function as the objective function for optimization supervision, so as to achieve synergistic optimization of tongue region segmentation accuracy, boundary details and background suppression while ensuring the model is lightweight.
[0096] The total loss function is defined as shown in formula (11):
[0097] in: The main segmentation loss is fused with the structural similarity loss. ), binary cross-entropy loss ( ) and shape consistency loss ( Simultaneously optimize the integrity and boundary clarity of the tongue region; this main segmentation loss... The formula is:
[0098] The target attention loss is the loss applied to obtain a refined target attention map, and its formula is:
[0099] The deep supervision loss is applied layer by layer, which is a differential loss applied to the semantic differences of features in stages E2 to E4. The formula is as follows:
[0100] Considering the semantic differences between features at different layers, this invention applies different supervision methods to each layer of features. Specifically, for features... F 2. A joint loss of L1 (mean absolute error) and MSE (mean squared error loss) is used to enhance the model's sensitivity to subtle boundary errors and stabilize the gradient:
[0101] in, Representation of features F 2. Segmentation results generated by the deep supervision module. This indicates the actual label.
[0102] For intermediate layer features F 3. While preserving certain boundary information, some semantic understanding is introduced. To balance accuracy and semantic expression, a loss function is incorporated. (Binary cross-entropy loss) Loss:
[0103] in, Representation of features F 3. Segmentation results generated by the deep supervision module.
[0104] Deep features F 4 is mainly used for decision output, which contains stronger global semantic information. Therefore, BCEWithLogits loss (i.e., the merging loss of sigmoid activation function and binary cross-entropy loss) is directly used for supervision.
[0105] in, Representation of features F 4. Segmentation results generated by the deep supervision module.
[0106] By jointly optimizing multiple loss components, the shallow layer of the network focuses on edge details, the middle layer balances accuracy and semantics, and the deep layer ensures global consistency, ultimately obtaining a high-precision and lightweight tongue image segmentation model.
[0107] During inference, the input tongue image is first preprocessed by normalizing its size and standardizing its pixel values to ensure that the size and numerical range of the input data are consistent with those of the training data. Then, the preprocessed tongue image is input into the trained ultralight tongue image segmentation network, and the network output is the segmentation result of the tongue region.
[0108] The tongue region segmentation results obtained from the partial tongue image samples in this embodiment are as follows: Figure 14 As shown, this figure displays the visualization of the segmentation results, including the original image, ground truth annotations, and segmentation results. Figure 14 It can be observed that the tongue image segmentation method proposed in this invention achieves complete segmentation of the tongue region, and the segmentation results are very fine and highly consistent with the actual annotations.
[0109] Finally, it should be noted that those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium for the program can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc. The above computer program embodiments can achieve the same or similar effects as any of the corresponding foregoing method embodiments.
[0110] Those skilled in the art will also understand that the various exemplary logic blocks, modules, circuits, and algorithm steps described in conjunction with the disclosure herein can be implemented as electronic hardware, computer software, or a combination of both. To clearly illustrate this interchangeability between hardware and software, the functionality of various illustrative components, blocks, modules, circuits, and steps has been generally described. Whether this functionality is implemented as software or as hardware depends on the specific application and the design constraints imposed on the system as a whole. Those skilled in the art can implement the functionality in various ways for each specific application, but such implementation decisions should not be construed as departing from the scope of the embodiments disclosed herein.
[0111] The above are exemplary embodiments disclosed in this invention. However, it should be noted that various changes and modifications can be made without departing from the scope of the embodiments of this invention as defined by the claims. The functions, steps, and / or actions of the methods according to the disclosed embodiments described herein do not need to be performed in any particular order. The sequence numbers of the disclosed embodiments of this invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments. Furthermore, although the elements disclosed in the embodiments of this invention may be described or claimed individually, they may be understood as multiple unless explicitly limited to a singular number.
[0112] It should be understood that, as used herein, the singular form “a” is intended to include the plural form as well, unless the context clearly supports an exception. It should also be understood that, as used herein, “and / or” refers to any and all possible combinations of one or more of the associated listed items.
[0113] Those skilled in the art should understand that the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of the invention (including the claims) is limited to these examples. Within the framework of the invention, technical features of the above embodiments or different embodiments can be combined, and many other variations of different aspects of the invention exist, which are not provided in the details for the sake of brevity. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the invention should be included within the protection scope of the invention.
Claims
1. A method for segmenting tongue images in Traditional Chinese Medicine for mobile terminals, characterized in that, This includes performing the following steps based on a tongue image segmentation network: Feature extraction was performed on the tongue image to obtain the features of the first tongue region and the second tongue region at different scales; Based on the features of the first tongue region, edge enhancement is performed on the features of the second tongue region to obtain edge-enhanced features of the second tongue region. The second tongue region features are globally enhanced by spatial shifting to obtain globally enhanced second tongue region features. The edge-enhanced second tongue region features and the globally enhanced second tongue region features are fused to obtain the tongue region segmentation result of the tongue image.
2. The method according to claim 1, characterized in that, Feature extraction was performed on the tongue image to obtain features of the first and second tongue regions at different scales, including: Feature extraction is performed on the tongue image to obtain the features of the first tongue body region; Based on different expansion rates, feature extraction and splicing of the first tongue region features are performed to obtain the second tongue region features.
3. The method according to claim 2, characterized in that, Based on different expansion rates, feature extraction and concatenation are performed on the first tongue region to obtain the second tongue region features, including: Based on the first dilation rate, dilation convolution is performed on the features of the first tongue region to obtain the global semantic features of the first tongue region. Based on the second dilation rate, dilation convolution is performed on the features of the first tongue region to obtain the edge detail features of the first tongue region. The global semantic features and edge detail features of the first tongue region are concatenated to obtain the features of the second tongue region.
4. The method according to claim 2, characterized in that, The second tongue region features are obtained by extracting and splicing features of the first tongue region based on different expansion rates, including: extracting and splicing features of the first tongue region layer by layer based on different expansion rates according to a preset number of layers, to obtain multiple second tongue region features. The step of globally enhancing the second tongue region features through spatial shifting to obtain globally enhanced second tongue region features includes: globally enhancing the second tongue region features obtained from the last layer of feature extraction through spatial shifting to obtain globally enhanced second tongue region features.
5. The method according to claim 1, characterized in that, Based on the features of the first tongue region, edge enhancement is performed on the features of the second tongue region to obtain edge-enhanced features of the second tongue region, including: Edge feature enhancement is performed on the first tongue region features to obtain an edge-enhanced tongue region feature map; Based on the edge-enhanced tongue region feature map and the second tongue region feature, the edge-enhanced second tongue region feature is obtained.
6. The method according to claim 5, characterized in that, Edge feature enhancement is performed on the first tongue region features to obtain an edge-enhanced tongue region feature map, including: Different edge detection operators are used to perform edge detection on the features of the first tongue region to obtain different tongue contour features; Different tongue contour features are spliced together to obtain the spliced tongue contour features; Using a target region attention mechanism, the edge-enhanced tongue region feature map is generated based on the spliced tongue contour features.
7. The method according to claim 1, characterized in that, Global enhancement of the second tongue region features is achieved by spatial shifting, resulting in the following globally enhanced second tongue region features: The second tongue region features are uniformly divided into multiple groups of features along the channel dimension, and each group of features is cyclically shifted to obtain globally enhanced second tongue region features.
8. The method according to claim 7, characterized in that, By performing cyclic shifting on each set of features, the globally enhanced second tongue region features are obtained, including: Each set of features is cyclically shifted along the height and width directions respectively to obtain the shifted features in the height direction and the shifted features in the width direction; By using a gated activation function, the globally enhanced second tongue region features are obtained based on the displacement features in the height direction and the displacement features in the width direction.
9. The method according to claim 1, characterized in that, It also includes the following tongue image segmentation network training steps: Construct a labeled tongue image segmentation dataset; The tongue images in the tongue image segmentation dataset are input into the tongue image segmentation network to obtain edge-enhanced second tongue region features, globally enhanced second tongue region features, and tongue region segmentation results. Based on the composite loss function, the composite loss is determined according to the edge-enhanced second tongue region features, the globally enhanced second tongue region features, and the tongue region segmentation results; The parameters of the tongue image segmentation network are updated based on the composite loss to train the tongue image segmentation network.
10. The method according to claim 9, characterized in that, Based on the edge-enhanced second tongue region features, the globally enhanced second tongue region features, and the tongue region segmentation results, the composite loss is determined to include: Based on the edge-enhanced features of the second tongue region, the target attention loss is determined. Based on the globally enhanced features of the second tongue region, a layer-by-layer deep supervision loss is determined; Based on the tongue region segmentation results, the main segmentation loss is determined; Based on the main segmentation loss, the target attention loss, and the layer-by-layer deep supervision loss, a composite loss is determined.