Hyperspectral tongue picture image segmentation method and system based on deep learning

Abstract

The invention discloses a hyperspectral tongue picture image segmentation method and system based on deep learning, and relates to the technical field of image segmentation. The method comprises the following specific steps: acquiring a hyperspectral image of a tongue picture and marking the hyperspectral image; preprocessing the hyperspectral image to obtain a first image; constructing an image segmentation model based on the 3D U-net and the active contour model; and training the image segmentation model by using the first image, and realizing segmentation of the hyperspectral tongue picture image by using the trained image segmentation model. According to the method, the tongue picture hyperspectral image is acquired and the segmentation model is established, so that the problem that the tongue picture segmentation accuracy is not high due to the fact that an existing tongue body segmentation model is not suitable for hyperspectral image data is effectively solved.

Description

Technical Field

[0001] The present invention relates to the technical field of image segmentation, and in particular to a hyperspectral tongue image segmentation method and system based on deep learning. Background Art

[0002] Tongue diagnosis in traditional Chinese medicine plays a very important role in the diagnosis and treatment of diseases. In the study of tongue image information analysis, tongue image segmentation is a necessary prerequisite for the objective study of tongue diagnosis in traditional Chinese medicine. Since the tongue morphology, tongue extension action and the connection between the tongue and the surrounding oral lips are different, the quality and efficiency of tongue image segmentation directly affect the accurate expression and digital transformation of tongue image information. The existing technology mainly uses the RGB color image of the tongue image to build a segmentation model based on color or texture features. However, the current tongue segmentation model of traditional Chinese medicine tongue image is mostly based on color RGB image data, which has poor applicability in hyperspectral images and does not make full use of spectral feature information, resulting in unsatisfactory segmentation results. In addition, the loss function used by the existing segmentation model is mostly a general model, which is not designed and considered according to the segmentation characteristics of medical images, especially the characteristics of the tongue edge contour. The obtained segmentation model edge contour may appear prominent, sharp and other segmentation results that do not conform to the actual tongue situation. Therefore, for those skilled in the art, how to segment the tongue image based on the spectral features and spatial features in the hyperspectral image is an urgent problem to be solved. Summary of the invention

[0003] The purpose of the present invention is to provide a hyperspectral tongue image segmentation method and system based on deep learning, which effectively solves the problem that the existing tongue segmentation model is not suitable for hyperspectral image data and thus has low tongue segmentation accuracy.

[0004] To achieve the above object, the present invention provides the following solutions: On the one hand, a hyperspectral tongue image segmentation method based on deep learning is provided, and the specific steps include the following:

[0005] Obtain and annotate hyperspectral images of the tongue;

[0006] Preprocessing the hyperspectral image to obtain a first image;

[0007] Build an image segmentation model based on 3D U-net and active contour model;

[0008] The image segmentation model is trained using the first image, and the trained image segmentation model is used to achieve segmentation of the hyperspectral tongue image.

[0009] Preferably, the specific steps of preprocessing the hyperspectral image are:

[0010] Adopting an adaptive band grouping method to calculate the correlation between different spectral bands of the hyperspectral image, and grouping the hyperspectral bands according to the correlation calculation results;

[0011] The first band data is selected from each band grouping interval as the representative feature of the group, and then the spatial domain singular spectrum analysis is performed on the representative features of different groups;

[0012] After completing the spatial domain singular spectrum analysis, the representative features of different groups are combined to obtain combined features, and principal component analysis and dimensionality reduction are performed on the combined features to obtain the first image.

[0013] Preferably, an active contour model is used as a loss function in the image segmentation model.

[0014] Preferably, the expression of the active contour model loss function is:

[0015] Loss = Length + λ·Region;

[0016]

[0017] where v∈[0,1] m*n represents the real segmentation image, u∈[0,1] m*n represents the image segmentation model predicting the segmented image, ds is the length of the Euclidean element, C is the length of the segmentation curve, Ω c is a closed subset of the segmented image domain Ω, c 1 and c 2 is the average of the segmented images marked as external and internal, and λ is an arbitrary fixed parameter used to control the regularization process and c 1 , c 2 balance between.

[0018] Preferably, the active contour model loss function is solved using the Heaviside function and partial differential equation of the level set method.

[0019] Preferably, the image segmentation model is a U-shaped structure, including an encoder network and a decoder network.

[0020] On the other hand, a hyperspectral tongue image segmentation system based on deep learning is provided, including an image acquisition module, a preprocessing module, a modeling module, and a training module; wherein:

[0021] The image acquisition module is used to obtain and annotate the hyperspectral image of the tongue;

[0022] The preprocessing module is used to preprocess the hyperspectral image to obtain a first image;

[0023] The modeling module is used to build an image segmentation model based on 3D U-net and active contour model;

[0024] The training module is used to train the image segmentation model using the first image, and use the trained image segmentation model to achieve segmentation of the hyperspectral tongue image.

[0025] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0026] (1) By constructing a hyperspectral tongue image segmentation method, an automatic tongue contour segmentation model based on spectral features and spatial features can be obtained, which effectively solves the problem that the existing tongue segmentation model is not suitable for hyperspectral image data and causes low tongue segmentation accuracy.

[0027] (2) The hyperspectral tongue segmentation loss function model based on the active contour model can achieve the global minimization of the active contour line energy of image segmentation, making the segmented tongue edge contour smoother and clearer. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0029] Figure 1 is a flow chart of the method of the present invention;

[0030] Figure 2 This is a structural diagram of the image segmentation model of the present invention. DETAILED DESCRIPTION

[0031] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0032] The purpose of this invention is to provide a hyperspectral tongue image segmentation method based on deep learning. Figure 1 As shown, the specific steps include the following:

[0033] S1, obtaining and annotating the hyperspectral image of the tongue;

[0034] S2, preprocessing the hyperspectral image to obtain a first image;

[0035] S3, build an image segmentation model based on 3D U-net and active contour model;

[0036] S4. Using the first image to train an image segmentation model, and using the trained image segmentation model to segment the hyperspectral tongue image.

[0037] Furthermore, the specific content of step S1 is:

[0038] Digital cameras can only obtain the imaging information of the tongue in the three primary color spectra of red, green and blue, but the characteristics in other spectral bands cannot be obtained and analyzed. Therefore, the current objective research methods of tongue diagnosis have the problem of insufficient characteristic information. Hyperspectral technology can obtain tongue characteristic information under hundreds of continuous spectral wavelengths, which effectively makes up for the problems existing in current research. Therefore, it is necessary to acquire hyperspectral data of tongue images through a hyperspectral image data acquisition platform. In order to ensure the reliability and stability of the construction of the tongue segmentation model, no less than 1,000 use cases need to be collected, and the collected tongue image data are judged and marked by TCM clinicians to depict the edge contour of the tongue.

[0039] The biggest difference between hyperspectral images and natural color images is the increase in spectral size, which can collect spectral information of hundreds of dimensions in different spectral bands. Therefore, before analysis, the hyperspectral image data is subjected to dimensionality reduction and denoising to reduce the number of features, reduce noise interference, and improve calculation speed. Hyperspectral images contain a large number of bands, so their data volume is large, but there is a high correlation between different bands of hyperspectral images, resulting in a large redundancy between bands. Extracting representative bands from hyperspectral tongue image data for analysis can effectively reduce the amount of calculated data, increase calculation speed, and improve segmentation accuracy.

[0040] Furthermore, the specific steps of preprocessing the hyperspectral image in step S2 are:

[0041] S21. Adopt the adaptive band grouping method to calculate the correlation between different spectral bands of the hyperspectral image, and group the hyperspectral bands according to the correlation calculation results; ensure that the spectral correlation within each group is high and the correlation between different groups is different, thereby effectively removing the spectral redundancy between the bands.

[0042] In this embodiment, in order to ensure the accuracy and stability of subsequent segmentation, the number of adaptive grouping groups is set to be no less than 6 groups. Too few groups will ignore the spectral characteristics of the tongue image, and too many groups will increase redundant data and increase computing costs.

[0043] S22. Select the first band data from each band grouping interval as the representative feature of the group, and then perform spatial domain singular spectrum analysis on the representative features of different groups to further reduce noise interference.

[0044] Spatial domain singular spectrum analysis can achieve effective noise reduction and signal enhancement while retaining structured features through steps such as data embedding, data resetting, singular value decomposition and signal hierarchical reconstruction.

[0045] S23. After completing the singular spectrum analysis in the spatial domain, the representative features of different groups are combined to obtain combined features, principal component analysis is performed on the combined features to reduce the dimension, and the first 6-dimensional data is used as the input of the image segmentation model.

[0046] Furthermore, in the present invention, a hyperspectral tongue image segmentation model is constructed based on 3D U-Net and active contour model. Figure 2 As shown, the segmentation model is a U-shaped structure. The left side is the encoder network, which can provide a nonlinear local-to-global representation of the hyperspectral image cube; the right side is the decoder network, which aims to supplement the encoder network by reconstructing the nonlinear global-to-local representation.

[0047] The encoder network consists of four encoder blocks of different resolutions. Each layer of the encoding module contains convolution, rectified linear unit (ReLu), and maximum pooling blocks. The convolution consists of two 3×3×3 convolution kernels, while the maximum pooling size is 2×2×2 with a stride of 2 in each dimension. Therefore, after one layer of encoding processing, the feature channels of the output data are doubled and the resolution is reduced by half.

[0048] The decoder network also consists of four decoding blocks. In the result synthesis path, each layer consists of 2×2×2 upsampling with a stride of 2 in each dimension, followed by two 3×3×3 convolution kernels, each followed by a rectified linear unit (ReLu) operation.

[0049] After being processed by the decoder network, the output result is convolved with a 1×1×1 kernel size with a stride of 1, and a 3D cube data result with the same size as the original input can be obtained. Finally, a 2D convolution layer is calculated to generate a 2D segmentation map.

[0050] Furthermore, by using the active contour model as the loss function in the image segmentation model, the automatic segmentation of the image can be achieved by globally minimizing the active contour energy.

[0051] The expression of the active contour model loss function is:

[0052] Loss = Length + λ·Region;

[0053]

[0054] where v∈[0,1] m*n represents the real segmentation image, u∈[0,1] m*n represents the image segmentation model predicting the segmented image, ds is the length of the Euclidean element, C is the length of the segmentation curve, Ω c is a closed subset of the segmented image domain Ω, c 1 and c 2 is the average of the segmented images marked as external and internal, and λ is an arbitrary fixed parameter used to control the regularization process and c 1 , c 2 balance between.

[0055] The Heaviside function and partial differential equation of the level set method are used to solve the above active contour model loss function. Therefore, the proposed active contour model loss function based on the boundary segmentation contour length can effectively find an active contour line, which is not only non-convex but also suitable for shape preservation, making the segmentation result smoother and more accurate.

[0056] On the other hand, a hyperspectral tongue image segmentation system based on deep learning is provided, including an image acquisition module, a preprocessing module, a modeling module, and a training module; wherein:

[0057] An image acquisition module is used to obtain and annotate hyperspectral images of the tongue;

[0058] A preprocessing module, used for preprocessing the hyperspectral image to obtain a first image;

[0059] Modeling module, used to build image segmentation models based on 3D U-net and active contour models;

[0060] The training module is used to train the image segmentation model using the first image, and use the trained image segmentation model to achieve segmentation of the hyperspectral tongue image.

[0061] The principles and implementation methods of the present invention are described in this article using specific examples. The description of the above embodiments is only used to help understand the method and core idea of ​​the present invention. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A hyperspectral tongue image segmentation method based on deep learning, characterized in that: The specific steps include the following: Obtain and annotate hyperspectral images of the tongue; Preprocessing the hyperspectral image to obtain a first image; Build an image segmentation model based on 3D U-net and active contour model; The image segmentation model is trained using the first image, and the trained image segmentation model is used to achieve segmentation of the hyperspectral tongue image.

2. The method for hyperspectral tongue image segmentation based on deep learning according to claim 1, characterized in that: The specific steps of preprocessing the hyperspectral image are: Adopting an adaptive band grouping method to calculate the correlation between different spectral bands of the hyperspectral image, and grouping the hyperspectral bands according to the correlation calculation results; The first band data is selected from each band grouping interval as the representative feature of the group, and then the spatial domain singular spectrum analysis is performed on the representative features of different groups; After completing the spatial domain singular spectrum analysis, the representative features of different groups are combined to obtain combined features, and principal component analysis and dimensionality reduction are performed on the combined features to obtain the first image.

3. The method for hyperspectral tongue image segmentation based on deep learning according to claim 1, characterized in that: In the image segmentation model, the active contour model is used as the loss function.

4. The method for hyperspectral tongue image segmentation based on deep learning according to claim 3 is characterized in that: The expression of the active contour model loss function is: Loss = Length + λ·Region; where v∈[0,1] m*n represents the real segmentation image, u∈[0,1] m*n represents the image segmentation model predicting the segmented image, ds is the length of the Euclidean element, C is the length of the segmentation curve, Ω c is a closed subset of the segmented image domain Ω, c1 and c2 are marked as the average of the external and internal segmented images, and λ is an arbitrary fixed parameter used to control the regularization process and the balance between c1 and c2.

5. The method for hyperspectral tongue image segmentation based on deep learning according to claim 4, characterized in that: The active contour model loss function is solved using the Heaviside function and partial differential equation of the level set method.

6. The method for hyperspectral tongue image segmentation based on deep learning according to claim 1, characterized in that: The image segmentation model is a U-shaped structure, including an encoder network and a decoder network.

7. A hyperspectral tongue image segmentation system based on deep learning, characterized in that: It includes image acquisition module, preprocessing module, modeling module and training module; among them, The image acquisition module is used to obtain and annotate the hyperspectral image of the tongue; The preprocessing module is used to preprocess the hyperspectral image to obtain a first image; The modeling module is used to build an image segmentation model based on 3D U-net and active contour model; The training module is used to train the image segmentation model using the first image, and use the trained image segmentation model to achieve segmentation of the hyperspectral tongue image.

Images