Tongue real physical size measuring method and device based on semantic segmentation

Abstract

The invention provides a method and a device for measuring the real physical size of a tongue based on semantic segmentation. The method comprises the following steps: step 1, fixing a standard gasket at the cheekbone and forehead position of a subject; 2, the subject needs to naturally spit the tongue and naturally fit the lower lip, and the front side face image of the subject is collected; step 3, preprocessing the collected front and side tongue images; step 4, using the processed tongue image, performing cross validation, and establishing and evaluating a special tongue edge attention segmentation model; 5, embedding the tongue edge attention model into a tongue measuring instrument device; step 6, counting the number of pixel points corresponding to the gasket area and the number of tongue pixel points according to the obtained binary front and side segmentation image, and establishing a true value mapping coefficient; step 7, calculating the length-width ratio, the radial ratio and the eccentricity of the scale gasket, and constructing a compensation factor so as to correct the craniofacial hypsokinesis error generated by the traction force of the root of the tongue; and step 8, according to the mapping coefficient and the supplementary factor, calculating actual geometric features including length, width, thickness, area and curvature of the front and side tongues.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence deep learning image processing, and in particular to a method and device for measuring the true geometry value of a tongue based on semantic segmentation. Background Art

[0002] Since the World Health Organization (WHO) included traditional Chinese medicine in the latest global medical guidelines, traditional Chinese medicine has been increasingly recognized and valued by the medical community. Tongue diagnosis is a traditional Chinese medicine method and technical feature of traditional Chinese medicine. With its advantages of painlessness, convenience and non-contact diagnosis, it has gradually attracted the attention of modern medicine and become a research hotspot. As a muscular organ, the tongue is covered with a layer of translucent mucous membrane. Through information such as color, texture, shape, wetness and tooth marks, it reveals diseases and even pathological changes of internal organs. In the process of qualitative judgment, there is ambiguity and subjectivity based on experience. Researchers have applied computerized tongue image analysis technology, pattern recognition technology and developed an automatic tongue diagnosis system to quantify indicators such as tongue color, texture and wetness. However, there are few in-depth quantitative studies on the shape and geometric characteristics of the tongue, and the shape of the tongue has always been regarded as an important basis for diagnosing diseases.

[0003] The objective quantitative analysis of the geometric features of the tongue is mainly divided into two stages: the segmentation tool clearly outlines the edge contour of the tongue, and the calibration method is used to measure the geometric features of the tongue. Currently, both stages face key problems.

[0004] First, tongue segmentation faces many challenges, including severe fissures on the tongue surface, abnormal color, incomplete protrusion of the tongue, tooth-revealing samples, tongue color similar to face color, lip color interference, mandibular wrinkle interference, and complex background. To address the above challenges, segmentation methods are mainly divided into two categories: traditional segmentation methods and deep learning segmentation methods. Traditional segmentation methods usually assume that the tongue contour is relatively fixed, and use snake contour evolution and double ellipse contour evolution. Snake contour evolution attracts the snake shape to move to the edge of the significant feature in the tongue. The double ellipse variable contour fitting technology can maintain the global shape while deforming locally and prevent the problem of bad aggregation and shrinkage of the snake shape. With the breakthrough of deep learning technology, the field of computer vision has achieved breakthrough achievements in the task of tongue image segmentation. Zhou et al. applied the deep learning framework to the field of tongue image segmentation, integrated the classic medical image segmentation structure and the squeeze excitation module, designed an image graying program to reduce the impact of tongue color error on segmentation, and achieved an average intersection-over-union ratio of 93.11% on the standard medical sampling data set, significantly improving the segmentation accuracy.

[0005] Although the above studies have made significant progress in tongue image segmentation, existing studies are still insufficient in dealing with fine segmentation of tongue edges. This is because traditional contour evolution methods only work on the tongue body, resulting in loss of information on edge positions such as the tongue root and tongue tip. During the convolution process, the attention mechanism of deep learning segmentation methods will focus more on the tongue surface, resulting in blurred tongue edges and inability to accurately distinguish fine edge conditions such as tongue surface fissures and lip color interference.

[0006] Next, the geometric features of the tongue have been widely studied and confirmed as key indicators for diagnosing patient diseases. Many studies have revealed that diseases affect the tongue tissue and thus change the size and shape of the tongue. Traditional diagnostic methods rely on the clinical experience of doctors, and the judgment process is ambiguous and subjective. In order to improve the accuracy and objectivity of diagnosis, it is urgent to carry out quantitative extraction of geometric features from tongue images. Huang et al. proposed a method based on the geometric features of tongue contour measurement to extract seven pixel geometric ratios of aspect ratio, radial ratio, area ratio and angle features from tongue segmentation images. The study finally achieved a classification accuracy of 90.3% for tongue shape by relying on seven key features through the hierarchical analysis method. The study has made a series of groundbreaking progress in the geometric features of tongue shape, but the common drawback is that the extracted geometric features are all relative values ​​at the pixel level, rather than the actual physical size of the tongue. This means that although these features are statistically correlated, they cannot directly reflect the actual geometric morphology of the tongue. In clinical medicine, doctors need specific quantitative geometric information to make diagnosis and treatment decisions. Summary of the invention

[0007] In order to solve the technical problems of the above two differences, the present invention provides a method and device for measuring the true geometry of the tongue based on semantic segmentation. The present invention solves the above problems from two aspects, and designs a proprietary tongue edge attention model, which includes a novel multi-scale channel temporal attention module, and a method for measuring the absolute true value of a segmented gasket marker. Compared with the prior art, this method more accurately extracts the edge contour of the tongue, especially the positions of the tongue root and tongue tip that are easily overlooked, and better distinguishes non-tongue parts. The absolute tongue geometric feature measurement method is a groundbreaking output of the true physical size of the tongue, including actual length, width, thickness, area and curvature.

[0008] The present invention establishes a proprietary tongue edge attention model and combines it with a novel multi-scale channel temporal attention module to refine channel features and capture long-range dependencies between multi-scale images and feature maps, and controllably fuses the two to improve the receptive field and edge segmentation capabilities during tongue segmentation.

[0009] The present invention relies on a method for determining the absolute true value of a standard segmentation gasket marker, accurately segments the gasket contour, uses its area as a standard reference, and constructs a mapping coefficient between pixel points and the real world.

[0010] The present invention takes into account the craniofacial tilt phenomenon caused by the traction force of the tongue root, proposes a supplementary factor to reduce the craniofacial tilt error, and effectively alleviates the error caused by craniofacial tilt. In addition, the calculation of the mapping coefficient and the compensation factor enables the present invention to obtain a series of real tongue geometric features through a standard gasket, including but not limited to the length, width and area of ​​the front tongue, the length, thickness, area and curvature of the tongue surface of the side tongue. This greatly provides value for medical clinical reference.

[0011] The technical solution of the present invention is: a method and device for measuring the true geometry value of the tongue based on semantic segmentation, comprising the following steps:

[0012] Step 1: Fix the standard gasket on the subject's cheekbone and forehead;

[0013] Step 2: The subject needs to stick out the tongue naturally and put it naturally against the lower lip, and collect the frontal and side craniofacial images of the subject;

[0014] Step 3, preprocessing the collected front and side images;

[0015] Step 4: Use the processed tongue images to establish and evaluate the tongue edge attention model through cross-validation;

[0016] Step 5, embedding the proposed tongue marginalism theoretical model into the tongue geometry measuring instrument;

[0017] Step 6: The tongue geometry measuring instrument obtains a binary front and side segmentation image in the intermediate step, counts the number of pixels corresponding to the gasket area and the tongue position, and establishes a real physical value mapping coefficient;

[0018] Step 7, calculating the aspect ratio, radial ratio and eccentricity of the gasket, and constructing a compensation factor to correct the craniofacial posterior tilt error caused by the traction force of the tongue root;

[0019] Step 8: Calculate the true geometric characteristic values ​​of the tongue including length, width, area and curvature on the front and side according to the mapping coefficients and supplementary factors.

[0020] Furthermore, the present invention adopts a washer scale as a real mapping method to realize the measurement of the real physical size of the tongue. For the frontal image, the washers fixed on both sides of the forehead and the cheekbones will cause the craniofacial tilt according to the traction force of the tongue. This craniofacial tilt behavior will distort the washer mapping angle and thus change the mapping relationship. In view of the craniofacial tilt behavior, the present invention introduces a compensation factor in the measurement process. This is a method for correcting the distortion mapping relationship of craniofacial tilt, which is obtained by calculating a series of geometric coefficients of the washers.

[0021] Beneficial effects:

[0022] For many years, the measurement of the true physical size of the tongue has been limited by two problems. First, tongue image segmentation is plagued by blurred edge contours, abnormal tongue color, severe tongue fissures and non-tongue parts. The tongue edge attention model proposed in the present invention relies on a unique architecture and multi-scale channels and temporal attention modules to effectively solve the problem of blurred tongue edges. Second, the measurement of tongue geometric features has been stagnant at the pixel level. This type of method fails to establish the relationship between pixels and the real world. The obtained geometric values ​​only act on the statistical significance within the data set and have no reference significance in clinical medicine. The present invention relies on a fixed gasket scale and two calculation modules to accurately calculate the true physical size of the tongue, foresightedly fills the calculation error caused by craniofacial posterior tilt, and pioneeringly calculates the true physical size of the tongue, providing a new approach and method for tongue analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0024] Figure 1 This is a step-by-step diagram for fixing the craniofacial rulers of the subjects, which details the methods for fixing the forehead position ruler and the zygomatic bone position ruler.

[0025] Figure 2 This is a standard step diagram for acquiring frontal and lateral tongue images, which details the errors that should be avoided due to the characteristics of the tongue software, as well as the standard image acquisition method.

[0026] Figure 3 This is a flow chart for preprocessing frontal and lateral tongue images, which details the preprocessing method for frontal and lateral tongue images, as well as the method of filling rectangles to keep the image length and width equal.

[0027] Figure 4 Architecture diagram of the tongue edge attention segmentation model.

[0028] Figure 5 Diagram of the multi-scale channel and temporal attention module, which explains the working principle and details of the module in detail.

[0029] Figure 6Schematic diagram of the indicators to be tested on the front and side.

[0030] Figure 7 It is a schematic diagram of craniofacial tilt, which details the shape of the gasket distortion caused by craniofacial tilt due to the traction force of the tongue root, as well as the calculation logic of the compensation factor.

[0031] Figure 8 The following is a complete processing flow chart. DETAILED DESCRIPTION

[0032] In order to clearly illustrate the specific implementation cases and implementation methods, the models and methods involved in the present invention are described in detail below. In the following description, examples will be cited to better illustrate the phenotypes of the models and methods of the present invention.

[0033] With the development of the field of artificial intelligence, its corresponding sub-fields have also gradually developed. The present invention combines the field of image segmentation, a sub-field of artificial intelligence. In order to calculate the real physical size of the tongue, the tongue edge segmentation is the basis. The tongue edge attention segmentation model proposed in the present invention uses a special and novel residual link upsampling architecture, and integrates multi-scale channels and temporal attention modules to effectively improve the accuracy of edge segmentation. After that, the present invention constructs a mapping scale factor and establishes a mapping ratio relationship between pixel points and real sizes. The present invention pays attention to the craniofacial tilt error caused by the traction force of the tongue root, and proposes a compensation factor, which calculates the degree of distortion by calculating the aspect ratio, eccentricity and meridian ratio of the gasket, thereby calculating the craniofacial tilt angle to compensate for the error caused by the craniofacial tilt. Finally, the tongue geometry measuring instrument will obtain the segmentation result through the tongue edge segmentation attention model according to the front and side craniofacial images, and then calculate the mapping scale factor and compensation factor to calculate the real physical characteristics of the tongue.

[0034] The operation examples of the tongue geometry measuring instrument will be described in detail in the explanation section below, and will be interpreted with the actual operation process and model operation process.

[0035] See also Figure 8 As shown, the technical solution of the present invention is: a method and device for measuring the real physical size of the tongue based on semantic segmentation, comprising the following steps:

[0036] Step 1: Fix the standard gasket on the subject's cheekbone and forehead;

[0037] Step 2: The subject needs to stick out the tongue naturally and put it naturally against the lower lip, and collect the frontal and side craniofacial images of the subject;

[0038] Step 3, preprocessing the collected front and side images;

[0039] Step 4: Using the processed tongue images, a proprietary tongue edge attention segmentation model is established and evaluated through cross-validation;

[0040] Step 5, the established tongue edge attention segmentation model is embedded into the tongue geometry measuring instrument;

[0041] Step 6: Based on the obtained binary front and side segmentation images, the number of pixels corresponding to the gasket area and the tongue position is counted to establish the real physical value mapping coefficient;

[0042] Step 7, calculating the aspect ratio, radial ratio and eccentricity of the gasket, and constructing a compensation factor to correct the craniofacial posterior tilt error caused by the traction force of the tongue root;

[0043] Step 8: Calculate the true geometric characteristic values ​​of the tongue including length, width, area and curvature on the front and side according to the mapping coefficients and supplementary factors.

[0044] In the following process, a set of examples will be described to fully illustrate the complete use process of the tongue tester and the calculation method of the mapping factor and the compensation factor. The process of handling new testers in the application process of the present invention will be further described.

[0045] The present invention recruited 194 example subjects to construct a data set. The data includes frontal and side tongue-sticking images of the subject in a natural situation, and the data includes labels for the length, width and area of ​​the front tongue, and labels for the length, thickness, area and bending curvature of the side tongue. This label is quantitatively measured by a professional sleep physician using image analysis software (Image J version 1.8.0, National Institutes of Health, Bethesda, MD). The front and side images of the 194 subjects all completely include the face and neck of the subject, and a 3 cm diameter ruler is fixed on the forehead, and a 1 cm diameter circular ruler is fixed on both sides of the cheekbones. The shooting background is pure white, and the subject does not wear any accessories.

[0046] Step 1: Fix the reference scale, see Figure 1 shown.

[0047] Step S1, fix the washer at the forehead position to ensure that the subject's face is clean and tidy, and no task accessories are worn. A white circular washer ruler with a diameter of 3 cm is fixed at the center of the subject's forehead between the eyebrows. In this example, the 3 cm washer is inferred from human anatomy, and its size is most suitable for the center of the eyebrows of Asians, and will not block other positions. In addition, through the observation of the traction force on the root of the tongue by the present invention, it is found that the cranial face backward behavior can be corrected by a circular washer, and its principle is explained in step 7.

[0048] Step S2, fix the zygomatic position gasket, and fix a 1 cm round gasket sticker on both sides of the subject's zygomatic bone. In this example, red gasket stickers are selected. The convex positions on both sides of the zygomatic bone are not suitable for fixation with transparent tape, so it is recommended to use stickers as a ruler here.

[0049] Step 2: Collect the front and side images, see Figure 2 shown.

[0050] Step S3, the subject naturally sticks out the tongue, ensuring that the middle of the lower tongue surface fits the lower lip, and there is no bad behavior such as holding the tongue or swallowing the tongue. In this example, the image is collected by the doctor in the ENT clinic, and the doctor guides the tongue sticking behavior during the collection process. In actual application, due to the characteristics of the tongue software, the standardization degree of tongue sticking behavior needs to be specified.

[0051] Step S4, take a frontal tongue-sticking image, the frontal tongue-sticking image angle is to face the subject directly, ensure that the ears on both sides are at the same height and are completely symmetrical and clearly visible, and completely include the neck and craniofacial area. The side tongue-sticking image angle is 90 degrees to the side, ensure that the ear position is in the middle position, and completely retain the tongue surface area.

[0052] Step 3, image preprocessing, see Figure 3 shown.

[0053] Step S5, using artificial intelligence face recognition technology, intercept the background part of the image, retain the entire head and neck area, and ensure that the tongue is complete and clearly visible. It should be noted that the two washers are fixed at the forehead and cheekbone positions of the image respectively, and the image position usually does not intercept the washer position.

[0054] Step S6, the image size transformation method is usually a reshaping method, which will stretch the image, causing the gasket in the image to deform, resulting in a huge deviation in the measurement. On this basis, the present invention uses an image filling method to unify the image size. This method is to fill the left and right sides of the image with a black rectangle (RGB (0, 0, 0)) to ensure that the length and width of the image size are equal to avoid distortion of the gasket position. Then use the standard tongue color correction tool to unify the tongue color image standard.

[0055] Step 4: Establish the tongue edge attention segmentation model, see Figure 4 shown.

[0056] The tongue edge attention segmentation model can be understood as two parts. The first part is the downsampling result, and its detailed steps are shown in S6. Its function is to extract the feature process of the front and side preprocessed images. The second part is the upsampling result, and its detailed steps are shown in S7. Its function is to deconvolute and restore the segmentation binary image based on the feature map.

[0057] In step S7, the encoder has four layers, each with a standard convolutional block. Multi-scale channels and temporal attention modules are applied in the second, third, and fourth layers to fuse multi-scale images and encoding features and enhance the model's attention perception of the tongue edge. At the junction of each layer, in order to reduce the spatial dimension of the feature and the height and width of the feature map, a maximum pooling layer is applied at the end of the four layers.

[0058] In step S8, the decoder still has a four-layer structure, and each layer uses upsampling to enhance the resolution of the feature map. In the upsampling process, a residual link structure is set up, whose starting sample is applied to the first and second upsampling structure layers, and after being extracted by a larger convolution kernel, it is concatenated with the third and fourth upsampling features, thereby optimizing the concatenated context information.

[0059] The structure of the multi-scale channel and temporal attention module embedded in the downsampling step S7 is as follows Figure 5 shown.

[0060] The multi-scale channel and temporal attention module consists of two parts, one for the processing of the feature map X1 during sampling, and the other for the processing of the multi-scale image X2. This novel module integrates channel and spatial attention, and uses residual links for feature fusion, which promotes the effective fusion of two different types of key information.

[0061] Step S9, for the processing of the feature graph X1 during the sampling process, X1∈R H·W·C As input, X tmp ∈R H·W·C As the result after two convolutions. In the equation, F(·) represents the process of two convolution operations, θ conv1 and Ω conv1 Represents the structure of two convolutional layer operations.

[0062] X tmp =F(X1;θ conv1 ,Ω conv1 )

[0063] According to the attention feature extraction of X1, X attn Represents the attention convolution channel F attn (·) The channel is firstly subjected to the non-complete squeezing operation by the average pooling layer AVGPool(·), and then the parameter θ attn The first fully connected layer controlled by Ω is parameterized, and attn Represents the second fully connected layer.

[0064] X attn =F attn(AVGPool(X1);θ attn ,Ω attn )

[0065] In extracting the attention feature X attn After two convolution operations on X1, is the feature vector X attn Multiply by matrix X1 and add X tmp ,Right now Among them, X attn is a feature vector of dimension C, X tmp ∈R H·W·C For X1∈R H·W·C The result after two convolution operations. A one-dimensional channel convolution layer with a 1×1 kernel, followed by batch normalization and ReLU function activation, called θ conv2 . After the one-dimensional channel convolution layer, a sigmoid function σ(·) is used to obtain spatial attention. After that, the spatial attention map and Multiply and manually set weights The original features are preserved while focusing on relevant regions, which enhances the model's ability to capture context and long-range dependencies.

[0066]

[0067] Step S10, for the input multi-scale image X2, the multi-scale channel and temporal attention module uses two different convolution operations. The first convolution operation θ conv2 A 3×3 convolutional layer is used, followed by batch normalization and ReLU activation, which is processed in the same way as θ conv1 The second convolution operation is convolution Ω ASPP To obtain

[0068]

[0069] In obtaining Afterwards, In the same way, we input it into the multi-scale channel and temporal attention modules to obtain and multiply it by Manually set weights To obtain X out To combine the results of the upper and lower parts and Add, that is

[0070]

[0071] Step 5: embed the trained tongue edge attention model into the tongue measurement instrument.

[0072] The tongue measuring instrument is developed based on the Raspberry Pi public development board, which includes a processing mainboard, a camera component, and a shooting component. The method of use is to pre-input the front or side angle of the subject, take the subject's tongue image, and obtain the tongue segmentation map with the help of the tongue edge attention segmentation model. Then the tongue measuring instrument will calculate the front true area, length and width, or the side true area, length, thickness and curvature according to the segmentation map. The segmentation method is described in step 4, and the calculation method is described in steps 6, 7, and 8.

[0073] Step 6, propose real physical mapping coefficients, such as Figure 6 shown.

[0074] The scale factor α is calculated based on the calibration ruler, and the scale factor is defined as the actual area of ​​the ruler divided by the number of pixels in the segmentation map. The ruler used in the present invention is a 3-cm circle fixed on the cranial surface with an area of ​​2.25π square centimeters, and a 1-cm red circle fixed on the cheekbones on both sides, with a total area of ​​0.5π square centimeters. It should be noted that the front tongue area, length and width, and the lateral tongue area, length and thickness are all linear geometric values, which can be calculated based on the real physical mapping coefficients. For the calculation of the lateral tongue curvature, the present invention applies nonlinear curve fitting to obtain a curve expression, and then obtains the curvature. In the medical definition, the curvature of the tongue refers only to the outer part of the tongue, so the nonlinear curve fitting only fits the outer part of the tongue. Assume that the nonlinear function y=f(x; a1, a2, ..., a n ), where a1, a2, ..., a n are the parameters that need to be fitted.

[0075]

[0076] Use Newton's method to solve the above formula to get the optimal parameters After the fitting is completed, the first-order derivative f′(x) and the second-order derivative f″(x) of y=f(x) are solved, and the derivatives are substituted into the curvature formula to obtain the curvature k.

[0077]

[0078] Step 7, calculate the compensation factor, such as Figure 7 shown.

[0079] According to the craniofacial anatomy, when the subject sticks out the tongue downward, in order to maintain the airway open, the traction force of the tongue root will cause the hyoid bone, mandible, larynx and skull base to move downward, resulting in a moment balance and causing the head to tilt back. In the performance of the present invention, in the frontal image, almost 85% of the subjects tilted their heads back, which would reduce the projection area of ​​the ruler fixed on the forehead, such as Figure 7 The present invention introduces a supplementary factor ω, which corrects the error of craniofacial posterior tilt according to the eccentricity and radial ratio of the standard washer. The center of gravity is defined as P c (x c ,y c ), where x c for|x max -x min | / 2,y c for |y max -y min | / 2. l1 is P c (x c ,y c ) horizontal major axis, l2 is P c (x c ,y c ) vertical minor axis, the aspect ratio is calculated as the ratio of the major axis to the minor axis and x c / y c , the eccentricity lw is calculated as the ratio of the major semiaxis to the minor semiaxis.

[0080]

[0081] From the center of gravity P c (x c ,y c ), the four meridian lines form angles of 30°, 60°, 120° and 150° with the horizontal line respectively, and their lengths are l 30° , l 60° , l 120° and l 150° .

[0082]

[0083] The supplementary factor ω calculates the average value of the eccentricity and the meridian ratio, and introduces a logarithmic function to mitigate the deviation caused by the small angle of craniofacial tilt, that is, The supplementation factor ω is defined as shown in Equation 14.

[0084] ω=ln(2+lw+Rl)-ln2

[0085] Step 8, calculate a series of real physical dimensions of the tongue.

[0086] In the present invention, the front tongue area, length and width, and the side tongue area, length and thickness are obtained by multiplying the scale factor α, the supplement factor ω and the pixel points, that is, the true value is the product of the scale factor, the supplement factor and the number of pixels. It should be noted that in the side image, the ruler is fixed beside the external ear bone, and the backward tilt of the skull and face will not cause the ruler to be distorted. At this time, the supplement factor ω can be set to 1.

[0087] Although the above describes the illustrative specific embodiments of the present invention to facilitate those skilled in the art to understand the present invention, and it should be clear that the present invention is not limited to the scope of the specific embodiments, for those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations using the concept of the present invention are protected.

Claims

1. A method and device for determining the true geometry value of the tongue based on semantic segmentation, characterized in that: The steps include: Step 1: Fix the standard gasket on the subject's cheekbone and forehead; Step 2: The subject needs to stick out the tongue naturally and put it naturally against the lower lip, and collect the frontal and side craniofacial images of the subject; Step 3, preprocessing the collected front and side images; Step 4: Using the processed tongue images, a proprietary tongue edge attention segmentation model is established and evaluated through cross-validation; Step 5, the established tongue edge attention segmentation model is embedded into the tongue geometry measuring instrument; Step 6: Based on the obtained binary front and side segmentation images, the number of pixels corresponding to the gasket area and the tongue position is counted to establish the real physical value mapping coefficient; Step 7, calculating the aspect ratio, radial ratio and eccentricity of the gasket, and constructing a compensation factor to correct the craniofacial posterior tilt error caused by the traction force of the tongue root; Step 8: Calculate the true geometric characteristic values ​​of the tongue including length, width, area and curvature on the front and side according to the mapping coefficients and supplementary factors.

2. According to the method and device for determining the true geometry value of the tongue based on semantic segmentation according to claim 1, it is characterized in that: The step 1 comprises the following sub-steps: Step 1.1: According to the preset conditions, fix the circular gasket on the forehead and cheekbones on both sides.

3. The method and device for determining the true geometry value of the tongue based on semantic segmentation according to claim 1, characterized in that: The step 2 comprises the following sub-steps: Step 2.1: To avoid the difference caused by the softness of the tongue, the subject needs to stick out the tongue naturally and ensure that the bottom of the tongue root fits naturally with the lower lip, and collect the frontal and lateral craniofacial images.

4. The method and device for determining the true geometry value of the tongue based on semantic segmentation according to claim 1, characterized in that: The step 3 comprises the following sub-steps: Step 3.

1. Cut out the original front and side craniofacial images, retain only the entire head and neck, make the left and right ears symmetrical, make the left and right zygomatic washers symmetrical, and keep the tongue intact in the image. Step 3.2, fill both sides of the image with a black pixel rectangle (RGB (0, 0, 0)) to make the image size equal in length and width; apply the tongue color correction method in the field of image processing to correct the front and side tongue color images.

5. The method and device for determining the true geometry value of the tongue based on semantic segmentation according to claim 1, characterized in that: The step 4 comprises the following sub-steps: Step 4.

1. Based on the semantic segmentation algorithm of deep learning, the tongue edge segmentation model architecture proposed in this study, which includes the multi-scale channel and temporal attention module of tongue edge design, uses the preprocessed frontal and side images as input to establish the model. Cross-validation and gradient descent algorithms are used to train the model and iterate and adjust the parameters to the optimal model state.

6. The method and device for determining the true geometry value of the tongue based on semantic segmentation according to claim 1, characterized in that: The step 5 comprises the following sub-steps: Step 5.1: The pre-trained tongue edge attention model is embedded into the tongue geometry measuring instrument, which is designed with a publicly available Raspberry Pi development computing chip base and includes shooting and camera components.

7. The method and device for determining the true geometry value of the tongue based on semantic segmentation according to claim 1, characterized in that: The step 6 comprises the following sub-steps: Step 6.1: The tongue edge attention segmentation model performs semantic segmentation on the front and side images to obtain a segmentation binary image. Step 6.2: Segment the binary image X∈R H×W×C , where H and W represent the length and width of the segmented binary image, respectively, and C represents the number of channels of the segmented binary image. Traverse the length and width of the segmented image and count the number of pixels where the gasket is located, denoted as N1. Use the same method to count the number of pixels where the tongue is located, denoted as N2. The mapping coefficient α is calculated based on the number of two pixels and the standard gasket area.

8. The method and device for determining the true geometry value of the tongue based on semantic segmentation according to claim 1, characterized in that: The step 7 comprises the following sub-steps: Step 7.

1. Calculate the aspect ratio of the circular gasket and locate the transverse major axis of the gasket |X max -X min | and longitudinal minor axis |Y max -Y min |, where the horizontal major axis and the vertical minor axis are defined as the coordinates of the pixel corner points. Step 7.2: Calculate the eccentricity of the circular washer and define the center of gravity of the washer as P c (x c ,y c ), where x c for|x max -x min | / 2,y c for |y max -y min | / 2. l1 is P c (x c ,y c ) horizontal major axis, l2 is P c (x c ,y c )’s vertical minor semi-axis, the eccentricity lw is calculated as the ratio of the major semi-axis to the minor semi-axis, which is defined as l1 / l2 expressed as the eccentricity. Step 7.3: Calculate the radial ratio of the circular gasket from the center of gravity point P c (x c ,y c ), the four meridian lines form angles of 30°, 60°, 120° and 150° with the horizontal line respectively, and their lengths are l 30° , l 60° , l 120° and l 150° , the radial ratio formula is defined as (l 30° +l 150° ) / (l 60° +l 120° ). Step 7.4: The aspect ratio, eccentricity and radial ratio calculated according to the above steps are introduced into a logarithmic function to map the angle of craniofacial tilt, so as to calculate a supplementary factor to compensate for the craniofacial tilt error.

9. The method and device for determining the true geometry value of the tongue based on semantic segmentation according to claim 1, characterized in that: The step 8 comprises the following sub-steps: Step 8.1: For the front tongue, the present invention calculates the real length, width and projection area of ​​the front tongue according to the real mapping coefficient and supplementary factor proposed above. Step 8.2: For the lateral tongue, the present invention calculates the actual length, thickness, protruding area and curvature of the lateral tongue according to the actual mapping coefficient and supplementary factor proposed above.

10. A method and device for determining the true geometry value of the tongue based on semantic segmentation, characterized in that The following modules. Preset gasket module: Fix the standard gaskets on both sides of the subject's cheekbones and forehead. Image acquisition module: The subject needs to stick out the tongue naturally and naturally touch the root of the tongue to the lower lip to collect the front and side images of the subject. Image preprocessing module: preprocess the collected frontal and side images. Example application module: After training, the proprietary tongue edge attention mechanism segmentation model will be embedded in the tongue geometry measuring instrument. This instrument needs to be preset to take a front or side image. After processing by the model, a segmentation map can be obtained. According to the segmentation result, the tongue measuring instrument will calculate the real geometric features according to the following modules. Mapping coefficient calculation module: Count the number of pixels where the gasket and the tongue are located, and establish the corresponding real mapping relationship coefficient based on the real area and the number of pixels. Supplementary factor calculation module: calculate the aspect ratio, radial ratio and eccentricity of the gasket, introduce a logarithmic function to model the craniofacial tilt angle, and construct a supplementary factor to correct the error caused by craniofacial tilt. Geometric feature calculation module: Based on the mapping coefficients and supplementary factors obtained above, the real length, width and protruding area of ​​the front tongue, the real length, thickness, protruding area and bending curvature of the side tongue are calculated.

Images