Prediction model training method, prediction method and device for identifying acra hypertrophy

Abstract

The invention discloses a prediction model training method, a prediction method and a prediction device for identifying acra hypertrophy, which can realize data volume expansion of a prediction model training data set and improve the prediction accuracy of a prediction model. The prediction model training method for identifying acra hypertrophy comprises the following steps: acquiring a face video, picture or CT image set of a patient with acra hypertrophy; reconstructing according to the CT image set to obtain a three-dimensional model of the head of the patient; intercepting face photos of at least two different angles of each three-dimensional model, and constructing a training data set together with face pictures of the acra hypertrophy patient; and inputting the training data set into a prediction model to carry out model training in combination with the attention mask.

Description

Technical Field

[0001] The present invention belongs to the technical field of acromegaly prediction, and in particular relates to a prediction model training method, a prediction method and a device for identifying acromegaly. Background Art

[0002] Acromegaly is an endocrine and metabolic disease characterized by abnormal enlargement of body shape and internal organs due to excessive secretion of growth hormone (GH) by the pituitary gland, accompanied by corresponding abnormal physiological functions.

[0003] Experienced endocrinologists and neurosurgeons can visually identify patients with acromegaly by their facial features, thereby reminding patients to seek medical treatment in a timely manner to reduce the risk of target organ damage and prolong their lifespan.

[0004] Doctors need rich experience to judge patients with this type of disease. In order to reduce the experience requirement, symptom screening is currently performed based on natural photos. Referring to the "A novel acromegaly screening system and screening method thereof" with application number CN202010843594.8, it obtains photos of the hands of the test subject and makes judgments through convolutional neural networks.

[0005] The training of convolutional neural networks requires a large amount of data. The larger the amount of training data, the higher the recognition accuracy when applied. However, the above method has the following problems:

[0006] 1. Since acromegaly is a rare disease with a small number of cases and a small amount of natural photo data based on the hands, the prediction model obtained by this method has low recognition accuracy.

[0007] 2. The accuracy of acromegaly screening based on hand images is not high, because other diseases also have the appearance of thickened hands, which may lead to misdiagnosis. Summary of the invention

[0008] In order to solve the problem that the existing methods for prediction based on patient hand photos have small data volume and limited model training sets, resulting in poor model prediction accuracy, the present invention provides a prediction model training method, prediction method and device for identifying acromegaly faces, which can expand the data volume of the training data set of the prediction model and improve the prediction accuracy of the prediction model.

[0009] The purpose of the present invention is achieved through the following technical solutions:

[0010] The first aspect of the present invention discloses a prediction model training method for identifying acromegaly, comprising the following steps:

[0011] obtaining a CT image set of a patient with acromegaly, wherein the CT image set includes at least one CT image of a patient with acromegaly;

[0012] Reconstructing a three-dimensional model of the patient's head according to the CT image set;

[0013] Capturing facial photos of at least two different angles of each of the three-dimensional models to obtain a training data set;

[0014] The training data set is input into the prediction model for model training.

[0015] A second aspect of the present invention discloses a prediction method for identifying acromegaly, comprising the following steps:

[0016] Obtaining a facial photo of the person to be screened;

[0017] The facial photo of the person to be screened is input into the prediction model obtained by using the prediction model training method for identifying acromegaly described in the first aspect to obtain a prediction result of the person to be screened, wherein the prediction result is whether the person has acromegaly or not.

[0018] The third aspect of the present invention discloses a prediction device for identifying acromegaly, comprising a data acquisition unit, a memory and a controller, wherein the data acquisition unit is used to collect facial photos, the memory stores a computer program, and the controller is used to read the facial photos and the computer program to execute the prediction method for identifying acromegaly described in the second aspect.

[0019] Compared with the prior art, the present invention has at least the following advantages and beneficial effects:

[0020] 1. The present invention obtains a three-dimensional model of the patient's head through CT image reconstruction, and obtains facial photos from different angles through the three-dimensional model to expand the data volume of the training data set of the prediction model and improve the prediction accuracy of the prediction model.

[0021] 2. This scheme is based on the patient's facial photo and focuses the features on the face, which improves the accuracy of acromegaly identification. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0023] Figure 1 The present invention is a flowchart of a prediction model training method for identifying acromegaly.

[0024] Figure 2 ROC curve feature map of the Facial Attention ResNet prediction model to focus on the nose.

[0025] Figure 3 The 68 facial feature points detected by the Dlib library.

[0026] Figure 4 The generation process of the attention mask. DETAILED DESCRIPTION

[0027] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0028] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0029] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features of the embodiments may be combined with each other.

[0030] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0031] In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inside", "outside", etc. indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, or the positions or positional relationships in which the product of the invention is usually placed when in use, or the positions or positional relationships commonly understood by those skilled in the art, which are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", etc. are only used to distinguish the description, and cannot be understood as indicating or implying relative importance.

[0032] A first aspect of the present invention discloses a prediction model training method for identifying acromegaly, the training method comprising steps S11 to S14.

[0033] Step S11, obtaining a CT image set of patients with acromegaly, wherein the CT image set includes CT images of at least six patients with acromegaly.

[0034] Step S12: reconstructing a three-dimensional model of the patient's head according to the CT image set.

[0035] This step can be implemented based on the existing Computed Tomography Three-Dimensional Reconstruction technology, that is, converting the two-dimensional tomographic image obtained from the CT scan into a three-dimensional image.

[0036] Step S13, capturing facial photos of at least two different angles of each of the three-dimensional models to obtain a training data set.

[0037] By adopting this step, the number of face photos in the obtained training data set is greater than or equal to twice the number of CT images in the CT image set, and at least the training data set is doubled compared to the number of patients.

[0038] Of course, in order to further increase the number of face photos in the training data set, each 3D model can capture multiple face photos from different angles.

[0039] In order to improve the feature strength of facial photos, before interception, the three-dimensional model can be processed by data enhancement, and the data enhancement method includes adjusting at least one of brightness contrast, Gaussian noise, color jitter, Gaussian blur, and motion blur; then the three-dimensional model is recorded for a 180-degree facial video, and at least two facial photos from different angles are intercepted to obtain a training data set.

[0040] Preferably, five data enhancement methods of adjusting brightness contrast, Gaussian noise, color jitter, Gaussian blur, and motion blur can be used simultaneously.

[0041] In order to further increase the number of facial photos in the training data set, facial photos of patients with acromegaly directly taken and / or facial photos taken at different angles from facial videos of patients with acromegaly can also be added to the training data set for training.

[0042] If it is a face photo, you can directly use it as the data in the training data set. If it is a face video, you can screenshot the face photos at different angles and use them as the data in the training data set. Similarly, it is recommended to capture at least two photos at different angles for each face video.

[0043] Step S14: input the training data set into the prediction model for model training.

[0044] The prediction model of this step can be any convolutional neural network, such as VGGNet, GoogLeNet and ResNet, etc. In order to improve the accuracy of acromegaly recognition, the preferred prediction model is Face Attention ResNet, which focuses on the human nose.

[0045] The changes in facial features caused by acromegaly are usually more obvious and concentrated, such as long and oblique forehead, protruding brow arches, thickened and enlarged nose wings, thick lips, enlarged tongue, protruding mandible, sparse teeth and malocclusion, etc., which makes it possible to further improve the recognition accuracy of acromegaly by using facial features instead of hand features. However, during the long-term research and development process of the inventor, it was further discovered that in the entire face of acromegaly patients, the columella-lip angle and nasofrontal angle are significantly smaller than those of normal people. When the prediction model is further focused on the nose of the face, the accuracy of acromegaly recognition can be further improved.

[0046] In this regard, a better prediction model uses attention mask design to make the model ignore other irrelevant information and focus only on the face of the portrait, especially the nose.

[0047] Face Attention ResNet combines the ResNet50 architecture with the attention mask mechanism, which improves the recognition ability and accuracy of the model by focusing on key facial areas, especially the nose. Specifically, Face Attention ResNet uses the deep residual network structure of ResNet50 to extract rich image features, and introduces the attention mask mechanism, which enables the model to automatically identify and focus on the facial areas that are most informative for diagnosis. This mechanism effectively filters out irrelevant or minor information, thereby improving the performance of the model in the acromegaly recognition task and realizing an automated process.

[0048] By adopting the method of this scheme, the training data set is doubled through the CT images of patients with acromegaly, and the data volume in the training data set is further expanded by combining directly taken facial photos. It has the advantages of easy sample acquisition and scalable feature acquisition, and creates a new data set construction method, which greatly improves the training data of the prediction model and improves the prediction accuracy of the trained prediction model.

[0049] The prediction model of this solution is based on Facial Attention ResNet, which combines the ResNet50 architecture with the attention mask mechanism to focus on the human nose, that is, to focus on analyzing the facial area with the most information for diagnosis, further improving the accuracy of model prediction.

[0050] Based on the prediction model obtained in the first aspect, the second aspect of the present invention discloses a prediction method for identifying acromegaly, which includes steps S21 to S22.

[0051] Step S21, obtaining a facial photo of the person to be screened.

[0052] The facial photo of the person to be screened may be a stored photo or a photo taken in real time.

[0053] Step S22, inputting the facial photo of the person to be screened into the prediction model obtained by using the prediction model training method for identifying acromegaly described in the first aspect, to obtain a prediction result of the person to be screened, wherein the prediction result is whether the person has acromegaly or not.

[0054] Based on the prediction method for identifying acromegaly in the second aspect, the third aspect of the present invention discloses a prediction device for identifying acromegaly, including a data acquisition unit, a memory and a controller, the data acquisition unit is used to collect facial photos, the memory stores a computer program, and the controller is used to read the facial photos and the computer program to execute the prediction method for identifying acromegaly described in the second aspect.

[0055] Specifically, the memory may include, but is not limited to, a random access memory (RAM), a read-only memory (ROM), a flash memory (Flash Memory), a first input first output (FIFO), and a first input last output (FILO), etc.; the controller may not be limited to a microcontroller of the STM32F105 series. In addition, the computer device may also include, but is not limited to, a power supply unit, a display screen, and other necessary components.

[0056] The prediction device for identifying acromegaly can also be a smart device such as a mobile phone, tablet, computer, or smart query terminal in a hospital. In order to achieve lightweight mobile terminals, it can be implemented based on WeChat applet. For the purpose of assisting patients in early self-examination, the classification model is embedded in the application. By optimizing the code, we ensure the running efficiency and response speed of the applet on mobile devices. Users only need to scan their faces with a selfie on their mobile phone to get a quick preliminary diagnosis. This applet is not only easy to operate, but also can provide personalized health advice to help users detect potential problems early and seek medical treatment in time. It has important clinical and social value.

[0057] For example, based on the method of the first aspect, 1200 images of the abnormal group and 1194 images of the control group are obtained, and the attention of the Face Attention ResNet is focused on the face and the nose respectively. The process of focusing attention on the nose is as follows: Figure 3 , 4 As shown, we select the whole face and the nose part to generate the attention mask, according to Figure 3 We have 68 feature points on the face of a human. We selected feature points 1-68 as the facial contour and feature points 28 to 36 as the nose area. The mask is initially an array of the same dimension as the grayscale version of the image, indicating the lowest attention. First, the entire facial contour is filled with an attention of 0.8, and then the nose part is filled with an attention of 1. In this way, the model can pay more attention to the facial features of the person in the image, especially the nose area. Finally, the mask array is normalized to the range of 0-255 and then converted into a PI L image format so that it can be passed into the model as a Tensor and processed together with the original image. In this way, the model can "pay attention" to facial features when processing images, especially the nose area, which is more important for distinguishing patients with acromegaly. Figure 4 This is the process of generating the attention mask, in which, first, 68 facial feature points are detected using the D lib library, the second step is to initialize the entire mask area to 0, and according to the facial feature points, the facial contour feature points 0-68 are selected and the mask is set to 0.8, the nose feature points 28-36 are selected and the mask is set to 1, and the third step is to generate the attention mask.

[0058] The prediction model Facial Attention ResNet is trained with the training data. When the Facial Attention ResNet prediction model focuses on the face, the prediction accuracy of acromegaly is 0.96, the precision is 0.96, the recall is 0.96, and the F1 Score is 0.96. When the Attention ResNet prediction model focuses on the nose, the prediction of acromegaly is performed after training with an accuracy of 0.98, a precision of 0.98, a recall of 0.98, and an F1 Score of 0.98. After focusing the attention of the Attention ResNet prediction model on the nose, its ROC curve characteristics are as follows Figure 2 shown.

[0059] Although the present invention has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments, or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A prediction model training method for identifying acromegaly, characterized in that: The following steps are involved: Obtaining a CT image set of acromegaly patients, wherein the CT image set includes CT images of at least one acromegaly patient; reconstructing a three-dimensional model of the patient's head according to the CT image set; Capturing facial photos of at least two different angles of each of the three-dimensional models to obtain a training data set; The training data set is input into the prediction model for model training.

2. A prediction model training method for identifying acromegaly according to claim 1, characterized in that: Before or after intercepting the facial photos of at least two different angles of each of the three-dimensional models to obtain the training data set, the following further comprises: Directly photographed facial photos of patients with acromegaly and / or facial photos taken at different angles from facial videos of patients with acromegaly are added to the training data set.

3. A prediction model training method for identifying acromegaly according to claim 1, characterized in that: The prediction model is Facial Attention ResNet, which focuses on the human nose.

4. A prediction model training method for identifying acromegaly according to claim 1, characterized in that: The step of intercepting at least two facial photos of each of the three-dimensional models from different angles to obtain a training data set comprises: The three-dimensional model is processed by data enhancement, and the data enhancement method includes adjusting at least one of brightness contrast, Gaussian noise, color jitter, Gaussian blur, and dynamic blur.

5. A prediction method for identifying acromegaly, characterized in that: The following steps are involved: Obtaining facial photos of the person to be screened; A facial photo of the person to be screened is input into a prediction model obtained by using a prediction model training method for identifying acromegaly as described in any one of claims 1 to 4 to obtain a prediction result of the person to be screened, wherein the prediction result is whether the person has acromegaly or not.

6. A prediction device for identifying acromegaly, characterized in that: It comprises a data acquisition unit, a memory and a controller, wherein the data acquisition unit is used to collect facial photos, the memory stores a computer program, and the controller is used to read the facial photos and the computer program to execute a prediction method for identifying acromegaly as described in claim 5.

Images