Breast analysis method based on infrared thermal image, electronic equipment and storage medium

Abstract

The invention provides a mammary gland analysis method based on an infrared thermal image, electronic equipment and a storage medium. The mammary gland analysis method comprises the steps that S1, the infrared thermal image including a light sensation value of a mammary gland area is acquired; s2, dividing the light sensation value into a plurality of color levels; s3, segmenting a mammary gland region and a blood vessel region of the infrared thermal image through a deep learning segmentation model; s4, extracting color gradation data of pixel points of the mammary gland area and the blood vessel area based on a segmentation result; and S5, analyzing the color gradation data through the decision tree model, judging the mammary gland type, and judging the blood vessel richness and the blood vessel symmetry based on the mammary gland type. And biological information for reference can be provided for auxiliary diagnosis more conveniently and quickly.

Description

Technical Field

[0001] The present invention relates to the technical field of image analysis, and in particular to a breast analysis method based on infrared thermal images, an electronic device and a storage medium. Background Art

[0002] Currently, the commonly used breast cancer screening imaging modalities, ultrasound and mammography, are morphological imaging modalities that provide only qualitative morphological analysis. While they are superior in detecting morphological abnormalities, they are unable to provide functional observations of abnormal cell metabolism. The main drawbacks of breast ultrasound are: 1. It is highly subjective. Breast ultrasound examinations often require one or two radiologists, and the examination primarily relies on imaging features. This is subject to significant subjective and human factors, and relies heavily on the physician's diagnostic skills and expertise, making misdiagnosis and missed diagnoses more likely in large-scale population surveys. 2. It has limitations. Clinically, ultrasound is superior for diagnosing established masses, such as breast cysts, abscesses, and cystic hyperplasia, but is less effective at detecting smaller masses or functional abnormalities, making missed diagnoses more likely. 3. The examination is lengthy. Because breast ultrasound primarily relies on imaging features for diagnosis, the average examination for a woman takes over 10 minutes. The ultrasound probe directly contacts the body during the examination, requiring the application of coupling gel, making the procedure more complex. The biggest disadvantage of molybdenum target X-ray examination is that X-rays cause certain radiation damage to the breast glands, which has the risk of inducing breast cancer, and multiple examinations cannot be performed. In addition, the breasts need to be squeezed during the examination, which will cause pain and discomfort to the examinee.

[0003] With the continuous development of artificial intelligence technology, it has found widespread application in various life scenarios, including medical image analysis, where deep learning has achieved the most fruitful results. Currently, deep learning primarily uses image classification to determine the presence of lesions in the patient; or to distinguish benign and malignant tumors from pathological sections of medical angiography; or, through image segmentation, to segment certain organs or lesions in medical images, thereby locating and highlighting the target. At this stage of deep learning technology development, building and training neural network models for specific task scenarios based on large amounts of annotated data is a key technical approach in this field.

[0004] In recent years, thermal imaging technology has shown promising prospects in early breast cancer screening and as an aid in breast cancer detection, thanks to its green, radiation-free, non-contact nature and ability to detect lesions early. Thermal imaging technology achieves disease screening and diagnosis by analyzing the metabolic state of internal tissue cells. When a lesion develops, the metabolic changes in local tissue cells occur first, and these functional changes precede morphological changes in the body's organs. When thermal imaging technology detects changes in cellular metabolism, other clinical examination methods may not reveal these abnormalities. This allows for early detection of the disease, guiding clinicians for further examination or treatment, and achieving the goal of early screening. Therefore, how to analyze infrared thermal images to assist in breast cancer screening is a pressing technical challenge. Summary of the Invention

[0005] To solve the above technical problems, the present invention provides a breast analysis method based on infrared thermal images, comprising the following steps:

[0006] Step S1: Acquire an infrared thermal image including light sensitivity values ​​of a breast area;

[0007] Step S2: Dividing the light sensitivity value into several color levels;

[0008] Step S3: Segmenting the infrared thermal image into breast area and blood vessel area using a deep learning segmentation model;

[0009] Step S4: extracting color scale data of pixel points in the breast area and blood vessel area based on the segmentation results;

[0010] Step S5: Analyze the color scale data using a decision tree model to determine the breast type, and determine the vascular richness and vascular symmetry based on the breast type.

[0011] Preferably, in step S3, the training method of the deep learning segmentation model includes:

[0012] a) The labeled data was divided into a training set and a validation set at a ratio of 9:1, and 10-fold cross validation was used;

[0013] b) The input data is a negative-format infrared thermal image that has been normalized and grayscale inverted;

[0014] c) The loss function uses the pixel cross entropy function, and the loss weight ratio of background, nipple and breast is set to 0.1:2.0:0.5;

[0015] d) The optimizer uses Adam. During the training process, the loss value and pixel accuracy of the training set and validation set are recorded, and an early stopping strategy is adopted.

[0016] Preferably, in step S3, when optimizing the deep learning segmentation model, the foreground target pixel accuracy index and the foreground target intersection index are used for optimization.

[0017] Preferably, in step S2, the light sensitivity value is divided into 8 color levels, and the method of dividing the light sensitivity value into 8 color levels includes:

[0018] a) Calculate y using the following formula:

[0019]

[0020] Wherein, x′ represents the normalized light value; when y <= 1, the color level is red 1; when 1 < y <= 2, the color level is red 2; when 2 < y <= 3, the color level is orange 1; when 3 < y <= 4, the color level is orange 2; when 4 < y <= 5, the color level is orange 3; when 5 < y <= 6, the color level is yellow 1; when 6 < y <= 7, the color level is yellow 2; when 7 < y, the color level is green.

[0021] Preferably, in step S5, the method for determining the breast type includes:

[0022] a) For each breast, calculate the breast classification index using the following formula:

[0023] y1 = yellow 2 pixels + total green pixels) / breast pixels;

[0024] y2 = yellow 2 pixels / (yellow 2 pixels + total green pixels);

[0025] y3 = yellow 2 pixels;

[0026] y4 = total green pixels / breast pixels;

[0027] b) When y1 < 0.07, or 0.07 ≤ y1 < 0.1 and y2 ≥ 0.9 and y3 < 2000, it is a uniform breast;

[0028] c) When 0.07≤y1<0.11 and y2<0.9, or 0.07≤y1<0.11 and y2≥0.9 and y3≥2000, or 0.1≤y1<0.35, or 0.35≤y1<0.5 and y4<0.25, or y1≥0.5 and y4<0.2, it is localized green breast;

[0029] d) When 0.35≤y1<0.5 and y4>0.25, or when y1≥0.5 and y4≥0.2, it is a cold milk mammary gland.

[0030] Preferably, the vascular richness is determined by the following method:

[0031] a) Using the straight line where the highest point of the breast region is located as the boundary between the inner and outer blood vessels, the pixel values ​​of each color level in the inner and outer blood vessel regions are extracted respectively;

[0032] b) Calculate the vascularity index using the following formula:

[0033] z1 = total blood vessel pixels / total breast pixels;

[0034] z2 = inner red 1 + inner red 2 + inner orange 1 blood vessel pixels / total inner blood vessel pixels;

[0035] z3 = inner red 1 + inner red 2 blood vessel pixels / total inner blood vessel pixels;

[0036] z4 = inner red 1 + inner red 2 + inner orange 1 + inner orange 2 blood vessel pixels / total breast pixels;

[0037] z5 = vascular pixels of inner red 1 + inner red 2 / vascular pixels of inner red 1 + inner red 2 + inner orange 1 + inner orange 2;

[0038] z6 = inner red 1 + inner red 2 + inner orange 1 + inner orange 2 + inner orange 3 blood vessel pixels / total breast pixels;

[0039] z7 = vascular pixels of inner red 1 + inner red 2 + inner orange 1 / vascular pixels of inner red 1 + inner red 2 + inner orange 1 + inner orange 2 + inner orange 3;

[0040] z8 = inner red 1 + inner red 2 + inner orange 1 blood vessel pixels / total breast pixels;

[0041] z9 = total inner vessel pixels;

[0042] z10 = inner red 1 + inner red 2 + inner orange 1 + inner orange 2 vascular pixels / total inner vascular pixels;

[0043] c) when the breast type is a homogeneous breast, determining the vascular richness by the following method: entering a first determination state when z1 < 0.065, or 0.065 ≤ z1 < 0.07 and z3 < 0.85, or z1 ≥ 0.09 and z3 < 0.85 and z2 < 0.5, or 0.07 ≤ z1 < 0.09 and z3 < 0.85 and z2 < 0.5, or 0.07 ≤ z1 < 0.09 and z3 < 0.85 and 0.5 ≤ z2 < 0.95 and z8 < 0.07;

[0044] When entering the first determination state, the following determination is made: when z9 < 2000, or 2000 ≤ z9 < 2700 and z3 < 0.9, or 2700 ≤ z9 < 3000 and z3 < 0.9 and z2 < 0.9, or 3000 ≤ z9 < 3500 and z3 < 0.8 and z2 < 0.9, or z9 ≥ 3500 and z3 < 0.8 and z2 < 0.9 and z10 < 0.9, the vascular richness is uniform or not rich; otherwise, it is uniform or rich.

[0045] d) When the breast type is partial green breast, the vascular richness is determined by the following method: when z4 ≥ 0.085, or 0.08 ≤ z4 < 0.085 and z5 ≥ 0.25, or 0.08 ≤ z4 < 0.085 and z5 < 0.25 and z2 ≥ 0.9 and z9 ≥ 3000, or 0.065 ≤ z4 < 0.08 and z5 ≥ 0.5, or 0.065 ≤ z4 < 0.08 and z5 < 0.5 and z2 ≥ 0.9 and z9 ≥ 3000, or z4 < 0.065 and z2 ≥ 0.9 and z9 ≥ 3000, the vascular richness is partial green rich, and the vascular richness in other states is partial green poor;

[0046] e) When the mammary gland type is cold milk mammary gland, the vascular richness is determined by the following method: when z6≥0.08, or z6<0.08 and z7≥0.15, the vascular richness is cold milk rich, and other states are cold milk poor.

[0047] Preferably, the symmetry of blood vessels is determined by the following method:

[0048] a) Using the straight line where the highest point of the breast region is located as the boundary between the inner and outer blood vessels, the pixel values ​​of each color level in the inner and outer blood vessel regions are extracted respectively;

[0049] b) Calculate the vascular asymmetry index using the following formula:

[0050] x1 = absolute value of inner left-inner right vessel pixels / total inner vessel pixels;

[0051] x2 = absolute value of the blood vessel pixel points of inner left red 1 + inner left red 2 - inner right red 1 - inner right red 2 / blood vessel pixel points of inner red 1 + red 2;

[0052] x3 = absolute value of the blood vessel pixel points of inner left red 1 + inner left red 2 + inner left orange 1 - inner right red 1 - inner right red 2 - inner right orange 1 / blood vessel pixel points of inner red 1 + inner red 2 + inner orange 1;

[0053] x4 = absolute value of the blood vessel pixel point of inner left orange 1 - inner right orange 1 / blood vessel pixel point of inner orange 1;

[0054] x5 = absolute value of the inner left blood vessel pixel - the inner right blood vessel pixel;

[0055] x6 = absolute value of the blood vessel pixel points of inner left red 1 + inner left red 2 - inner right red 1 - inner right red 2;

[0056] x7 = absolute value of the blood vessel pixel points of inner left red 1 + inner red 2 + inner orange 1 - inner right red - inner right red 2 - inner right orange 1;

[0057] x8 = absolute value of the blood vessel pixel point of inner left orange 1 - inner right orange 1;

[0058] x9 = total blood vessel pixels / total breast pixels;

[0059] x10 = absolute value of the blood vessel pixels of inner left red 1 + inner red 2 + inner orange 1 + inner orange 2 - inner right red 1 - inner right red 2 - inner right orange 1 - inner right orange 2 / absolute value of the blood vessel pixels of inner red 1 + inner red 2 + inner orange 1 + inner orange 2;

[0060] x11 = the absolute value of the blood vessel pixel points of inner left red 1 + inner left red 2 + inner orange 1 + inner orange 2 - inner right red 1 - inner right red 2 - inner right orange 1 - inner right orange 2;

[0061] x12 = the absolute value of the blood vessel pixel points of inner left red 1 + inner left red 2 + inner left orange 1 + inner left orange 2 + inner left orange 3 - inner right red 1 - inner right red 2 - inner right orange 1 - inner right orange 2 - inner right orange 3;

[0062] x13 = absolute value of the blood vessel pixel point of inner left orange 2 - inner right orange 2;

[0063] x14 = absolute value of the blood vessel pixel point of inner left orange 2 - inner right orange 2 / blood vessel pixel point of inner orange 2;

[0064] c) when the breast type is a uniform breast, determining the symmetry of the blood vessels by the following method: when x1 < 0.1, or 0.1 ≤ x1 < 0.2 and x5 ≥ 900 and x7 < 450, or 0.1 ≤ x1 < 0.2 and x5 < 900 and x5 < 600, or 0.1 ≤ x1 < 0.2 and 600 ≤ x5 < 900 and x7 < 600, or x1 ≥ 0.2 and x2 < 0.3, or x1 ≥ 0.2 and x2 ≥ 0.3 and x5 ≥ 900 and x6 < 350, or x1 ≥ 0.2 and x2 ≥ 0.3 and x5 < 850, or x1 ≥ 0.2 and x2 ≥ 0.3 and 850 ≤ x5 < 900 and x6 < 600, entering the first determination state;

[0065] When entering the first determination state, the following determination is performed: when x1 < 0.2, or x1 ≥ 0.2 and x3 < 0.2, or x1 ≥ 0.2 and x3 ≥ 0.2 and x5 < 1000, or x1 ≥ 0.2 and x3 ≥ 0.2 and x5 ≥ 1000 and x7 < 1000, or x1 ≥ 0.2 and x3 ≥ 0.2 and x5 ≥ 1000 and 1000 ≤ x7 < 1500 and x6 < 500, the second determination state is entered;

[0066] When entering the first determination state, the following determination is performed: when x2 ≥ 0.7 and x6 < 600, or x2 < 0.7 and x9 < 0.03, or 0.5 ≤ x2 < 0.7 and x9 ≥ 0.03 and x6 < 700, or x2 < 0.25 and x9 ≥ 0.03, or 0.25 ≤ x2 < 0.5 and x9 ≥ 0.03 and x6 < 800, the blood vessel symmetry is uniform symmetrical blood vessels, and other states are uniform asymmetric blood vessels;

[0067] d) When the breast type is partial green breast, determining vascular symmetry by the following method: when x1 < 0.2, or 0.2 ≤ x1 < 0.3 and x1 < 0.3 and x5 ≥ 1500, or 0.2 ≤ x1 < 0.3 and x5 < 1000, or 0.2 ≤ x1 < 0.3 and x5 ≥ 1000 and x3 < 0.3, or x1 ≥ 0.3 and x2 ≥ 0.25 and x5 < 1500, or x1 ≥ 0.3 and x2 < 0.25 and x3 < 0.3, or x1 ≥ 0.3 and x2 < 0.25 and x3 ≥ 0.3 and x5 < 1500, or x1 ≥ 0.3 and x2 < 0.25 and x3 ≥ 0.3 and x5 ≥ 1500 and x6 < 150, entering the first determination state;

[0068] When entering the first determination state, the following determination is performed: when x1 < 0.3 and x3 < 0.4, or x1 < 0.3 and x3 ≥ 0.4 and x5 < 500, or x1 < 0.3 and x3 ≥ 0.4 and x5 ≥ 500 and x7 < 800, or x1 ≥ 0.3 and x3 < 0.25, or x1 ≥ 0.3 and x3 ≥ 0.25 and x5 < 900, or x1 ≥ 0.3 and x3 ≥ 0.25 and x5 ≥ 900 and x7 < 900, the vascular symmetry is a locally green symmetrical vascular vessel, and other states are locally green asymmetric vascular vessels;

[0069] e) When the breast type is a cold milk breast, the vascular symmetry is determined by the following method: when x1 is less than 0.2, or 0.2≤x1<0.3 and x5≥1500 and x2<0.3, or 0.2≤x1<0.3 and x5<1000, or 0.2≤x1<0.3 and x5≥1000 and x3<0.3, or x1≥0.3 and x2≥0.25 and x5<1500, or x1≥0.3 and x2<0.25 and x3<0.3, or x1≥0.3 and x2<0.25 and x3≥0.3 and x5<1500, or x1≥0.3 and x2<0.25 and x3≥0.3 and x5≥1500 and x6<150, the vascular symmetry is cold milk symmetrical blood vessels, and other states are cold milk asymmetric blood vessels.

[0070] The present invention also provides an electronic device, comprising: a memory, a processor and a computer program, wherein the computer program is stored in the memory and configured to be executed by the processor to implement the above method.

[0071] The present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement the above method.

[0072] The beneficial effects of the present invention include at least: the present invention obtains the relationship between human body temperature and light sensitivity value based on thermal radiation theory, converts the information in the human body into visual information through light sensitivity value, and then conducts in-depth analysis of the light sensitivity value information. First, the light sensitivity value is converted into color scale, with the purpose of visualizing the difference in human body temperature in the form of color scale, so as to more intuitively understand the human body temperature situation. Then, the breast thermal image and blood vessel image are identified and extracted, with the purpose of converting the image data into numerical data. Finally, data preprocessing is performed on the numerical data, feature engineering is constructed, and judgment rules are constructed. Through numerical analysis, breast type, blood vessel richness and blood vessel symmetry are classified and identified, so as to more conveniently and quickly provide biological information for reference for auxiliary diagnosis. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] Figure 1 Schematic diagram of a method flow in an embodiment of the present invention;

[0074] Figure 2 This is a schematic diagram of an infrared thermal image obtained for training according to an embodiment of the present invention;

[0075] Figure 3 This is a schematic diagram of breast marking on an infrared thermal image according to an embodiment of the present invention;

[0076] Figure 4 This is a schematic diagram of blood vessel marking in an infrared thermal image according to an embodiment of the present invention;

[0077] Figure 5 Schematic diagram of the structure of a segmentation model according to an embodiment of the present invention;

[0078] Figure 6 Schematic diagram of the structure of the bottleneck module of the segmentation model according to an embodiment of the present invention;

[0079] Figure 7 A breast type classification decision tree according to an embodiment of the present invention;

[0080] Figure 8 A first decision tree for classifying vascular richness of uniform breasts according to an embodiment of the present invention;

[0081] Figure 9 A second decision tree for classifying vascular richness of uniform breasts according to an embodiment of the present invention;

[0082] Figure 10 A decision tree for vascular richness classification of partially green breasts according to an embodiment of the present invention;

[0083] Figure 11 A decision tree for classifying the vascular richness of a cold-milk mammary gland according to an embodiment of the present invention;

[0084] Figure 12 A first decision tree for classifying vascular symmetry of a uniform breast according to an embodiment of the present invention;

[0085] Figure 13 A second decision tree for classifying vascular symmetry of a uniform breast according to an embodiment of the present invention;

[0086] Figure 14 A third decision tree for classifying vascular symmetry of a uniform breast according to an embodiment of the present invention;

[0087] Figure 15 The first decision tree for vascular symmetry classification of a partially green breast according to an embodiment of the present invention;

[0088] Figure 16 The second decision tree for vascular symmetry classification of a partially green breast according to an embodiment of the present invention;

[0089] Figure 17 A decision tree for classifying blood vessel symmetry of a cold-fed mammary gland according to an embodiment of the present invention;

[0090] Figure 18 This is a schematic diagram of an infrared thermal image used for testing an embodiment of the present invention;

[0091] Figure 19 For the embodiment of the present invention Figure 18 Schematic diagram of the negative film obtained by conversion;

[0092] Figure 20 For the embodiment of the present invention Figure 18 Schematic diagram of breast identification after identification;

[0093] Figure 21 For the embodiment of the present invention Figure 18 Schematic diagram of blood vessel identification after identification;

[0094] Figure 22 For the embodiment of the present invention Figure 18 Schematic diagram of blood vessel color conversion after color conversion. DETAILED DESCRIPTION

[0095] The following is a clear and complete description of the technical solutions in the embodiments of the present invention, in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts are within the scope of protection of the present invention.

[0096] An embodiment of the present invention provides a breast analysis method based on infrared thermal images, comprising the following steps:

[0097] Step S1: Acquire an infrared thermal image including light sensitivity values ​​of a breast area.

[0098] Specifically, in this embodiment, the original database is data collected from medical institutions. After sorting and deleting unqualified data, 530 cases of data can be used for the breast infrared thermal image segmentation algorithm and 1,870 cases of data can be used for the blood vessel segmentation algorithm.

[0099] The data were cleaned. Specifically, during cleaning, the cleaning rules were as follows: query sensitive information, check body position and angle; cleaning method: manual observation of each image; quality control requirements: clean the original data to ensure that the data quality meets the requirements of breast infrared thermal image region segmentation algorithm and blood vessel segmentation algorithm training and testing; cleaning results: 17 cases of breast segmentation algorithm were deleted, leaving 513 cases; 99 cases of blood vessel segmentation algorithm were deleted, leaving 1781 cases.

[0100] A round of labeling and a round of review are carried out until all data are successfully and accurately labeled. The breast labeling results are as follows: Figure 3 As shown, the vascular labeling results are as follows Figure 4 shown.

[0101] Step S2: Divide the light sensitivity value into several color levels.

[0102] Specifically, the infrared thermal image contains two formats: a single-channel light sensitivity value matrix of the breast area and a three-channel pseudo-color matrix. The image of the dataset in this embodiment is a negative format image obtained by normalizing the light sensitivity value of the thermal image and performing grayscale inversion processing.

[0103] According to thermal radiation theory, the relationship between human body temperature and light sensitivity value can be obtained. By converting the light sensitivity value into color levels, the difference in human body temperature can finally be visualized in the form of color levels, giving a more intuitive understanding of human body temperature.

[0104] The light sensitivity value is then standardized using the following formula:

[0105]

[0106] Among them, x represents the original light sensitivity value, x ’ Indicates the light sensitivity value after processing, x max Indicates the maximum light sensitivity value, x min Indicates the minimum light sensitivity value.

[0107] After the light sensitivity values ​​are standardized, they are classified and divided using the K-means clustering analysis method, and then expressed in color scales. After continuous testing, it was finally determined that a K-means clustering K value of 8 would have the best effect. Therefore, in the embodiment of the present invention, there are 8 color scale names: red 1, red 2, orange 1, orange 2, orange 3, yellow 1, yellow 2, and green.

[0108] When dividing the color levels, the following formula is used for calculation:

[0109]

[0110] When y is less than or equal to 1, the color level is divided into red 1; when 1 is less than or equal to 2, the color level is divided into red 2; when 2 is less than or equal to 3, the color level is divided into orange 1; when 3 is less than or equal to 4, the color level is divided into orange 2; when 4 is less than or equal to 5, the color level is divided into orange 3; when 5 is less than or equal to 6, the color level is divided into yellow 1; when 6 is less than or equal to 7, the color level is divided into yellow 2; when 7 is less than or equal to green.

[0111] Step S3: Segment the infrared thermal image into breast area and blood vessel area using a deep learning segmentation model.

[0112] The segmentation model structure is as follows Figure 5 As shown, the structure of the bottleneck module is as follows Figure 6 As shown, no further details are given here.

[0113] During training, qualified data was divided into a 9:1 data set. The training data set was then cyclically trained and validated using the f-fold scheme. In this scheme, k=10, which means the data set was evenly divided into 10 parts. Nine of the parts were used for model training each time, and one part was used for performance verification during model training. After 10 cycles, the average of the indicators was taken as the final model training result.

[0114] The loss function is set as the pixel cross entropy function between the model prediction label and the annotated label, and the loss ratio of the three categories of objects, background, nipple and breast, is set to 0.1:2.0:0.5.

[0115] The Adam optimizer and 0.01 parameter decay were used during training. The loss value and pixel accuracy of the model on the training set and the tuning set were recorded during training, and the early stopping strategy was implemented.

[0116] 10% of the training dataset is used for algorithm optimization, and the optimization indicators are the foreground object pixel accuracy index PAFO and the foreground object intersection-over-union ratio IOUFO.

[0117] Breast area tuning indicators:

[0118] 1) Calculate PAFO on the tuning set

[0119] The percentage of foreground object FO pixels correctly classified by the segmentation model. The PAFO index is calculated based on the effective area of ​​the breast during testing.

[0120] The pixel accuracy of the effective breast area is recorded as PA breast The calculation formula is:

[0121]

[0122] Among them, PA breast represents the pixel accuracy of the breast area; P breast represents the set of pixels predicted by the model to be the breast area; L breast Represents a set of pixels labeled as breast area; Card() indicates the number of elements in the set.

[0123] 2) IOUFO indicator

[0124] The IOU of the predicted label and the FO pixel in the marked label. The IOUFO indicator is calculated for the effective breast area during the prediction process.

[0125] The pixel accuracy of the effective breast area is recorded as IOU breast , the calculation formula is:

[0126]

[0127] 3) Cross-validation tuning, rotation training tuning, and 10 rounds of cross-tuning tests.

[0128] Vascular region tuning index:

[0129] 1) Calculate PAFO on the tuning set

[0130] The percentage of FO pixels correctly classified by the segmentation model. The PAFO index is calculated based on the effective area of ​​the blood vessel during the test.

[0131] The pixel accuracy of the effective blood vessel area is recorded as PA vessels The calculation formula is:

[0132]

[0133] Where, PA vessels Indicates the pixel accuracy of the blood vessel area; P vessels represents the pixel set predicted by the model as a vascular region; L vessels Represents the set of pixels labeled as blood vessel regions.

[0134] 2) IOUFO indicator

[0135] The IOU of the predicted label and the FO pixel in the marked label. The IOUFO indicator is calculated for the effective area of ​​the blood vessel during the prediction process.

[0136] The pixel accuracy of the effective blood vessel area is recorded as IOU vessels , the calculation formula is:

[0137]

[0138] 3) Cross-validation tuning, rotation training tuning, and 10 rounds of cross-tuning tests.

[0139] Step S4: Extracting color scale data of the pixels in the breast area and blood vessel area based on the segmentation results

[0140] Step S5: Analyze the color scale data using a decision tree model to determine the breast type, and determine the vascular richness and vascular symmetry based on the breast type.

[0141] Specifically, according to the differences in the color gradation of infrared thermal images, the decision tree method is used to classify infrared thermal images into three different categories: uniform, partial green, and cold milk. The image category name only plays a classification role and has no decisive effect on the grading results.

[0142] According to the trained breast recognition model, the breast area of ​​the heat map is identified, and the midline of the human body in the heat map is used as the dividing line between the left and right breasts. The pixel values ​​of each color level in the left and right breast areas are extracted respectively as the original data for heat map classification.

[0143] The unilateral breast classification indicators are set as shown in Table 1.

[0144] Table 1

[0145] Indicator code Indicator name Indicator formula y1 Yellow-green total ratio (yellow 2 pixels + green total pixels) / total breast pixels y2 Yellow 2 ratio Yellow 2 pixels / (yellow 2 pixels + total green pixels) y3 Yellow 2 value Yellow 2 pixel value y4 Green Total green pixels / breast pixels

[0146] For each breast, the decision tree is as follows Figure 7 As shown in the figure, the input indicators of the enriched decision tree are y1, y2, y3, and y4 of the unilateral breast. After the decision tree is judged, if the output result is 0, the unilateral breast type is determined to be uniform; if the output result is 1, the unilateral breast type is determined to be partial green; if the output result is 2, the unilateral breast type is determined to be cold breast.

[0147] For vascular richness and vascular symmetry, the vascular area of ​​the heat map is identified based on the trained vascular recognition model. The midline of the human body in the heat map is used as the dividing line between the left and right blood vessels. The pixel values ​​of each color level in the left and right blood vessel areas are extracted respectively. The straight line where the highest point of the left and right breast areas is located is used as the dividing line between the inner and outer blood vessels. The pixel values ​​of each color level in the inner and outer blood vessel areas are extracted respectively as the original data of the vascular model.

[0148] Vascular richness indices are shown in Table 2 .

[0149] Table 2

[0150]

[0151]

[0152] When the breast type is uniform, the vascularity rule is given by Figure 8 and Figure 9 The first tree is composed of two trees. The input result is the four indicators z1, z2, z3 and z8 of the uniform heat map. If the output result is 1, it is judged that the result is uniformly rich. If the output result is 0, it enters the second tree. If the second tree returns a result of 1, it is judged that the result is uniformly rich. If the second tree returns a result of 0, it is judged that the result is uniformly not rich.

[0153] When the breast type is localized green, the vascularity rule is composed of Figure 10 The decision tree decides that the input is z2, z4, z5 and z9 of the local green heat map. If the decision tree returns a result of 1, it is judged that the local green is rich. If the return result is 0, it is judged that the local green is not rich.

[0154] When the breast type is cold milk, the vascularity rule is composed of Figure 11 The decision tree of is input as z6 and z7 of the cold milk heat map. If the decision tree returns a result of 1, it is determined that cold milk is abundant. If the decision tree returns a result of 0, it is determined that cold milk is not abundant.

[0155] For vascular symmetry, the index calculation is shown in Table 3.

[0156] Table 3

[0157]

[0158]

[0159] When the breast type is uniform, the vascular symmetry rule is composed of Figures 12 to 14 The three decision trees determine Figure 12 The first tree of the uniform heat map has four indicators x1, x2, x5, x6 and x7 as input. If the output is 1, it is determined that the result is uniformly asymmetric. If the output is 0, it enters Figure 13 If the second tree returns a result of 1, the result is determined to be uniformly asymmetric. If the second tree returns a result of 0, then enter Figure 14 If the third tree returns a result of 1, the result is determined to be uniformly asymmetric. If the third tree returns a result of 0, the result is determined to be uniformly symmetric.

[0160] When the breast type is localized, the vascular symmetry rule is composed of Figures 15 and 16 The three decision trees of Figure 15 The first tree of , the input result is the five indicators x1, x2, x5, x6 and x7 of the local green heat map. If the output result is 1, it is determined that the result is local green asymmetry. If the output result is 0, it enters Figure 16 If the second tree returns a result of 1, the result is determined to be asymmetric green. If the second tree returns a result of 0, the result is determined to be symmetric green.

[0161] When the breast type is cold milk, the vascular symmetry rule is composed of Figure 17 The decision tree determines that if the returned result is 1, the result is judged to be cold-rolled asymmetry; if the returned result is 0, the result is judged to be cold-rolled symmetry.

[0162] by Figure 18 The test infrared thermal image shown is used to illustrate the embodiment of the present invention, and the results after processing are as follows: Figures 19 to 22 The calculation of its various indicators is shown in Tables 4 to 8.

[0163] Table 4

[0164]

[0165] Table 5

[0166]

[0167] Table 6

[0168]

[0169]

[0170] Table 7

[0171]

[0172] Table 8

[0173]

[0174] The mammary gland of the tester is identified by the mammary gland recognition model, and through the heat map classification rule: when the right mammary gland meets the rule 0.07 = <y1 < 0.10 and y2 >= 0.9 and y3 < 2000, it returns 0; when the left mammary gland meets the rule y1 < 0.07, it returns 0. The heat map type of this tester for the left and right mammary glands is judged to be a uniform heat map, which also meets the uniform richness rule: Z1 >= 0.07 and Z8 >= 0.07 and Z2 >= 0.50, and it is judged to meet the rich blood vessel type. At the same time, it also meets the uniform asymmetry rule 0.20 > X1 >= 0.10 and X5 >= 600 and X7 >= 600, and it is judged to meet the asymmetric blood vessel type.

[0175] To sum up, the blood vessels of the tester are identified through the blood vessel model, and the blood vessels are extracted. Then, color is attached and visualized to the blood vessels through color scale classification. Finally, a blood vessel heat map with data information is obtained. Ultimately, the body temperature is converted into a light perception value, and the light perception value is converted into a color scale, so as to obtain a color scale heat map with temperature information, providing a relatively convenient and fast biological information for auxiliary diagnosis.

[0176] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. Only the preferred embodiments of the present invention are expressed. The description is relatively specific and detailed, but it should not be construed as a limitation to the scope of the present invention. As long as the combination of these technical features does not conflict, it should be considered as the scope described in this specification.

[0177] It should be noted that for those of ordinary skill in the art, without departing from the concept of the present invention, several modifications and improvements can still be made, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the appended claims.

Claims

1. A breast analysis method based on infrared thermal images, characterized by: The following steps are involved: Step S1: Acquire an infrared thermal image including light sensitivity values ​​of a breast area; Step S2: Dividing the light sensitivity value into several color levels; Step S3: Segmenting the infrared thermal image into breast area and blood vessel area using a deep learning segmentation model; Step S4: extracting color scale data of pixel points in the breast area and blood vessel area based on the segmentation results; Step S5: Analyze the color scale data using a decision tree model to determine the breast type, and determine the vascular richness and vascular symmetry based on the breast type.

2. The breast analysis method based on infrared thermal images according to claim 1, characterized in that: In step S3, the training method of the deep learning segmentation model includes: a) The labeled data was divided into a training set and a validation set at a ratio of 9:1, and 10-fold cross validation was used; b) The input data is a negative-format infrared thermal image that has been normalized and grayscale inverted; c) The loss function uses the pixel cross entropy function, and the loss weight ratio of background, nipple and breast is set to 0.1:2.0:0.5; d) The optimizer uses Adam. During the training process, the loss value and pixel accuracy of the training set and validation set are recorded, and an early stopping strategy is adopted.

3. The breast analysis method based on infrared thermal images according to claim 2, characterized in that: In step S3, when optimizing the deep learning segmentation model, the foreground target pixel accuracy index and the foreground target intersection index are used for optimization.

4. The breast analysis method based on infrared thermal images according to claim 1, characterized in that: In step S2, the light sensitivity value is divided into 8 color levels. The method of dividing the light sensitivity value into 8 color levels includes: a) Calculate y using the following formula: Wherein, x′ represents the normalized light value; when y <= 1, the color level is red 1; when 1 < y <= 2, the color level is red 2; when 2 < y <= 3, the color level is orange 1; when 3 < y <= 4, the color level is orange 2; when 4 < y <= 5, the color level is orange 3; when 5 < y <= 6, the color level is yellow 1; when 6 < y <= 7, the color level is yellow 2; when 7 < y, the color level is green.

5. The breast analysis method based on infrared thermal images according to claim 4, characterized in that: In step S5, the method for determining the breast type includes: a) For each breast, calculate the breast classification index using the following formula: y1 = yellow 2 pixels + total green pixels) / breast pixels; y2 = yellow 2 pixels / (yellow 2 pixels + total green pixels); y3 = yellow 2 pixels; y4 = total green pixels / breast pixels; b) When y1 < 0.07, or 0.07 ≤ y1 < 0.1 and y2 ≥ 0.9 and y3 < 2000, it is a uniform breast; c) When 0.07≤y1<0.11 and y2<0.9, or 0.07≤y1<0.11 and y2≥0.9 and y3≥2000, or 0.1≤y1<0.35, or 0.35≤y1<0.5 and y4<0.25, or y1≥0.5 and y4<0.2, it is localized green breast; d) When 0.35≤y1<0.5 and y4>0.25, or when y1≥0.5 and y4≥0.2, it is a cold milk mammary gland.

6. The breast analysis method based on infrared thermal images according to claim 5, characterized in that: Vascularity was determined by the following methods: a) Using the straight line where the highest point of the breast region is located as the boundary between the inner and outer blood vessels, the pixel values ​​of each color level in the inner and outer blood vessel regions are extracted respectively; b) Calculate the vascularity index using the following formula: z1 = total blood vessel pixels / total breast pixels; z2 = inner red 1 + inner red 2 + inner orange 1 blood vessel pixels / total inner blood vessel pixels; z3 = inner red 1 + inner red 2 blood vessel pixels / total inner blood vessel pixels; z4 = inner red 1 + inner red 2 + inner orange 1 + inner orange 2 blood vessel pixels / total breast pixels; z5 = vascular pixels of inner red 1 + inner red 2 / vascular pixels of inner red 1 + inner red 2 + inner orange 1 + inner orange 2; z6 = inner red 1 + inner red 2 + inner orange 1 + inner orange 2 + inner orange 3 blood vessel pixels / total breast pixels; z7 = vascular pixels of inner red 1 + inner red 2 + inner orange 1 / vascular pixels of inner red 1 + inner red 2 + inner orange 1 + inner orange 2 + inner orange 3; z8 = inner red 1 + inner red 2 + inner orange 1 blood vessel pixels / total breast pixels; z9 = total inner vessel pixels; z10 = inner red 1 + inner red 2 + inner orange 1 + inner orange 2 vascular pixels / total inner vascular pixels; c) when the breast type is a homogeneous breast, determining the vascular richness by the following method: entering a first determination state when z1 < 0.065, or 0.065 ≤ z1 < 0.07 and z3 < 0.85, or z1 ≥ 0.09 and z3 < 0.85 and z2 < 0.5, or 0.07 ≤ z1 < 0.09 and z3 < 0.85 and z2 < 0.5, or 0.07 ≤ z1 < 0.09 and z3 < 0.85 and 0.5 ≤ z2 < 0.95 and z8 < 0.07; When entering the first determination state, the following determination is made: when z9 < 2000, or 2000 ≤ z9 < 2700 and z3 < 0.9, or 2700 ≤ z9 < 3000 and z3 < 0.9 and z2 < 0.9, or 3000 ≤ z9 < 3500 and z3 < 0.8 and z2 < 0.9, or z9 ≥ 3500 and z3 < 0.8 and z2 < 0.9 and z10 < 0.9, the vascular richness is uniform or not rich; otherwise, it is uniform or rich. d) When the breast type is partial green breast, the vascular richness is determined by the following method: when z4 ≥ 0.085, or 0.08 ≤ z4 < 0.085 and z5 ≥ 0.25, or 0.08 ≤ z4 < 0.085 and z5 < 0.25 and z2 ≥ 0.9 and z9 ≥ 3000, or 0.065 ≤ z4 < 0.08 and z5 ≥ 0.5, or 0.065 ≤ z4 < 0.08 and z5 < 0.5 and z2 ≥ 0.9 and z9 ≥ 3000, or z4 < 0.065 and z2 ≥ 0.9 and z9 ≥ 3000, the vascular richness is partial green rich, and the vascular richness in other states is partial green poor; e) When the mammary gland type is cold milk mammary gland, the vascular richness is determined by the following method: when z6≥0.08, or z6<0.08 and z7≥0.15, the vascular richness is cold milk rich, and other states are cold milk poor.

7. The method for breast analysis based on infrared thermal images according to claim 5, characterized in that: Vascular symmetry is determined by: a) Using the straight line where the highest point of the breast region is located as the boundary between the inner and outer blood vessels, the pixel values ​​of each color level in the inner and outer blood vessel regions are extracted respectively; b) Calculate the vascular asymmetry index using the following formula: x1 = absolute value of inner left-inner right vessel pixels / total inner vessel pixels; x2 = absolute value of the blood vessel pixel points of inner left red 1 + inner left red 2 - inner right red 1 - inner right red 2 / blood vessel pixel points of inner red 1 + red 2; x3 = absolute value of the blood vessel pixel points of inner left red 1 + inner left red 2 + inner left orange 1 - inner right red 1 - inner right red 2 - inner right orange 1 / blood vessel pixel points of inner red 1 + inner red 2 + inner orange 1; x4 = absolute value of the blood vessel pixel point of inner left orange 1 - inner right orange 1 / blood vessel pixel point of inner orange 1; x5 = absolute value of the inner left blood vessel pixel - the inner right blood vessel pixel; x6 = absolute value of the blood vessel pixel points of inner left red 1 + inner left red 2 - inner right red 1 - inner right red 2; x7 = absolute value of the blood vessel pixel points of inner left red 1 + inner red 2 + inner orange 1 - inner right red - inner right red 2 - inner right orange 1; x8 = absolute value of the blood vessel pixel point of inner left orange 1 - inner right orange 1; x9 = total blood vessel pixels / total breast pixels; x10 = absolute value of the blood vessel pixels of inner left red 1 + inner red 2 + inner orange 1 + inner orange 2 - inner right red 1 - inner right red 2 - inner right orange 1 - inner right orange 2 / absolute value of the blood vessel pixels of inner red 1 + inner red 2 + inner orange 1 + inner orange 2; x11 = the absolute value of the blood vessel pixel points of inner left red 1 + inner left red 2 + inner orange 1 + inner orange 2 - inner right red 1 - inner right red 2 - inner right orange 1 - inner right orange 2; x12 = the absolute value of the blood vessel pixel points of inner left red 1 + inner left red 2 + inner left orange 1 + inner left orange 2 + inner left orange 3 - inner right red 1 - inner right red 2 - inner right orange 1 - inner right orange 2 - inner right orange 3; x13 = absolute value of the blood vessel pixel point of inner left orange 2 - inner right orange 2; x14 = absolute value of the blood vessel pixel point of inner left orange 2 - inner right orange 2 / blood vessel pixel point of inner orange 2; c) when the breast type is a uniform breast, determining the symmetry of the blood vessels by the following method: when x1 < 0.1, or 0.1 ≤ x1 < 0.2 and x5 ≥ 900 and x7 < 450, or 0.1 ≤ x1 < 0.2 and x5 < 900 and x5 < 600, or 0.1 ≤ x1 < 0.2 and 600 ≤ x5 < 900 and x7 < 600, or x1 ≥ 0.2 and x2 < 0.3, or x1 ≥ 0.2 and x2 ≥ 0.3 and x5 ≥ 900 and x6 < 350, or x1 ≥ 0.2 and x2 ≥ 0.3 and x5 < 850, or x1 ≥ 0.2 and x2 ≥ 0.3 and 850 ≤ x5 < 900 and x6 < 600, entering the first determination state; When entering the first determination state, the following determination is performed: when x1 < 0.2, or x1 ≥ 0.2 and x3 < 0.2, or x1 ≥ 0.2 and x3 ≥ 0.2 and x5 < 1000, or x1 ≥ 0.2 and x3 ≥ 0.2 and x5 ≥ 1000 and x7 < 1000, or x1 ≥ 0.2 and x3 ≥ 0.2 and x5 ≥ 1000 and 1000 ≤ x7 < 1500 and x6 < 500, the second determination state is entered; When entering the first determination state, the following determination is performed: when x2 ≥ 0.7 and x6 < 600, or x2 < 0.7 and x9 < 0.03, or 0.5 ≤ x2 < 0.7 and x9 ≥ 0.03 and x6 < 700, or x2 < 0.25 and x9 ≥ 0.03, or 0.25 ≤ x2 < 0.5 and x9 ≥ 0.03 and x6 < 800, the blood vessel symmetry is uniform symmetrical blood vessels, and other states are uniform asymmetric blood vessels; d) When the breast type is partial green breast, determining vascular symmetry by the following method: when x1 < 0.2, or 0.2 ≤ x1 < 0.3 and x1 < 0.3 and x5 ≥ 1500, or 0.2 ≤ x1 < 0.3 and x5 < 1000, or 0.2 ≤ x1 < 0.3 and x5 ≥ 1000 and x3 < 0.3, or x1 ≥ 0.3 and x2 ≥ 0.25 and x5 < 1500, or x1 ≥ 0.3 and x2 < 0.25 and x3 < 0.3, or x1 ≥ 0.3 and x2 < 0.25 and x3 ≥ 0.3 and x5 < 1500, or x1 ≥ 0.3 and x2 < 0.25 and x3 ≥ 0.3 and x5 ≥ 1500 and x6 < 150, entering the first determination state; When entering the first determination state, the following determination is performed: when x1 < 0.3 and x3 < 0.4, or x1 < 0.3 and x3 ≥ 0.4 and x5 < 500, or x1 < 0.3 and x3 ≥ 0.4 and x5 ≥ 500 and x7 < 800, or x1 ≥ 0.3 and x3 < 0.25, or x1 ≥ 0.3 and x3 ≥ 0.25 and x5 < 900, or x1 ≥ 0.3 and x3 ≥ 0.25 and x5 ≥ 900 and x7 < 900, the vascular symmetry is a locally green symmetrical vascular vessel, and other states are locally green asymmetric vascular vessels; e) When the breast type is a cold milk breast, the vascular symmetry is determined by the following method: when x1 is less than 0.2, or 0.2≤x1<0.3 and x5≥1500 and x2<0.3, or 0.2≤x1<0.3 and x5<1000, or 0.2≤x1<0.3 and x5≥1000 and x3<0.3, or x1≥0.3 and x2≥0.25 and x5<1500, or x1≥0.3 and x2<0.25 and x3<0.3, or x1≥0.3 and x2<0.25 and x3≥0.3 and x5<1500, or x1≥0.3 and x2<0.25 and x3≥0.3 and x5≥1500 and x6<150, the vascular symmetry is cold milk symmetrical blood vessels, and other states are cold milk asymmetric blood vessels.

8. An electronic device comprising: A memory, a processor and a computer program, characterized in that the computer program is stored in the memory and is configured to be executed by the processor to implement the method according to any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement the method according to any one of claims 1 to 7.

Images