An early heart disease auxiliary diagnosis method based on neural network and infrared thermal imaging

Abstract

The present application relates to the field of infrared thermal imaging technology, especially to an early heart disease auxiliary diagnosis method based on neural network and infrared thermal imaging. In view of the problem that the popularization degree of infrared thermal imaging diagnosis theory is insufficient and it is difficult to realize rapid promotion, the following scheme is proposed, including the following steps: S1: using infrared thermal imaging equipment to collect human body trunk and upper limb infrared thermal image; S2: image data is preprocessed and segmented to obtain target region image; S3: image sample is classified and labeled, and a data set is established; S4: the data set is expanded; S5: a transform neural network model is constructed; S6: the neural network is trained using the data set; combined with the infrared thermal imaging equipment, the present application can realize the auxiliary diagnosis of early heart disease, realize the nondestructive testing without causing damage to the body, the diagnosis accuracy and image precision are significantly improved, and the popularization degree of infrared thermal imaging diagnosis theory is greatly improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of infrared thermal imaging technology, and particularly relates to an early heart disease auxiliary diagnosis method based on neural networks and infrared thermal imaging. BACKGROUND

[0002] At present, the main examination methods of heart disease include X-ray, ultrasound, CT, magnetic resonance and other image classification diagnosis technologies. The advantage of X-ray diagnosis is low cost, convenience, small radiation dose, and is suitable for most patients for routine examination. However, it has ionizing radiation, and is not suitable for women in the preparatory period, early pregnant women and special groups for examination, and there is image overlap and subtle structure cannot be identified. The advantage of CT diagnosis is fast imaging and high image quality, which solves the image overlap problem of X-ray examination to a certain extent, and CT examination is generally preferred for critically ill patients. However, CT is also not suitable for women in the preparatory period, early pregnant women and special groups for examination, and the soft tissue resolution is not high. The advantage of magnetic resonance diagnosis is that there is no ionizing radiation and high soft tissue resolution. However, the equipment and examination price is relatively high, patients with heart pacemaker or metal foreign body in the body, pregnant women within three months cannot do magnetic resonance examination, the scanning time is long, the patient needs to keep quiet, and it is not suitable for critically ill patients, patients with mental illness and infants for examination. Ultrasound diagnosis also has no ionizing radiation, can check whether the solid organs are normal and whether there are tumors, can display real-time images, and assist in judging the function of the organs. However, the image clarity is not high, and the hollow organs cannot be examined by B-ultrasound.

[0003] Infrared thermal imaging is a new imaging technology developed in recent years. The temperature field distribution of the normal human body has certain rules. When the human body begins to change, the body surface temperature will also change. Infrared thermal imaging realizes the early examination and diagnosis of heart disease. Since the diagnosis theory of infrared thermal imaging has been gradually developed and improved in recent years, the popularization degree among doctors is not high, and the number of experienced doctors is limited, which seriously limits the application and promotion of infrared thermal imaging technology in the medical examination field. Therefore, we propose an early heart disease auxiliary diagnosis method based on neural networks and infrared thermal imaging. SUMMARY

[0004] The purpose of the present application is to solve the problems of insufficient popularization of existing technology infrared thermal imaging diagnosis theory and difficulty in rapid promotion, and to propose an early heart disease auxiliary diagnosis method based on neural networks and infrared thermal imaging.

[0005] In order to achieve the above purpose, the present application adopts the following technical scheme:

[0006] An early heart disease auxiliary diagnosis method based on neural networks and infrared thermal imaging, comprising the following steps:

[0007] S1: acquiring human body trunk and upper limb infrared thermal image by using infrared thermal imaging equipment;

[0008] S2: pre-processing and segmentation of image data to obtain target region image;

[0009] S3: classifying and labeling image samples and establishing data set;

[0010] S4: expanding data set;

[0011] S5: constructing Transformer neural network model;

[0012] S6: training neural network using data set;

[0013] S7: inputting image to be diagnosed into trained effective model for auxiliary diagnosis.

[0014] Preferably, in S1, data is acquired, the patient stands naturally upright in a weak light collection environment, looks straight ahead, the upper limbs naturally droop, are placed on both sides of the body, the palms face forward, the fingers are separated, the feet are separated and stand with the same width as the shoulders, stand in front of the infrared imaging equipment at a certain distance, use the infrared thermal imaging diagnosis system in the computer equipment to acquire images, the initial resolution is 1600*900, and the patient number, name, gender, age, sampling time and evaluation suggestion of the physician expert are given; the infrared image in.jpg format and the evaluation suggestion in word format are obtained by cutting and downloading from the computer equipment, in this process, the image type is correct, the resolution size is lossless, and the image and the evaluation suggestion are strictly matched.

[0015] Preferably, in S2, image pre-processing, the data set is acquired, irrelevant information in the image is eliminated, real information required for heart auxiliary diagnosis is obtained, the detectability of relevant information is enhanced, the data is maximally simplified, the image is cropped to 310*150 from 1600*900 through image processing, the position of the feature image required for early heart disease diagnosis is extracted, and thus an image diagnosis position with good performance is selected.

[0016] Preferably, in S4, data augmentation, when an image sample is augmented, all operations are performed around the image sample itself, geometric transformation is performed on the image by using geometric transformation, including rotation, shifting, cropping, deformation and scaling operation, the scheme of cropping to a fixed size input is selected, the sample number of all data categories is close, and the influence of insufficient data amount on the training effect in the model training process is solved.

[0017] Preferably, in S5, the input of the transformer is converted into a feature vector by a word embedding method such as Word2Vec, and in self-attention, each word has three different vectors, which are multiplied by three different weight matrices W Q , W K , W V to obtain Query vector (Q), Key vector (K) and Value vector (V), all with a length of 64, wherein the three matrices have the same size and are calculated as follows:

[0018]

[0019] For the image classification task, a classifier unit is added to divide the adjusted human chest thermography image into subgraphs of a fixed size, adjust the dimensions of the connected subgraphs, and obtain an image embedding vector; perform two-dimensional position encoding to obtain a two-dimensional position encoding vector and connect the image embedding vector as the model input; the connected vector is input into the Transformer model to extract the image features, and finally the vector output by the model is converted into a probability representation through the classifier unit to complete the classification of the human chest infrared image.

[0020] Preferably, in S5, the model is designed for training and classification, a Transformer neural network is built, the input model after the above image processing is trained and recognized, the network structure is continuously adjusted and experimented to improve the accuracy, AUC value, precision and recognition rate, the data set is set, the sample size is set to 310x150, the image is processed, the data set is reasonably divided, and the 5-fold cross-validation method is used to divide the data set into 5 non-repeating and equal parts. Select 4 parts as the training set and the remaining 1 part as the test set. Repeat the same method 5 times, and the average value of the 5 experiments is used as the final result. Build a Pytorch framework on a Windows 1064-bit operating system, perform image classification tasks based on the above data set, and set the sample input size to 310x150.

[0021] Preferably, in S5, pre-training is performed, the Transformer neural network is pre-trained using the public data sets CPM-RadPath2020 and CheXpert, common features are extracted therefrom to obtain corresponding pre-training features, the network model is used to retrain the training set of the self-made data set obtained, and the weights of the model pre-trained in the first step are used as the initial parameters of the network to enhance the training efficiency and generalization ability; adjust the appropriate hyperparameters, use the Adam optimization algorithm, the training number is 200 cycles, the initial learning rate is 10 -3, the weight decay rate is 0.01; in the training stage, 200 cycles are set, each cycle has 10 segments, a total of 2000 segments, each segment is an meta task, each task contains 2 classes; in the test stage, 60 segments are randomly sampled in the test set, and the average accuracy with a confidence interval of 95% is used as the test result; the network model is used for classification of the test set of the obtained self-made data set, the processed image is input into the trained Transform neural network for feature extraction, the final features extracted are detected, and the picture is classified into the highest scoring class, and the class of the final heart part infrared image is output.

[0022] Preferably, in S5, evaluation indicators are established, and accuracy and precision are selected as indicators for evaluating the model, the accuracy is defined as the proportion of correctly identified positive and negative samples in the total samples, and the formula is as follows:

[0023]

[0024] The precision is the ratio of all correctly identified positive samples to all network positive samples, and the formula is as follows:

[0025]

[0026] The AUC value is the area under the ROC curve, as an evaluation indicator, the larger the AUC value, the better the model, and the AUC value pays more attention to the ratio of positive samples, wherein TP represents the number of samples predicted as positive samples in the positive samples, TN represents the number of samples predicted as negative samples in the negative samples, FP represents the number of samples predicted as positive samples in the negative samples, and FN represents the number of samples predicted as negative samples in the positive samples.

[0027] The beneficial effects of the present application are:

[0028] 1. The infrared thermal imager is used for non-destructive testing of the human body, and an infrared medical image with high imaging accuracy is generated, the infrared image is collected, the doctor combines the clinical experience to diagnose and analyze the heart disease, and the diagnosis suggestion is obtained.

[0029] 2. According to the amount of infrared radiation energy of the imaged object surface, the temperature distribution field of the measured object surface is obtained by using the infrared thermal imaging technology, the human body is non-destructively detected, the temperature distribution anomaly of the heart-related meridian area in the early stage of heart problems is detected, and early detection and diagnosis of heart disease can be realized.

[0030] The present application combines an infrared thermal imaging device, and can realize early auxiliary diagnosis of heart disease, realize non-destructive testing without causing damage to the body, significantly improve the diagnosis accuracy and image accuracy, and greatly improve the popularization degree of infrared thermal imaging diagnosis theory. BRIEF DESCRIPTION OF DRAWINGS

[0031] Figure 1 is a flowchart of an early heart disease auxiliary diagnosis method based on a neural network and infrared thermal imaging proposed by the present application;

[0032] Figure 2 is a human infrared thermal map of an early heart disease auxiliary diagnosis method based on a neural network and infrared thermal imaging proposed by the present application;

[0033] Figure 3 is a Transformer overall architecture diagram of an early heart disease auxiliary diagnosis method based on a neural network and infrared thermal imaging proposed by the present application;

[0034] Figure 4 is a neural network model schematic diagram of an early heart disease auxiliary diagnosis method based on a neural network and infrared thermal imaging proposed by the present application. DETAILED DESCRIPTION

[0035] The technical solutions in the embodiments of the present application will be described clearly and completely below. Obviously, the described embodiments are only part of the embodiments of the present application, not all.

[0036] Embodiment one

[0037] Reference Figures 1-4 An early heart disease auxiliary diagnosis method based on a neural network and infrared thermal imaging includes the following steps:

[0038] S1: Use an infrared thermal imaging device to collect human torso and upper limb infrared thermal image data;

[0039] S2: Preprocess and segment the image data to obtain target region images;

[0040] S3: Classify and label the image samples, and establish a data set;

[0041] S4: Expand the data set;

[0042] S5: Build a Transformer neural network model;

[0043] S6: Train the neural network using the data set;

[0044] S7: Input the image to be diagnosed into the trained effective model for auxiliary diagnosis.

[0045] In this embodiment, the data is obtained, the patient is in a weak light collection environment, the body is naturally upright, the eyes are straight forward, the upper limbs are naturally drooping, the palms are forward, the fingers are separated, the feet are separated and stand with the same width as the shoulders, and the infrared thermal imaging diagnosis system in the computer device is used to obtain the image, the initial resolution is 1600*900, and the patient number, name, gender, age, sampling time and doctor's expert evaluation suggestion are given; the infrared image in.jpg format and the evaluation suggestion in word format are obtained by cutting and downloading from the computer device. In this process, the image type is correct, the resolution size is lossless, and the image and the evaluation suggestion are strictly matched.

[0046] In this embodiment, the image preprocessing is performed, the data set is obtained, the irrelevant information in the image is eliminated, the real information required for heart auxiliary diagnosis is obtained, the detectability of the relevant information is enhanced, and the data is maximally simplified. Image cropping operation is performed through image processing, 1600*900 is cropped into 310*150 image, the position of the feature image required for early heart disease diagnosis is extracted, and the image diagnosis position with good selection is selected.

[0047] In this embodiment, when enhancing an image sample, all operations are performed around the image sample itself, geometric transformation is performed on the image using geometric transformation classes, including rotation, shift, cropping, deformation and scaling operation, the scheme of cropping into fixed size input is selected, the sample number of all data categories is close, and the influence of insufficient data amount on the training effect in the model training process is solved.

[0048] In this embodiment, the input of the transformer is converted into a feature vector by a word embedding method such as Word2Vec, in the self-attention, each word has three different vectors, which are obtained by multiplying the embedding vector X by three different weight matrices W Q , W K , W V to obtain Query vector (Q), Key vector (K) and Value vector (V), the lengths are all 64, the sizes of the three matrices are the same, and the calculation method is as follows:

[0049]

[0050] A classifier unit is added for the image classification task. The adjusted human chest thermography image is divided into sub-images of a fixed size. After connecting the sub-images, the dimension is adjusted to obtain an image embedding vector. Two-dimensional position encoding is performed to obtain a two-dimensional position encoding vector. The two-dimensional position encoding vector and the image embedding vector are connected to serve as the model input. The connected vector is input into the Transformer model to extract the image features. Finally, the vector output by the model is converted into a probability representation by the classifier unit to complete the classification of the human chest infrared image.

[0051] In this embodiment, the model is trained and classified, and the Transformer neural network is built. The input model after the above image processing is trained and recognized. Through continuous network structure adjustment and experiment, the accuracy, AUC value, precision rate and identification rate are improved. The data set is set, the sample size is set to 310x150, the image is shuffled, and the data set is reasonably divided. The 5-fold cross-validation method is used to divide the data set into 5 equal and non-repeating parts. Four of them are selected as the training set, and the remaining one is used as the test set. The same method is repeated 5 times, and the average value of the 5 experiments is used as the final result. The Pytorch framework is built on the Windows 1064-bit operating system, and the image classification task is performed based on the above data set. The sample input size is set to 310x150.

[0052] In this embodiment, the Transformer neural network is pre-trained using the public data sets CPM-RadPath2020 and CheXpert to extract common features and obtain corresponding pre-training features. The network model is used to retrain the training set of the self-made data set obtained, and the weights of the model pre-trained in the first step are used as the initial parameters of the network to enhance the training efficiency and generalization ability. The appropriate hyperparameters are adjusted and set. The Adam optimization algorithm is used, the training number is 200 cycles, the initial learning rate is 10 -3 , and the weight decay rate is 0.01. In the training stage, 200 cycles are set, each cycle has 10 segments, a total of 2000 segments, each segment is a meta-task, and each task contains 2 classes. In the test stage, 60 segments are randomly sampled in the test set, and the average accuracy with a confidence interval of 95% is used as the test result. The network model is used to classify the test set of the self-made data set obtained. The processed image is input into the trained Transformer neural network for feature extraction. The final features extracted are detected, and the image is classified into the highest scoring class. Meanwhile, the class of the final heart part infrared image is output.

[0053] In this embodiment, evaluation indicators are established, and accuracy and precision are selected as indicators for evaluating the proposed model. The accuracy is defined as the proportion of correctly identified positive and negative samples in the total samples, and the formula is as follows:

[0054]

[0055] The precision is the ratio of all correctly identified positive samples to all network positive samples, and the formula is as follows:

[0056]

[0057] The AUC value is the area under the ROC curve, and as an evaluation indicator, the larger the AUC value, the better the model. The AUC value places more emphasis on the ratio of positive samples. In the formula, TP represents the number of samples predicted as positive samples in the positive samples, TN represents the number of samples predicted as negative samples in the negative samples, FP represents the number of samples predicted as positive samples in the negative samples, and FN represents the number of samples predicted as negative samples in the positive samples.

[0058] Example Two

[0059] Reference Figures 1-4 An early heart disease auxiliary diagnosis method based on neural network and infrared thermal imaging, comprising the following steps:

[0060] S1: Collecting human torso and upper limb infrared thermal images using an infrared thermal imaging device;

[0061] S2: Preprocessing and segmenting the image data to obtain target region images;

[0062] S3: Classifying and labeling the image samples and establishing a data set;

[0063] S4: Expanding the data set;

[0064] S5: Constructing a Transformer neural network model;

[0065] S6: Training the neural network using the data set;

[0066] S7: Inputting the image to be diagnosed into the trained effective model for auxiliary diagnosis.

[0067] In this embodiment, the data is obtained, the patient is in a weak light collection environment, the body is naturally upright, the eyes are straight forward, the upper limbs are naturally drooping, the feet are separated and stand with the same width as the shoulders, and the infrared thermal imaging diagnosis system in the computer device is used to obtain images, the initial resolution is 1600*900, and the infrared image in.jpg format and the evaluation suggestion in word format are obtained by cutting and downloading from the computer device. In this process, the image type is correct, the resolution size is lossless, and the image and the evaluation suggestion are strictly matched.

[0068] In this embodiment, the image is preprocessed, the data set is obtained, the irrelevant information in the image is eliminated, the useful real information required for heart auxiliary diagnosis is obtained, the detectability of the relevant information is enhanced, and the data is maximally simplified. Through image processing, image cropping operation is performed, the position of the feature image required for early heart disease diagnosis is extracted, and the image diagnosis position with good image diagnosis position is selected.

[0069] In this embodiment, data augmentation, when enhancing an image sample, all operations are performed around the image sample itself, geometric transformation is performed on the image using geometric transformation, the scheme of cutting to a fixed size input is selected, the number of sample quantities of all data categories is close, and the influence of insufficient data quantity on the training effect in the model training process is solved.

[0070] In this embodiment, the input of the transformer is converted into a feature vector by a word embedding method such as Word2Vec, in the self-attention, each word has three different vectors, which are obtained by multiplying the embedding vector X by three different weight matrices W Q , W K , W V to obtain Query vector (Q), Key vector (K) and Value vector (V), all of which have a length of 64, wherein the three matrices have the same size, and the calculation method is as follows:

[0071]

[0072] A classifier unit is added for image classification tasks, the adjusted human chest thermal imaging picture is divided into a fixed size sub-picture, the dimensions of each sub-picture are adjusted after being connected, and a picture embedding vector is obtained; two-dimensional position coding is performed, the connected vector is input into the Transformer model, the picture features are extracted, and finally the vector output by the model is converted into a probability representation through the classifier unit during decoding, and the classification of the human chest infrared image is completed.

[0073] In this embodiment, the design model is trained and classified, a Transformer neural network is built, the input model after the above image processing is trained and recognized, the data set is set, the sample size is set to 310x150, the image is shuffled, the data set is reasonably divided, the 5-fold cross-validation method is used, the data set is divided into 5 parts that are not repeated and equal, 4 parts are selected as the training set, the remaining 1 part is used as the test set, the same method is repeated 5 times, the average value of 5 experiments is used as the final result, the Pytorch framework is built on the Windows 1064-bit operating system, and the image classification task is performed based on the above data set. The sample input size is set to 310x150.

[0074] In this embodiment, pre-training is performed, the Transformer neural network is pre-trained using the public data sets CPM-RadPath2020 and CheXpert to enhance training efficiency and generalization ability; adjust the appropriate hyperparameters, use the Adam optimization algorithm, the training round is 200 cycles, the initial learning rate is 10 -3 , the weight decay rate is 0.01; set 200 cycles in the training stage, 10 segments per cycle, a total of 2000 segments, each segment is a meta task, each task contains 2 classes; in the test stage, randomly sample 60 segments in the test set, and use the average accuracy with a confidence interval of 95% as the test result; use the network model to classify the test set of the self-made data set, input the processed image into the trained Transformer neural network for feature extraction, detect the final features extracted, and classify the picture into the class with the highest score. The class of the final heart part infrared image is output.

[0075] In this embodiment, evaluation indicators are established, accuracy and precision are selected as indicators for evaluating the proposed model, accuracy is defined as the proportion of correctly identified positive and negative samples to the total samples, and the formula is as follows:

[0076]

[0077] The precision is the ratio of all correctly identified positive samples to all network positive samples, and the formula is as follows:

[0078]

[0079] The AUC value is the area under the ROC curve, and the AUC value is larger, the better the model, and the AUC value pays more attention to the ratio of positive samples. In the formula, TP represents the number of samples predicted as positive samples in the positive samples, TN represents the number of samples predicted as negative samples in the negative samples, FP represents the number of samples predicted as positive samples in the negative samples, and FN represents the number of samples predicted as negative samples in the positive samples.

[0080] Example three

[0081] Reference Figures 1-4 An early heart disease auxiliary diagnosis method based on neural network and infrared thermal imaging, comprising the following steps:

[0082] S1: acquiring human body trunk and upper limb infrared thermal image by using infrared thermal imaging equipment;

[0083] S2: pre-processing and segmentation of image data to obtain target region image;

[0084] S3: classifying and labeling image samples, and establishing a data set;

[0085] S4: expanding the data set;

[0086] S5: constructing a Transformer neural network model;

[0087] S6: training the neural network using the data set;

[0088] S7: inputting the image to be diagnosed into the trained effective model for auxiliary diagnosis.

[0089] In this embodiment, the data is obtained, the patient is naturally upright in a weak light collection environment, looks straight ahead with both eyes, naturally drops both upper limbs on both sides of the body, palms forward, five fingers apart, and stands with both feet apart, as wide as the shoulders, the initial rate is 1600*900, and the patient number, name, gender, age, sampling time and physician expert evaluation suggestion are given; The infrared image in.jpg format and the evaluation suggestion in word format are obtained by cutting and downloading from the computer device. In this process, the image type is correct, the resolution size is lossless, and the image and the evaluation suggestion are strictly matched.

[0090] In this embodiment, the image is pre-processed, the data set is obtained, the irrelevant information in the image is eliminated, the real information required for heart auxiliary diagnosis is obtained, the detectability of the relevant information is enhanced, the data is maximally simplified, the image is cropped to 310*150 from 1600*900, the feature image position required for early heart disease diagnosis is extracted, and the image diagnosis position with good image diagnosis position is selected.

[0091] In this embodiment, data augmentation, when enhancing an image sample, all operations are performed around the image sample itself, using geometric transformation classes to perform geometric transformation on the image, including rotation, shift, cropping, deformation and scaling operations, the number of samples of all data categories is close, solving the influence of insufficient data amount on the training effect in the model training process.

[0092] In this embodiment, the input of the transformer is converted into a feature vector by a word embedding method such as Word2Vec, and in self-attention, each word has three different vectors, which are obtained by multiplying the embedding vector X by three different weight matrices W Q , W K , W V to obtain Query vector (Q), Key vector (K) and Value vector (V), all with a length of 64, wherein the three matrices have the same size, and the calculation method is as follows:

[0093]

[0094] For the image classification task, a classifier unit is added to divide the adjusted human chest thermography picture into sub-pictures of a fixed size, adjust the dimensions after connecting the sub-pictures, and obtain a picture embedding vector; perform two-dimensional position coding, input the connected vector into the Transformer model, extract the picture features, and finally convert the vector output by the model into a probability representation through the classifier unit to complete the classification of the human chest infrared image.

[0095] In this embodiment, the model is designed for training and classification, a Transformer neural network is built, the input model after the above image processing is trained and recognized, the data set is set, the sample size is set to 310x150, the data set is divided into 5 non-repeating and equal parts, 4 of which are selected as the training set, and the remaining 1 is used as the test set. Repeat 5 times in the same way, and the average value of the 5 experiments is used as the final result. Build a Pytorch framework on a Windows 1064-bit operating system, perform image classification tasks based on the above data set, and set the sample input size to 310x150.

[0096] In this embodiment, pre-training is performed, and the Transformer neural network is pre-trained using the public data sets CPM-RadPath2020 and CheXpert to extract common features and enhance training efficiency and generalization ability; adjust the appropriate hyperparameters, use the Adam optimization algorithm, and train for 200 cycles, with an initial learning rate of 10 -3, the weight decay rate is 0.01; 200 cycles are set in the training stage, each cycle has 10 segments, a total of 2000 segments, each segment is a meta task, each task contains 2 classes; in the test stage, 60 segments are randomly sampled in the test set, and the average accuracy with a confidence interval of 95% is used as the test result; using the network model to classify the test set of the self-made data set, the processed image is sent into the trained Transformer neural network for feature extraction, the final features extracted are detected, and the picture is classified into the highest scoring class, and the class of the final heart part infrared image is output.

[0097] In this embodiment, evaluation indexes are established, and accuracy and precision are selected as indexes for evaluating the proposed model. The accuracy is defined as the proportion of correctly identified positive and negative samples in the total samples, and the formula is as follows:

[0098]

[0099] The precision is the ratio of all correctly identified positive samples to all network positive samples, and the formula is as follows:

[0100]

[0101] The AUC value is the area under the ROC curve, which is used as an evaluation index. The larger the AUC value, the better the model. The AUC value pays more attention to the ratio of positive samples. In the formula, TP represents the number of samples predicted as positive samples in the positive samples, TN represents the number of samples predicted as negative samples in the negative samples, FP represents the number of samples predicted as positive samples in the negative samples, and FN represents the number of samples predicted as negative samples in the positive samples.

[0102] Comparative Example 1

[0103] The difference between Example 1 and Comparative Example 1 is that S1: using an infrared thermal imaging device to collect human infrared thermal image data, the patient is naturally upright in a weak light collection environment, looking straight ahead, with both upper limbs naturally hanging down on both sides of the body, palm facing forward, five fingers apart, and both feet apart standing with the same width as the shoulders, standing in front of the infrared imaging device, using the infrared thermal imaging diagnosis system in the computer device to obtain images, the initial resolution is 1600*900, and the patient number, name, gender, age, sampling time and evaluation suggestion of the physician expert are given; the infrared image in.jpg format and the evaluation suggestion in word format are obtained by cutting and downloading from the computer device.

[0104] Comparative Example 2

[0105] Different from example two, S2: pre-processing and image segmentation are performed on the collected image data to obtain a human target region image, a data set is obtained, irrelevant information in the image is eliminated, useful real information required for heart auxiliary diagnosis is obtained, the detectability of relevant information is enhanced, the data is maximally simplified, image cropping operation is performed through image processing, the position of the feature image required for early heart disease diagnosis is extracted, and thus an image diagnosis position with better image diagnosis position is selected.

[0106] Comparative example three

[0107] Different from example three, S4: the data set is expanded through an image data enhancement method, when an image sample is enhanced, all operations are performed around the image sample itself, geometric transformation is performed on the image using a geometric transformation class, including rotation, shifting, cropping, deformation and scaling operations, the number of samples of all data categories is close, and the influence of insufficient data on the training effect in the model training process is solved.

[0108] Experimental example

[0109] The early heart disease auxiliary diagnosis method based on a neural network and infrared thermal imaging of example one, example two and example three is tested, and the results are as follows:

[0110] Example One Example Two Example Three Prior methods Diagnostic accuracy 91％ 65％ 71％ 53％ Image accuracy 96％ 74％ 67％ 53％

[0111] The early heart disease auxiliary diagnosis method based on a neural network and infrared thermal imaging of example one, example two and example three has a significantly improved diagnosis accuracy and image precision compared with the existing early heart disease auxiliary diagnosis method based on a neural network and infrared thermal imaging, and example one is the best embodiment.

[0112] Detection report

[0113] The purpose of the present application is to solve the problems of insufficient popularization of infrared thermal imaging diagnosis theory and difficulty in rapid promotion, and an early heart disease auxiliary diagnosis method based on a neural network and infrared thermal imaging is proposed, which combines an infrared thermal imaging device and can realize auxiliary diagnosis of early heart disease, realizes non-destructive testing without causing damage to the body, significantly improves the diagnosis accuracy and image precision, and greatly improves the popularization of infrared thermal imaging diagnosis theory.

[0114] Finally, it should be noted that the above examples are only used to illustrate the technical solutions of the present application, and are not intended to limit the same; although the present application has been described in detail with reference to the foregoing examples, those of ordinary skill in the art should understand that the technical solutions recorded in the foregoing examples can still be modified, or some of the technical features can be replaced by equivalents, but these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for early auxiliary diagnosis of heart disease based on neural networks and infrared thermal imaging, characterized in that, Includes the following steps: S1: Use infrared thermal imaging equipment to acquire infrared thermal images of the human torso and upper limbs; S2: Preprocess and segment the image data to obtain the target region image; S3: Classify and label image samples, and build a dataset; S4: Expand the dataset; data augmentation, use geometric transformation classes to perform geometric transformations on the images, including rotation, shift, cropping, warping and scaling operations, select a cropping scheme to a fixed-size input, and ensure that the number of samples for all data categories is close; S5: Construct a Transformer neural network model; the input of the transformer is transformed into a feature vector by Word2Vec, which is then multiplied by the embedding vector X by three different weight matrices W. Q W K W V We obtain the Query vector (Q), Key vector (K), and Value vector (V), each with a length of 64. The three matrices have the same size, and the calculation method is as follows: ； To address the image classification task, a classifier unit was added. The adjusted thermal image of the human chest was divided into fixed-size sub-images. After concatenating the sub-images, the dimensions were adjusted to obtain image embedding vectors. Two-dimensional positional encoding was performed, and the resulting two-dimensional positional encoding vector was concatenated with the image embedding vector as model input. The concatenated vector was then fed into a Transformer model to extract image features. Finally, during decoding, the classifier unit converted the model's output vector into a probability representation, completing the classification of the human chest infrared image. A model was designed for training and classification. A Transformer neural network was built, and the input model after the above image processing was used for training and recognition. Through continuous network structure adjustments and experiments, the accuracy, AUC value, precision, and recognition speed metrics were improved. The dataset was set with a sample size of 310×150. The images were shuffled, and the dataset was then rationally divided using a 5-fold cross-validation method. The dataset was divided into five distinct and equal parts, with four parts selected as the training set and the remaining part as the test set. This process was repeated five times, and the average of the five experiments was used as the final result. The model was then tested on Windows 10. A PyTorch framework was built on a 64-bit operating system, and an image classification task was performed based on the above dataset. The sample input size was set to 310×150. S6: Train the neural network using the dataset; S7: Input the image to be diagnosed into the trained effective model for assisted diagnosis.

2. The method for early cardiac auxiliary diagnosis based on neural networks and infrared thermal imaging according to claim 1, characterized in that, In step S1, data is acquired by having the patient stand naturally upright in a low-light environment, with eyes looking straight ahead, arms hanging naturally at their sides, palms facing forward, fingers spread, and feet shoulder-width apart, in front of the infrared imaging device. The infrared thermal imaging diagnostic system in the computer device is used to capture images, with an initial resolution of 1600*900. The system also provides the patient's ID, name, gender, age, sampling time, and expert evaluation suggestions. The system then crops and downloads the infrared image in .jpg format and the evaluation suggestions in Word format from the computer device.

3. The method for early auxiliary diagnosis of heart disease based on neural networks and infrared thermal imaging according to claim 1, characterized in that, In step S2, image preprocessing is performed to obtain the dataset, remove irrelevant information from the image, and perform image cropping operations to crop the 1600*900 image to a 310*150 image, thereby extracting the location of the feature images required for early heart disease diagnosis.

4. The method for early auxiliary diagnosis of heart disease based on neural networks and infrared thermal imaging according to claim 1, characterized in that, In step S5, pre-training is performed using the publicly available datasets CPM-RadPath2020 and CheXpert to pre-train the Transformer neural network, extracting common features to obtain corresponding pre-trained features. The network model is then retrained on the training set of the self-made dataset, using the weights from the model pre-training in step one as the initial parameters of the network. Appropriate hyperparameters are adjusted and set, and the Adam optimization algorithm is used with 200 training epochs and an initial learning rate of 10. -3 The weight decay rate is 0.01; during the training phase, 200 epochs are set, with 10 segments per epoch, for a total of 2000 segments. Each segment is a meta-task, and each task contains 2 classes; 60 segments are randomly sampled from the test set, and the average accuracy with a 95% confidence interval is used as the test result. The network model is used to classify the test set of the self-made dataset. The processed images are fed into the trained Transformer neural network for feature extraction. The extracted features are detected and the images are classified into the class with the highest score. At the same time, the category of the final infrared image of the heart area is output.

5. The method for early auxiliary diagnosis of heart disease based on neural networks and infrared thermal imaging according to claim 4, characterized in that, In step S5, evaluation metrics are established, with accuracy and precision selected as the metrics for evaluating the proposed model. Accuracy is defined as the proportion of correctly identified positive and negative samples out of the total sample, as shown in the following formula: ； Precision is the ratio of all correctly identified positive samples to all samples that the network considers positive, and the formula is as follows: ; The AUC value is the area under the ROC curve and is used as an evaluation metric. In the formula, TP represents the number of positive samples predicted as positive, TN represents the number of negative samples predicted as negative, FP represents the number of negative samples predicted as positive, and FN represents the number of positive samples predicted as negative.

Images