Coronary heart disease tendency identification equipment based on hand image features

Abstract

The invention discloses coronary heart disease tendency recognition equipment based on hand image features, and belongs to the technical field of medical image processing and deep learning. According to the method, a hand image is collected through a hand diagnosis instrument camera, image features are extracted and analyzed through a deep learning model, and whether a detected person has the tendency of coronary heart disease or not is judged. The method comprises the specific steps of collecting hand image data of a plurality of detected persons, preprocessing images to extract key features, constructing a deep learning model based on ResNet50 for training and classification, and finally outputting a prediction result of the coronary heart disease tendency. The accuracy of the method on a test set can reach more than 85%, and the method has the advantages of high efficiency, non-invasiveness and low cost, is suitable for scenes of family health monitoring, community medical screening, hospital auxiliary diagnosis and the like, and promotes early screening and prevention of coronary heart disease.

Description

Technical Field

[0001] The present invention relates to the field of medical image processing and artificial intelligence technology, and in particular to a device for identifying coronary heart disease propensity based on hand image features using deep learning technology. Background Art

[0002] Coronary heart disease (CHD) is a common cardiovascular disease with an increasing incidence and mortality rate year by year. Traditional diagnostic methods such as coronary angiography and electrocardiogram (ECG), while highly accurate, are highly invasive, costly, and complex to operate, making them difficult to meet the early screening needs of large populations. In recent years, studies have found that hand features (such as nails, palm texture, and skin color) are correlated with CHD. For example, changes in nail bed color may reflect blood circulation status, while palm texture may be related to cardiovascular health. Therefore, non-invasive detection methods based on hand features have become a research hotspot.

[0003] Existing technologies often rely on manual observation for hand feature analysis, lacking automated and standardized tools. Some studies have attempted to analyze hand images using machine learning methods, but these methods are limited by feature extraction accuracy and model performance, resulting in low recognition accuracy and insufficient clinical application. Furthermore, existing methods suffer from limitations in dataset size, model generalization, and computational efficiency, limiting their widespread application.

[0004] Therefore, there is an urgent need for an efficient, accurate, and non-invasive method that uses deep learning technology to analyze hand image features to achieve early screening and tendency prediction of coronary heart disease. Summary of the Invention

[0005] To address the shortcomings of existing technologies, the present invention provides a coronary artery disease (CHD) propensity identification system based on hand image features. This system uses a hand diagnostic camera to capture hand images, extracts key features through a deep learning model, and classifies them to determine whether the subject has a CHD propensity. This system offers the advantages of being non-invasive, highly accurate, and low-cost, and can be used for early screening and auxiliary diagnosis of CHD.

[0006] The present invention is achieved through the following technical solutions:

[0007] A device for identifying a tendency toward coronary heart disease based on hand image features, the device comprising: a memory and a processor, the memory storing a computer program, and the processor implementing the following steps when executing the computer program:

[0008] Using the front and back image data of the subject's left and right hands, palm texture features and nail bed color texture features are generated. The palm texture features and nail bed color texture features are input into the trained ResNet50 model to identify the subject's coronary heart disease predisposition and coronary heart disease predisposition probability. Palm texture includes: vein distribution features including blue veins of the thumb, blue veins of thenar eminence, and blood vessels of the middle finger; thenar region features including thenar morphology, thenar color, thenar folds, hypothenar morphology, and hypothenar color; nail bed color includes nail color and nail texture;

[0009] Before training the ResNet50 model, the front and back image data of the hand are input into the ResNet50 model, and the hand features are enhanced using the YCrCb color space combined with the Otsu segmentation technology and the Clahe algorithm. The hand features are also annotated with category labels of no tendency to coronary heart disease and with tendency to coronary heart disease according to the clinical diagnosis results of the subjects.

[0010] The ResNet50 model includes 5 stages connected in sequence. The last convolutional layer of the last stage is connected in series with the global average pooling layer, Dropout layer, fully connected layer, Softmax layer, cross entropy loss function layer, and output layer. The ResNet50 model is also removed from the top fully connected layer. The output dimension of the fully connected layer is 2, corresponding to the two categories of "no tendency to coronary heart disease" and "with tendency to coronary heart disease", the softmax activation function outputs a probability distribution, and the cross entropy loss function layer outputs the difference between the probability distribution predicted by the ResNet50 model and the probability distribution of the true label, which is used to update the parameters of the ResNet50 model through back propagation.

[0011] The global average pooling layer is used to compress the feature map output by the last convolutional layer of the last stage into a 1-dimensional vector, reducing the dimension and reducing the risk of overfitting.

[0012] The Dropout layer sets the dropout rate to 0.3 and randomly drops 30% of the neurons to enhance the generalization ability of the model on hand image data.

[0013] The ResNet50 model sets the ResNet50 convolutional base as trainable, allowing all layers to be fine-tuned during training to adapt to the feature distribution of hand and back images. During the fine-tuning process, the residual connection Add layer of ResNet50 ensures effective gradient transfer.

[0014] The optimizer of the cross entropy loss function layer is Adam, the initial learning rate is 0.001, and the evaluation indicator is accuracy.

[0015] The front and back image data of the left and right hands are collected by a high-definition camera. The shooting conditions are natural light, the shooting distance is 20 cm, and the image storage format is JPEG. After the collection is completed, the image resolution is uniformly adjusted to 224×224 pixels.

[0016] The hand front and back image data input to the ResNet50 model are enhanced using the YCrCb color space combined with the Otsu segmentation technique and the Clahe algorithm to enhance hand features. Based on the clinical diagnosis results of the subjects, the categories of no coronary heart disease tendency and coronary heart disease tendency are annotated, including:

[0017] Step 2.1, Image adjustment and denoising: Convert the image to RGB format and use Gaussian blur method to remove image noise;

[0018] Step 2.2, Feature Enhancement: Using visible light and infrared image segmentation technology based on the YCrCb color space and the Otsu algorithm, and the Clahe algorithm to enhance the vein image, the contrast of hand features, including palm texture, is enhanced;

[0019] Step 2.3, data augmentation: randomly rotate the training set images by ±10°, translate them by ±10 pixels, and flip them horizontally to increase data diversity;

[0020] Step 2.4: Normalize the image pixel values ​​to the range [0, 1] and apply mean and standard deviation normalization to match the pre-trained input distribution of the ResNet50 model on the ImageNet dataset.

[0021] Step 2.5, Data Labeling: Label each image with a category label based on the clinical diagnosis of the subject, where 0 indicates no tendency to coronary heart disease and 1 indicates a tendency to coronary heart disease.

[0022] The specific implementation of step 2.2 includes:

[0023] Step 2.21. YCrCb color space conversion: Use the cv2.cvtColor function of the OpenCV library to convert the input RGB format image to the YCrCb color space to separate the luminance information and chrominance information;

[0024] Step 2.22, Otsu threshold segmentation: Apply the Otsu threshold segmentation algorithm in the Cr channel of the YCrCb image to automatically determine the optimal threshold and achieve binary segmentation of the hand area;

[0025] Step 2.23, Clahe algorithm vein enhancement: Use the Clahe algorithm to enhance the hand vein texture and improve the visibility of vein distribution.

[0026] The specific implementation of step 2.22 includes:

[0027] Step 2.221, extract the Cr channel data of the YCrCb image and generate a grayscale histogram;

[0028] Step 2.222. Use OpenCV's cv2.threshold function to calculate the threshold T that maximizes the between-class variance.

[0029] Step 2.223, based on the threshold T, binarize the Cr channel image into foreground and background to generate a binary mask; the foreground includes the hand area and has a value of 255; the background has a value of 0;

[0030] Step 2.224: Apply a binary mask to extract the hand area and filter out irrelevant background and noise.

[0031] The specific implementation of step 2.23 includes:

[0032] Step 2.231. Use OpenCV's cv2.cvtColor function to convert the BGR image to a grayscale image.

[0033] Step 2.232. Create a Clahe object, set the contrast limit parameter to 2.0, and the grid size to 8x8; use cv2.createCLAHE(clipLimit=2.0, tileGridSize=(8,8));

[0034] Step 2.233. Apply the Clahe algorithm to the grayscale image to enhance the contrast of the vein texture and highlight the hand vein features.

[0035] The present invention uses a hand diagnostic camera to capture hand images, utilizes the YCrCb color space combined with Otsu segmentation technology and the Clahe algorithm to enhance hand features, and combines the ResNet50 deep learning model for feature extraction and classification, achieving efficient prediction of coronary heart disease propensity. The innovations of the present invention are:

[0036] 1) An automatic hand region extraction method based on YCrCb+Otsu segmentation is proposed to reduce illumination and background interference;

[0037] 2) Enhance vein texture through Clahe algorithm to improve feature expression ability;

[0038] 3) Use ResNet50 transfer learning technology, combined with fine-tuning and Dropout mechanisms, to improve model accuracy and generalization capabilities.

[0039] The method of the present invention has the advantages of being non-invasive, highly precise (test set accuracy of 85.5%) and low-cost. A single test takes only 2 seconds and is suitable for scenarios such as family health monitoring, community medical screening and hospital auxiliary diagnosis, and has broad application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0041] Figure 1 This is the result of hand image acquisition, showing examples of hand images taken by a hand diagnosis device.

[0042] Figure 2 This is the original hand image before YCrCb+Otsu segmentation.

[0043] Figure 3 The hand image after YCrCb+Otsu segmentation shows the segmentation effect.

[0044] Figure 4 Grayscale hand image before Clahe enhancement.

[0045] Figure 5 This is a hand image enhanced by Clahe, showing the vein texture enhancement effect.

[0046] Figure 6 The overall process and ResNet50 model structure are demonstrated, showing the method process and network architecture.

[0047] Figure 7 This is the training accuracy curve, which shows the changes in accuracy during model training.

[0048] Figure 8 The training loss curve shows the change in loss during model training.

[0049] Figure 9 This is the appearance of the hand diagnosis instrument, showing the physical structure of the device. DETAILED DESCRIPTION

[0050] In order to facilitate ordinary technicians in this field to understand and implement the present invention, the present invention is further described in detail below with reference to the accompanying drawings and implementation examples. It should be understood that the implementation examples described herein are only used to illustrate and explain the present invention and are not used to limit the present invention.

[0051] Example 1

[0052] A method for identifying coronary heart disease tendency based on hand image features comprises the following steps:

[0053] Step 1: Collection of hand image data:

[0054] The hand image data of the subject is collected through the high-definition camera (resolution 1920×1080) on the hand diagnosis device, such as Figure 1 As shown in the figure, the data collection subjects included healthy individuals and patients with confirmed coronary heart disease, totaling 3,000 people, aged 18-70 years. The data originated from a clinical research project conducted in collaboration with the Affiliated Hospital of Beijing University of Chinese Medicine. All participants signed informed consent forms, and data collection complied with medical ethics standards and the requirements of the Personal Information Protection Law. For each person, one image of the palm and one of the back of the hand were collected, for a total of 6,000 images. The images were captured under natural lighting at a shooting distance of approximately 20 cm, and stored in PNG format. After collection, the image resolution was uniformly adjusted to 224×224 pixels to accommodate subsequent model input requirements.

[0055] Step 2: Preprocessing and labeling of hand image data:

[0056] Preprocess the hand image to enhance feature expression and improve model training results. The specific operations include:

[0057] Step 2.1, Image Adjustment and Denoising: Convert the image to RGB format, remove the transparent channel (if any); use Gaussian blur method (kernel size 5×5, standard deviation σ=1) to remove image noise and ensure image quality. Figure 2 shown.

[0058] Step 2.2, Feature Enhancement: Using visible light and infrared image segmentation technology based on the YCrCb color space and the Otsu algorithm, and performing vein image enhancement processing using the Clahe algorithm, the contrast of hand features (such as nail bed color and palm texture) is enhanced, improving the visibility of key features. Palm texture includes: vein distribution characteristics: blue veins on the thumb, blue veins on the thenar eminence, and blood vessels on the middle finger. Thenar region characteristics: thenar morphology, color, folds, hypothenar morphology, and color. Nail bed color: nail color and nail texture.

[0059] The specific steps include:

[0060] Step 2.21. YCrCb color space conversion: Use the cv2.cvtColor function in the OpenCV library to convert the input RGB image to the YCrCb color space to separate the luminance information (Y channel) and chrominance information (Cr and Cb channels). The YCrCb color space can effectively reduce the impact of lighting changes on image segmentation and enhance the distinction between the hand area and the background. The specific implementation method is as follows:

[0061] Step 2.211, read the input image (visible light or infrared image, format is RGB);

[0062] Step 2.212. Convert the RGB image to a YCrCb image using cv2.cvtColor(image,cv2.COLOR_RGB2YCrCb) where the Y channel represents the brightness and the Cr and Cb channels represent the red and blue chrominance components, respectively.

[0063] Step 2.213: Output the image in YCrCb format for subsequent processing.

[0064] Step 2.22, Otsu threshold segmentation: In the Cr channel of the YCrCb image, apply the Otsu threshold segmentation algorithm to automatically determine the optimal threshold and achieve binary segmentation of the hand area. The Cr channel is sensitive to the chromaticity information of the hand skin and can effectively distinguish the hand from the background. Figure 3 The specific steps include:

[0065] Step 2.221, extract the Cr channel data of the YCrCb image and generate a grayscale histogram;

[0066] Step 2.222. Use OpenCV's cv2.threshold function (parameter cv2.THRESH_OTSU) to calculate the threshold T that maximizes the between-class variance.

[0067] Step 2.223, based on the threshold T, binarize the Cr channel image into foreground (hand area, value 255) and background (value 0), and generate a binary mask;

[0068] Step 2.224: Apply a binary mask to extract the hand area and filter out irrelevant background (such as objects in the shooting environment) and noise.

[0069] Step 2.23, Clahe algorithm vein enhancement: Figure 4 As shown in Figure 2, the Clahe (Contrast Limited Adaptive Histogram Equalization) algorithm is used to enhance the hand vein texture, improve the visibility of vein distribution, and provide clearer details for subsequent feature extraction. Figure 5 The specific steps include:

[0070] Step 2.231. Use OpenCV's cv2.cvtColor function to convert the BGR image to a grayscale image.

[0071] Step 2.232. Create a Clahe object, set the contrast limit parameter to 2.0, and the grid size to 8x8; that is, use cv2.createCLAHE(clipLimit = 2.0, tileGridSize = (8, 8)) to create the Clahe object; the clipLimit parameter controls the contrast limit, and the tileGridSize parameter defines the size of the grid;

[0072] Step 2.233. Apply the Clahe algorithm to the grayscale image to enhance the contrast of the vein texture and highlight the hand vein features.

[0073] Step 2.3, data augmentation: Randomly rotate (±10°), translate (±10 pixels), and horizontally flip the training set images to increase data diversity, simulate different shooting angles and postures, improve the model's generalization ability for unknown data, and reduce the risk of overfitting.

[0074] Step 2.4, normalization: Normalize the image pixel values ​​to the [0, 1] interval (by image / 255.0), and apply mean and standard deviation normalization (mean [0.485, 0.456, 0.406], standard deviation [0.229, 0.224, 0.225]) to match the pre-training input distribution of the ResNet50 model on the ImageNet dataset to ensure stable model performance.

[0075] Step 2.5, Data Labeling: Based on the patient's clinical diagnosis, label each image with a category label, where 0 indicates no risk of CHD and 1 indicates risk of CHD. Use the to_categorical function to convert the labeled data into a one-hot encoding format for subsequent model training.

[0076] The above two algorithms are used to more clearly observe the following features:

[0077] Distribution characteristics of blue veins: blue veins on the thumb, blue veins on the thenar eminence, and blood vessels on the middle finger.

[0078] Characteristics of thenar region: thenar morphology, thenar color, thenar folds, hypothenar morphology, hypothenar color.

[0079] Nail characteristics: nail color, nail texture.

[0080] The blue veins on the thumb are usually related to the spleen and stomach function. In traditional Chinese medicine theory, the spleen and stomach are the "foundation of acquired constitution" and are responsible for digestion and absorption. If blue veins appear on the thumb, it may mean poor spleen and stomach function, such as indigestion, loss of appetite, abdominal distension, etc. The blue veins on the thumb mainly refer to the veins from the base of the thumb to the fingertips. It is usually easier to observe on the inner side of the thumb (the side close to the palm)

[0081] The veins on the thenar (the muscular bulge at the base of the thumb) are related to the lungs and respiratory system. Traditional Chinese medicine believes that the lungs control the skin and fur, and the veins on the thenar may reflect the functional state of the respiratory system. The thenar is the muscular bulge at the base of the thumb, located between the thumb and the palm.

[0082] The shape of the thenar eminence can reflect the function of the lungs and respiratory system. A full thenar eminence generally indicates good lung function; a shrunken or sunken thenar eminence may indicate poor lung function. The color of the thenar eminence is related to the respiratory system and the circulation of Qi and blood. Normally, the thenar eminence should be rosy. A whitish color may indicate Qi and blood deficiency; a reddish color may indicate inflammation or heat syndrome; and a purple color may indicate Qi and blood stasis.

[0083] The color of the thenar eminence is related to the respiratory system and the circulation of Qi and blood. Normally, the thenar eminence should be rosy. A whitish color may indicate Qi and blood deficiency; a reddish color may indicate inflammation or heat syndrome; and a purple color may indicate Qi and blood stasis.

[0084] Thenar wrinkles are related to lung function. Excessive wrinkles may indicate poor lung function, such as chronic coughing and asthma.

[0085] The shape of the hypothenar eminence (the muscular bulge at the base of the little finger) is related to kidney function. A full hypothenar eminence generally indicates good kidney function, while a shrunken or sunken hypothenar eminence may indicate poor kidney function. The hypothenar eminence is the muscular bulge at the base of the little finger, located between the little finger and the palm.

[0086] The color of the hypothenar eminence is related to kidney function and the reproductive system. Normally, the hypothenar eminence should be rosy. A whitish color may indicate kidney yang deficiency; a darker color may indicate kidney yin deficiency or impaired kidney function.

[0087] The blood vessels of the middle finger are often associated with the cardiovascular system. Clearly visible blood vessels in the middle finger may be related to blood circulation. Abnormal blood vessel color (such as purple or black) may indicate poor circulation or cardiovascular problems. The blood vessels in the middle finger primarily refer to the veins on the front of the middle finger, which are usually visible on both sides or the front of the middle finger.

[0088] The color of your nails can reflect the state of your Qi and blood. Normally, nails should be rosy and shiny. Pale nails may indicate Qi and blood deficiency; bluish nails may indicate cold syndrome or Qi and blood stasis; and purple nails may indicate poor circulation or hypoxia.

[0089] The texture of your nails can reflect your health. Normally, nails should be smooth and have no noticeable texture. Vertical lines on your nails may indicate a chronic illness or overwork; white spots on your nails may indicate a calcium deficiency or malnutrition; and horizontal lines on your nails may indicate a serious illness or trauma.

[0090] Step 3: Construction of deep learning model:

[0091] like Figure 6 As shown in the figure, a deep learning model based on ResNet50 was constructed for hand image feature extraction and classification. This step uses transfer learning, utilizing a pre-trained ResNet50 model and adapting it to the requirements of the coronary artery disease propensity identification task. ResNet50 was chosen because its deep network structure (50 layers) and residual connection design effectively extract complex features while alleviating the vanishing gradient problem in deep network training, making it suitable for feature extraction tasks in hand images.

[0092] Specifically, ResNet50 is an image classification algorithm based on deep convolutional neural network (CNN). It was proposed by Kaiming He et al. from Microsoft Research in 2015 and is an important member of the ResNet series. Compared with the traditional CNN model, ResNet50 has a deeper network structure. By introducing residual connections, it solves the gradient vanishing problem in the deep network training process and effectively improves the performance of the model. The specific implementation process is as follows Figure 6 shown.

[0093] ResNet50 consists of five stages. Stage 0 has a relatively simple structure and can be considered input preprocessing. The next four stages all consist of bottlenecks and have similar structures. Stage 1 contains three bottlenecks, and the remaining three stages have four, six, and three bottlenecks, respectively. We will now describe Stage 0 and Stage 1 in detail; the next three stages can be understood by analogy.

[0094] In Stage 0, (3,224,224) refers to the number of channels, height, and width of the input, or (C,H,W). Assuming the height and width of the input are equal, we use (C,W,W).

[0095] The first layer in this stage includes three operations in sequence: ①CONV is the abbreviation for Convolution, 7×7 refers to the kernel size, 64 refers to the number of kernels (i.e., the number of channels output by this convolution layer), and / 2 refers to the kernel stride of 2. ②BN is the abbreviation for Batch Normalization, commonly known as the BN layer. ③RELU refers to the ReLU activation function. The second layer in this stage is MaxPooling, or the maximum pooling layer, with a kernel size of 3×3 and a stride of 2. (64,56,56) is the number of channels, height, and width of the stage's output, where 64 equals the number of kernels in the first convolution layer of this stage, and 56 equals 224 / 2 / 2 (a stride of 2 halves the input size).

[0096] In summary, in Stage 0, the input with shape (3, 224, 224) passes through the convolution layer, BN layer, ReLU activation function, and MaxPooling layer to obtain the output with shape (64, 56, 56).

[0097] After understanding Stage 0 and being familiar with the meaning of various symbols in the diagram, it is easy to understand Stage 1. The specific structure of Bottleneck introduced in Stage 1 is as follows Figure 6 Shown on the right.

[0098] The two Bottleneck methods correspond to two situations: BTNK2 ("BTNK" is the abbreviation of BottleNeck) is used when the number of input and output channels is the same, and BTNK1 is used when the number of input and output channels is different.

[0099] BTNK2 has two variable parameters, C and W, representing the c and W in the input shape (C, W, W). Let the input of shape (C, W, W) be X, and the three convolutional blocks on the left side of BTNK2 (as well as the associated BN and ReLU) be the function F(X). The sum of these two (F(X) + X) and the subsequent activation with a ReLU function yield the output of BTNK2, which still has the shape (C, W, W). This corresponds to the case where BTNK2 has the same number of input and output channels as mentioned above.

[0100] BTNK1 has four variable parameters: C, W, C1, and S. Compared to BTNK2, BTNK1 has an additional convolutional layer on the right, representing the function G(X). BTNK1 accommodates situations where the input X and output F(X) have different channel counts. This added convolutional layer transforms X into G(X), matching the input and output dimensionality (G(X) and F(X) have the same number of channels), allowing for the sum G(X) + F(X).

[0101] After the output of the last convolutional layer of STAGE4, ​​the last stage of ResNet50, a global average pooling layer (GlobalAveragePooling), a dropout layer, a fully connected layer (Fully Connected), a softmax layer, a cross entropy loss function layer (Categorical Crossentropy), and an output layer are added.

[0102] The specific operations to improve the original ResNet50 model are as follows:

[0103] Step 3.1. Load the pre-trained ResNet50 model: Use ResNet50 as the base model and load the weights pre-trained on the ImageNet dataset to utilize its common visual features learned on large-scale image data (such as edge contours, color texture, region shape, and structural combination) to improve feature extraction efficiency and generalization ability.

[0104] Setting include_top = False removes the top fully connected layer of ResNet50, retaining only the convolutional base as a feature extractor. Because the ImageNet dataset has 1000 categories, this example migrates the weights of the model trained on ImageNet. The parameters of the final fully connected layer are suitable for 1000-class classification. Since this example is for binary classification, the fully connected layer is removed and replaced with a classification layer with 2 classes. The input image size of ResNet50 is 224×224×3, which is suitable for preprocessed hand images.

[0105] The convolutional base of the ResNet50 in each stage consists of multiple residual blocks (e.g., conv2_block1 to conv5_block3), which extract features layer by layer through convolutional layers, batch normalization, and ReLU activation functions. For example, the output shape of the first convolutional layer (conv1_conv) is (None, 112, 112, 64) with 9472 parameters; the output shape of the last convolutional layer (conv5_block3_out) is (None, 7, 7, 2048), providing high-level features for subsequent classification.

[0106] Step 3.2. Add a custom classification layer: After the output of the ResNet50 convolutional base, add a global average pooling layer (GlobalAveragePooling2D) to compress the feature map from (None, 7, 7, 2048) to a 1-dimensional vector of length 2048, reducing the dimensionality and reducing the risk of overfitting.

[0107] Add a Dropout layer, set the dropout rate to 0.3, and randomly drop 30% of the neurons to enhance the generalization ability of the model on hand image data.

[0108] Add a fully connected layer (Dense) and set the output dimension to 2, corresponding to the two categories of "no CAD tendency" and "CAD tendency". Use the softmax activation function to output the probability distribution. The number of parameters in the final layer is 12294.

[0109] Step 3.3: Model fine-tuning settings: Set the ResNet50 convolutional base to trainable (trainable = True), allowing all layers to be fine-tuned during training to adapt to the characteristic distribution of hand images (such as nail bed color and palm texture). During fine-tuning, the residual connections (Add layer) of ResNet50 ensure that gradients are effectively transferred, alleviating the vanishing gradient problem in deep network training.

[0110] Step 3.4 Model compilation: Set the loss function to the multi-class cross entropy loss function categorical_crossentropy, which is suitable for multi-class classification tasks, the optimizer to Adam, the initial learning rate to 0.001, and the evaluation indicator to accuracy.

[0111] The total number of model parameters is 23,600,006, of which 23,546,886 are trainable parameters and 53,120 are non-trainable parameters.

[0112] Step 4: Training of deep learning model:

[0113] Use the preprocessed hand image dataset to train the model and optimize the model parameters. The specific operations are as follows:

[0114] Step 4.1, Dataset Division: The training data is divided into a training set and a validation set in a ratio of 80% and 20%, and the training set and validation set are kept balanced in categories.

[0115] Step 4.2, training parameter settings: batch size is 32, number of training rounds is 40 epochs. Use the following callback function to optimize the training process:

[0116] ReduceLROnPlateau: monitors the validation set accuracy (val_accuracy), with a patience value of 5 and a learning rate reduction factor of 0.2;

[0117] EarlyStopping: Monitors the validation set accuracy with a patience value of 20. If the accuracy does not improve for 20 consecutive epochs, stop training and save the best model.

[0118] Step 4.3, training process: input the training set and validation set into the model, calculate the loss and update the parameters. During the training process, the training loss decreased from 0.8136 to 0.0076, and the validation loss decreased from 3780.39 to 0.5293; the training accuracy increased from 0.2031 to 1.0, and the validation accuracy increased from 0.25 to 0.5625. The final model had an accuracy of 85.5% on the test set and an AUC value of 0.89. Figure 7 、 Figure 8 shown.

[0119] Step 5: Hand image feature extraction and classification:

[0120] The pre-processed hand image of the person being tested is input into the trained ResNet50 model for feature extraction and classification:

[0121] Step 5.1, Feature Extraction: The image is passed through the convolutional layers of ResNet50 to extract low-level features (such as edges and textures) and high-level features (such as nail bed color distribution and palm texture pattern). A global average pooling layer compresses the features into a 1-dimensional vector.

[0122] Step 5.2, Classification Prediction: The feature vector passes through the fully connected layer, which outputs the probability distribution of two categories (no CHD tendency / with CHD tendency). If the probability value is greater than 0.5, the patient is judged to have CHD tendency, otherwise it is judged to have no CHD tendency.

[0123] Step 6: Output of coronary heart disease tendency prediction results:

[0124] The model prediction results are output in the form of probability values ​​and category judgments. The specific implementation is as follows:

[0125] Step 6.1, result generation: Through the software interface of the hand diagnosis instrument, the prediction results are displayed as "Probability of coronary heart disease tendency: X.XX" (such as "Probability of coronary heart disease tendency: 0.72") and "Prediction result: With / without coronary heart disease tendency" (such as "Prediction result: With coronary heart disease tendency").

[0126] Step 6.2, Output Method: Results are displayed in real time on the hand diagnosis device's display screen, with a single test taking approximately 2 seconds. The hand diagnosis device integrates a high-definition camera (resolution 1920×1080) and an embedded computing module, enabling fully automated operations from image acquisition to result output.

[0127] Example 2

[0128] A coronary heart disease propensity recognition system based on hand image features, the system being used to execute the recognition method described in Example 1, comprising:

[0129] Image acquisition module: used to collect image data of the front and back of the hands of the subjects. The subjects include healthy people and patients diagnosed with coronary heart disease. One image of each person's left and right hands is collected;

[0130] Image preprocessing module: used to adjust, denoise, enhance features and normalize the collected image data;

[0131] Deep learning model module: used to extract and classify hand image features through the ResNet50 model;

[0132] Result output module: used to output prediction results through the display screen and support real-time feedback.

[0133] Example 3

[0134] A hand diagnosis device, such as Figure 9 As shown, the system includes the system described in Example 2, and also includes: a high-definition camera with a resolution of 1920×1080.

[0135] Example 4

[0136] A device for identifying coronary heart disease tendency based on hand image features, the device comprising: a memory and a processor, the memory storing a computer program, and the processor performing the steps of the method described in Example 1 when executing the computer program.

[0137] Example 5

[0138] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the method described in Example 1.

[0139] The above embodiments are only used to illustrate the present invention and are not intended to limit the technical solutions described in the present invention. Although this specification has described the present invention in detail with reference to the above embodiments, the present invention is not limited to the above specific implementation methods. Therefore, any modification or equivalent replacement of the present invention; and all technical solutions and improvements thereof that do not depart from the spirit and scope of the invention are included in the scope of the claims of the present invention.

Claims

1. A device for identifying a tendency toward coronary heart disease based on hand image features, the device comprising: A memory and a processor, wherein the memory stores a computer program, and wherein the processor implements the following steps when executing the computer program: Using the front and back image data of the subject's left and right hands, palm texture features and nail bed color texture features are generated. The palm texture features and nail bed color texture features are input into the trained ResNet50 model to identify the subject's coronary heart disease predisposition and coronary heart disease predisposition probability. Palm texture includes: vein distribution features including blue veins of the thumb, blue veins of thenar eminence, and blood vessels of the middle finger; thenar region features including thenar morphology, thenar color, thenar folds, hypothenar morphology, and hypothenar color; nail bed color includes nail color and nail texture; Before training the ResNet50 model, the front and back image data of the hand are input into the ResNet50 model, and the hand features are enhanced using the YCrCb color space combined with the Otsu segmentation technology and the Clahe algorithm. The hand features are also annotated with category labels of no tendency to coronary heart disease and with tendency to coronary heart disease according to the clinical diagnosis results of the subjects. The ResNet50 model includes 5 stages connected in sequence. The last convolutional layer of the last stage is connected in series with the global average pooling layer, Dropout layer, fully connected layer, Softmax layer, cross entropy loss function layer, and output layer. The ResNet50 model is also removed from the top fully connected layer. The output dimension of the fully connected layer is 2, corresponding to the two categories of "no tendency to coronary heart disease" and "with tendency to coronary heart disease", respectively. The softmax activation function outputs a probability distribution, and the cross entropy loss function layer outputs the difference between the probability distribution predicted by the ResNet50 model and the probability distribution of the true label, which is used to update the parameters of the ResNet50 model through back propagation.

2. The device for identifying coronary heart disease tendency based on hand image features according to claim 1, characterized in that: The global average pooling layer is used to compress the feature map output by the last convolutional layer of the last stage into a 1-dimensional vector, reducing the dimension and reducing the risk of overfitting.

3. The device for identifying coronary heart disease tendency based on hand image features according to claim 1, characterized in that: The Dropout layer sets the dropout rate to 0.3 and randomly drops 30% of the neurons to enhance the generalization ability of the model on hand image data.

4. The device for identifying coronary heart disease tendency based on hand image features according to claim 1, characterized in that: The ResNet50 model sets the ResNet50 convolutional base as trainable, allowing all layers to be fine-tuned during training to adapt to the feature distribution of hand images. During the fine-tuning process, the residual connection Add layer of ResNet50 ensures effective gradient transfer.

5. The device for identifying coronary heart disease tendency based on hand image features according to claim 1, characterized in that: The optimizer of the cross entropy loss function layer is Adam, the initial learning rate is 0.001, and the evaluation indicator is accuracy.

6. The device for identifying coronary heart disease tendency based on hand image features according to claim 1, characterized in that: The front and back image data of the left and right hands are collected by a high-definition camera. The shooting conditions are natural light, the shooting distance is 20 cm, and the image storage format is JPEG. After the collection is completed, the image resolution is uniformly adjusted to 224×224 pixels.

7. The method for identifying coronary heart disease tendency based on hand image features according to claim 1, characterized in that: The hand front and back image data input to the ResNet50 model are enhanced using the YCrCb color space combined with the Otsu segmentation technique and the Clahe algorithm to enhance hand features. Based on the clinical diagnosis results of the subjects, the categories of no coronary heart disease tendency and coronary heart disease tendency are annotated, including: Step 2.1, Image adjustment and denoising: Convert the image to RGB format and use Gaussian blur method to remove image noise; Step 2.2, Feature Enhancement: Using visible light and infrared image segmentation technology based on the YCrCb color space and the Otsu algorithm, and the Clahe algorithm to enhance the vein image, the contrast of hand features, including palm texture, is enhanced; Step 2.3, data augmentation: randomly rotate the training set images by ±10°, translate them by ±10 pixels, and flip them horizontally to increase data diversity; Step 2.4: Normalize the image pixel values ​​to the range [0, 1] and apply mean and standard deviation normalization to match the pre-trained input distribution of the ResNet50 model on the ImageNet dataset. Step 2.5, Data Labeling: Label each image with a category label based on the clinical diagnosis of the subject, where 0 indicates no tendency to coronary heart disease and 1 indicates a tendency to coronary heart disease.

8. The method for identifying coronary heart disease tendency based on hand image features according to claim 7, characterized in that: The specific implementation of step 2.2 includes: Step 2.

21. YCrCb color space conversion: Use the cv2.cvtColor function of the OpenCV library to convert the input RGB format image to the YCrCb color space to separate the luminance information and chrominance information; Step 2.22, Otsu threshold segmentation: Apply the Otsu threshold segmentation algorithm in the Cr channel of the YCrCb image to automatically determine the optimal threshold and achieve binary segmentation of the hand area; Step 2.23, Clahe algorithm vein enhancement: Use the Clahe algorithm to enhance the hand vein texture and improve the visibility of vein distribution.

9. The method for identifying coronary heart disease tendency based on hand image features according to claim 8, characterized in that: The specific implementation of step 2.22 includes: Step 2.221, extract the Cr channel data of the YCrCb image and generate a grayscale histogram; Step 2.

222. Use OpenCV's cv2.threshold function to calculate the threshold T that maximizes the between-class variance. Step 2.223, based on the threshold T, binarize the Cr channel image into foreground and background to generate a binary mask; the foreground includes the hand area and has a value of 255; the background has a value of 0; Step 2.224: Apply a binary mask to extract the hand area and filter out irrelevant background and noise.

10. The method for identifying coronary heart disease tendency based on hand image features according to claim 9, characterized in that: The specific implementation of step 2.23 includes: Step 2.

231. Use OpenCV's cv2.cvtColor function to convert the BGR image to a grayscale image. Step 2.

232. Create a Clahe object, set the contrast limit parameter to 2.0, and the grid size to 8x8; use cv2.createCLAHE(clipLimit=2.0, tileGridSize=(8,8)); Step 2.

233. Apply the Clahe algorithm to the grayscale image to enhance the contrast of the vein texture and highlight the hand vein features.

Images