A coronary heart disease tendency recognition device based on hand image features

Abstract

The application discloses a kind of coronary heart disease tendency identification equipment based on hand image feature, belong to medical image processing and depth learning technical field.The method is by hand diagnosis instrument camera hand image acquisition, utilize depth learning model to extract and analyze image feature, judge whether the person to be detected exists coronary heart disease tendency.Specific steps include: collect the hand image data of a plurality of persons to be detected, pre-process image to extract key features, construct the depth learning model based on ResNet50 and train and classify, finally output the prediction result of coronary heart disease tendency.The accuracy of the method on test set can reach more than 85%, with the advantages of high efficiency, non-invasive and low cost, applicable to family health monitoring, community medical screening and hospital auxiliary diagnosis etc.Situation, promote the early screening and prevention of coronary heart disease.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of medical image processing and artificial intelligence, and particularly relates to a device for recognizing coronary heart disease tendency based on hand image features by using deep learning technology. BACKGROUND

[0002] Coronary heart disease (CHD) is a common cardiovascular disease with increasing incidence and mortality. Traditional diagnostic methods such as coronary angiography and electrocardiogram have high accuracy, but are invasive, costly and complex to operate, making it difficult to meet the needs of early screening for large populations. In recent years, research has found that hand features such as nail, palm texture and skin color are related to coronary heart disease. For example, changes in nail bed color may reflect blood circulation, and palm texture may be related to cardiovascular health. Therefore, non-invasive detection methods based on hand features have become a research hotspot.

[0003] In the prior art, the analysis of hand features relies on manual observation, and lacks automated and standardized tools. Some studies have attempted to use machine learning methods to analyze hand images, but due to the accuracy of feature extraction and model performance, the recognition accuracy is low, making it difficult to meet the needs of clinical applications. In addition, existing methods have limitations in data set size, model generalization ability and computational efficiency, limiting their application.

[0004] Therefore, there is an urgent need for an efficient, accurate and non-invasive method to analyze hand image features using deep learning technology to achieve early screening and tendency prediction of coronary heart disease. SUMMARY

[0005] To overcome the shortcomings of the prior art, the present application provides a coronary heart disease tendency recognition system based on hand image features, which uses a hand diagnostic instrument camera to capture hand images, extracts key features through a deep learning model and classifies them to determine whether the subject has a tendency of coronary heart disease. The present application has the advantages of non-invasiveness, high precision and low cost, and can be used for early screening and auxiliary diagnosis of coronary heart disease.

[0006] The present application is achieved by the following technical solutions:

[0007] A coronary heart disease tendency recognition device based on hand image features, comprising a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the following steps:

[0008] The palm texture features and the nail bed color texture features are generated using the front and back hand image data of the left and right hands of the detected person, and the palm texture features and the nail bed color texture features are input into the trained ResNet50 model to identify the coronary heart disease tendency of the detected person and the coronary heart disease tendency probability; the palm texture includes: the blue vein distribution features including the thumb blue vein, the thenar blue vein and the middle finger blood vessel; the thenar region features including the thenar shape, the thenar color and the thenar wrinkle; the nail bed color includes the nail color and the nail texture;

[0009] Before the training of the ResNet50 model, the front and back hand image data input into the ResNet50 model is enhanced in hand features by using the YCrCb color space in combination with the Otsu segmentation technology and the Clahe algorithm, and the class labels are labeled as no coronary heart disease tendency and coronary heart disease tendency according to the clinical diagnosis results of the detected person;

[0010] The ResNet50 model includes five stages connected in sequence, and the last convolution layer of the last stage is connected in sequence with a global average pooling layer, a Dropout layer, a full connection layer, a Softmax layer, a cross-entropy loss function layer and an output layer; the ResNet50 model is further removed from the top full connection layer, the output dimension of the full connection layer is 2, corresponding to two classes of "no coronary heart disease tendency" and "coronary heart disease tendency", the softmax activation function outputs the probability distribution, and the cross-entropy loss function layer outputs the difference between the probability distribution predicted by the ResNet50 model and the probability distribution of the real label, which is used to update the parameters of the ResNet50 model through back propagation.

[0011] The global average pooling layer is used to compress the feature map output by the last convolution layer of the last stage into a 1-dimensional vector, reduce the dimension and reduce the risk of overfitting.

[0012] The Dropout layer sets the dropout rate to 0.3, and randomly discards 30% of the neurons to enhance the generalization ability of the model on the hand image data.

[0013] The ResNet50 model sets the ResNet50 convolution base as trainable, allowing all layers to be fine-tuned during the training process to adapt to the feature distribution of the front and back hand image, and the residual connection Add layer of the ResNet50 ensures effective gradient transmission during the fine-tuning process.

[0014] The optimizer of the cross-entropy loss function layer is Adam, the initial learning rate is 0.001, and the evaluation index is accuracy.

[0015] The left and right hand front and back image data are collected by a high-definition camera, the shooting conditions are natural light, the shooting distance is 20 cm, the image storage format is JPEG, and after the collection is completed, the image resolution is uniformly adjusted to 224x224 pixels.

[0016] The input ResNet50 model hand front and back image data is enhanced by using YCrCb color space combined with Otsu segmentation technology and Clahe algorithm to enhance hand features, and according to the clinical diagnosis result of the detected person, the class label is labeled as no coronary heart disease tendency and coronary heart disease tendency including:

[0017] Step 2.1, image adjustment and denoising: convert the image to RGB format, and remove image noise by Gaussian blur method;

[0018] Step 2.2, feature enhancement: through the visible light and infrared image segmentation technology based on YCrCb color space and Otsu algorithm, and the Clahe algorithm for vein image enhancement processing, the contrast of hand features including palm texture is enhanced;

[0019] Step 2.3, data enhancement: randomly rotate the training set images by ±10°, translate by ±10 pixels and horizontally flip, to increase data diversity;

[0020] Step 2.4, normalization processing: normalize the image pixel value to the interval [0, 1], and apply mean and standard deviation normalization to match the pre-training input distribution of ResNet50 model on ImageNet dataset;

[0021] Step 2.5, data labeling: according to the clinical diagnosis result of the detected person, label the class label for each image, 0 represents no coronary heart disease tendency, and 1 represents coronary heart disease tendency.

[0022] The specific implementation of step 2.2 includes:

[0023] Step 2.21, YCrCb color space conversion: use the cv2.cvtColor function of OpenCV library to convert the input RGB format image to YCrCb color space to separate brightness information and chrominance information;

[0024] Step 2.22, Otsu threshold segmentation: in the Cr channel of YCrCb image, apply Otsu threshold segmentation algorithm to automatically determine the best threshold to realize binaryzation segmentation of hand region;

[0025] Step 2.23, Clahe algorithm vein enhancement: use Clahe algorithm to enhance hand vein texture and improve the visibility of vein distribution.

[0026] The step 2.22 is specifically implemented as follows:

[0027] Step 2.221, extract the Cr channel data of the YCrCb image to generate a gray histogram;

[0028] Step 2.222, calculate the threshold T of the maximum inter-class variance using the cv2.threshold function of OpenCV;

[0029] Step 2.223, according to the threshold T, binarize the Cr channel image into foreground and background to generate a binary mask; the foreground includes the hand region and the value is 255; the background value is 0;

[0030] Step 2.224, apply the binary mask to extract the hand region and filter out irrelevant background and noise.

[0031] The step 2.23 is specifically implemented as follows:

[0032] Step 2.231, convert the BGR format image to a gray image using the cv2.cvtColor function of OpenCV;

[0033] Step 2.232, create a Clahe object and set the contrast limit parameter to 2.0 and the grid size to 8x8; use cv2.createCLAHE(clipLimit=2.0,tileGridSize=(8,8));

[0034] Step 2.233, apply the Clahe algorithm to the gray image to enhance the contrast of the vein texture and highlight the hand vein features.

[0035] The present application collects hand images through a camera of a hand diagnosis instrument, enhances hand features by using YCrCb color space combined with Otsu segmentation technology and Clahe algorithm, extracts and classifies features by combining a ResNet50 deep learning model, and realizes efficient prediction of coronary heart disease tendency. The innovation of the present application lies in that:

[0036] 1) An automatic hand region extraction method based on YCrCb+Otsu segmentation is proposed to reduce light and background interference;

[0037] 2) The Clahe algorithm is used to enhance the vein texture and improve the feature expression ability;

[0038] 3) ResNet50 transfer learning technology is used to combine fine-tuning and Dropout mechanism to improve model accuracy and generalization ability.

[0039] The method has the advantages of non-invasiveness, high precision (test set accuracy 85.5%), and low cost, and only takes 2 seconds for single detection, and is suitable for family health monitoring, community medical screening, and hospital auxiliary diagnosis, and has wide application prospects. BRIEF DESCRIPTION OF DRAWINGS

[0040] The drawings described herein are used to provide further understanding of the present application, and form a part of the present application, the illustrative embodiments of the present application and the description thereof are used to explain the present application, and do not constitute improper limitation on the present application. In the drawings:

[0041] Figure 1 It is the hand image acquisition result, and shows the hand image example shot by the hand diagnosis instrument.

[0042] Figure 2 It is the hand original image before YCrCb+Otsu segmentation.

[0043] Figure 3 It is the hand image after YCrCb+Otsu segmentation, and shows the segmentation effect.

[0044] Figure 4 It is the gray hand image before Clahe enhancement.

[0045] Figure 5 It is the hand image after Clahe enhancement, and shows the vein texture enhancement effect.

[0046] Figure 6 It is the overall process and ResNet50 model structure, and shows the method flow and network architecture.

[0047] Figure 7 It is the training accuracy curve, and shows the accuracy change in the model training process.

[0048] Figure 8 It is the training loss curve, and shows the loss change in the model training process.

[0049] Figure 9 It is the appearance of the hand diagnosis instrument, and shows the physical structure of the device. DETAILED DESCRIPTION

[0050] In order to facilitate those skilled in the art to understand and implement the present application, the present application will be further described in detail below in combination with the drawings and implementation examples, and it should be understood that the implementation examples described herein are only used to illustrate and explain the present application, and do not limit the present application.

[0051] Embodiment 1

[0052] A coronary heart disease tendency recognition method based on hand image features, comprising the following steps:

[0053] Step 1, Collection of Hand Image Data:

[0054] The hand image data of the subjects was collected by a high-definition camera (resolution: 1920x1080) on the hand diagnosis instrument device, as shown in Figure 1 The subjects included healthy people and diagnosed coronary heart disease patients, a total of 3000 people, aged 18-70 years old. The data came from a clinical research project cooperated with the Affiliated Hospital of Beijing University of Chinese Medicine, and all participants signed the informed consent form, and the data collection met the requirements of medical ethics and the Personal Information Protection Law. Each person collected one image of the palm and the back of the hand, a total of 6000 images. The shooting conditions were natural light, the shooting distance was about 20 centimeters, and the image storage format was PNG. After the collection was completed, the image resolution was uniformly adjusted to 224x224 pixels to adapt to the subsequent model input requirements.

[0055] Step 2, Preprocessing and Annotation of Hand Image Data:

[0056] The hand image was preprocessed to enhance feature expression and improve model training effect, including:

[0057] Step 2.1, Image Adjustment and Denoising: Convert the image to RGB format, remove the transparent channel (if present); use the Gaussian blur method (kernel size 5x5, standard deviation σ=1) to remove image noise and ensure image quality. As shown in Figure 2 .

[0058] Step 2.2, Feature Enhancement: Through visible light and infrared image segmentation technology based on YCrCb color space and Otsu algorithm, and Clahe algorithm for vein image enhancement processing, enhance the contrast of hand features (such as nail bed color, palm texture), and improve the visibility of key features. Palm texture includes: distribution characteristics of blue veins: thumb blue veins, thenar blue veins, middle finger blood vessels. Thenar region features: thenar shape, thenar color, thenar wrinkles, hypothenar shape, hypothenar color. Nail bed color: nail color, nail texture.

[0059] Specifically including the following steps:

[0060] Step 2.21, YCrCb Color Space Conversion: Use the cv2.cvtColor function of the OpenCV library to convert the input RGB format image to YCrCb color space to separate the brightness information (Y channel) and chrominance information (Cr, Cb channel). YCrCb color space can effectively reduce the influence of light changes on image segmentation, and enhance the distinction between hand region and background. The specific implementation method is:

[0061] Step 2.211, Read the input image (visible light or infrared image, format is RGB);

[0062] Step 2.212, Convert the RGB image to YCrCb image by cv2.cvtColor(image, cv2.COLOR_RGB2YCrCb), where Y channel represents luminance, and Cr and Cb channels represent red and blue chrominance components respectively;

[0063] Step 2.213, Output the YCrCb format image for subsequent processing.

[0064] Step 2.22, Otsu threshold segmentation: In the Cr channel of the YCrCb image, apply the Otsu threshold segmentation algorithm to automatically determine the optimal threshold, realizing the binary segmentation of the hand region. The Cr channel is sensitive to the chrominance information of the hand skin, which can effectively distinguish the hand from the background. As shown in Figure 3 The specific steps include:

[0065] Step 2.221, Extract the Cr channel data of the YCrCb image to generate a grayscale histogram;

[0066] Step 2.222, Use the cv2.threshold function of OpenCV (parameter cv2.THRESH_OTSU) to calculate the threshold T that maximizes the inter-class variance;

[0067] Step 2.223, According to the threshold T, binarize the Cr channel image into foreground (hand region, value is 255) and background (value is 0) to generate a binary mask;

[0068] Step 2.224, Apply the binary mask to extract the hand region, filtering out irrelevant background (such as objects in the shooting environment) and noise.

[0069] Step 2.23, Clahe algorithm vein enhancement: As shown in Figure 4 , use the Clahe (Contrast Limited Adaptive Histogram Equalization) algorithm to enhance the hand vein texture and improve the visibility of vein distribution, providing clearer details for subsequent feature extraction, as shown in Figure 5 The specific steps include:

[0070] Step 2.231, Use the cv2.cvtColor function of OpenCV to convert the BGR format image to a grayscale image;

[0071] Step 2.232, Create a Clahe object with contrast limit parameter set to 2.0 and grid size set to 8x8; i.e., use cv2.createCLAHE(clipLimit=2.0, tileGridSize=(8, 8)) to create the Clahe object; the clipLimit parameter controls the contrast limit, and the tileGridSize parameter defines the size of the grid.

[0072] Step 2.233, Apply the Clahe algorithm to the grayscale image to enhance the contrast of the vein texture and highlight the features of the hand veins.

[0073] Step 2.3, Data augmentation: Randomly rotate (±10°), translate (±10 pixels), and horizontally flip the training set images to increase data diversity, simulate different shooting angles and poses, improve the model's generalization ability for unknown data, and reduce the risk of overfitting.

[0074] Step 2.4, Normalization: Normalize the image pixel values to the [0, 1] interval (through image / 255.0), and apply mean and standard deviation normalization (mean [0.485, 0.456, 0.406], standard deviation [0.229, 0.224, 0.225]) to match the pre-trained input distribution of the ResNet50 model on the ImageNet dataset, ensuring stable model performance.

[0075] Step 2.5, Data labeling: According to the clinical diagnosis results of the detected subjects, label each image with a class label, 0 represents no coronary heart disease tendency, and 1 represents coronary heart disease tendency. Label data is converted to One-Hot encoding form through the to_categorical function for subsequent model training.

[0076] The processing of the above two algorithms is to more clearly observe the following features:

[0077] Blue vein distribution features: thumb blue veins, thenar blue veins, middle finger blood vessels.

[0078] Thenar region features: thenar shape, thenar color, thenar wrinkles, hypothenar shape, hypothenar color.

[0079] Nail features: nail color, nail texture.

[0080] The blue veins on the thumb are usually related to the function of the spleen and stomach. In traditional Chinese medicine theory, the spleen and stomach are the "root of acquired constitution" and are responsible for digestion and absorption. If there are blue veins on the thumb, it may mean that the function of the spleen and stomach is not good, such as indigestion, loss of appetite, abdominal distension, etc. The thumb blue veins mainly refer to the venous blood vessels from the root of the thumb to the fingertips. Usually, it is easier to observe on the inner side of the thumb (close to the palm side)

[0081] The large muscle at the base of the thumb (the large muscle at the base of the thumb) is related to the lungs and respiratory system. Traditional Chinese medicine believes that the lungs control the skin and hair, and the large muscle at the base of the thumb may reflect the functional status of the respiratory system. The large muscle at the base of the thumb is the muscle at the base of the thumb, located between the thumb and the palm

[0082] The shape of the large muscle at the base of the thumb can reflect the function of the lungs and respiratory system. If the large muscle at the base of the thumb is full, it usually indicates good lung function; if the large muscle at the base of the thumb is atrophic or sunken, it may indicate poor lung function. The color of the large muscle at the base of the thumb is related to the respiratory system and blood circulation. Under normal circumstances, the color of the large muscle at the base of the thumb should be red. If the color is white, it may indicate insufficient blood; if the color is red, it may indicate inflammation or heat; if the color is purple, it may indicate blood stasis.

[0083] The color of the large muscle at the base of the thumb is related to the respiratory system and blood circulation. Under normal circumstances, the color of the large muscle at the base of the thumb should be red. If the color is white, it may indicate insufficient blood; if the color is red, it may indicate inflammation or heat; if the color is purple, it may indicate blood stasis.

[0084] The wrinkles of the large muscle at the base of the thumb are related to the functional status of the lungs. Too many wrinkles may indicate poor lung function, such as chronic cough, asthma, etc.

[0085] The shape of the small muscle at the base of the little finger (the muscle at the base of the little finger) is related to the function of the kidneys. If the small muscle at the base of the little finger is full, it usually indicates good kidney function; if the small muscle at the base of the little finger is atrophic or sunken, it may indicate poor kidney function. The small muscle at the base of the little finger is the muscle at the base of the little finger, located between the little finger and the palm.

[0086] The color of the small muscle at the base of the little finger is related to the function of the kidneys and the reproductive system. Under normal circumstances, the color of the small muscle at the base of the little finger should be red. If the color is white, it may indicate kidney yang deficiency; if the color is black, it may indicate kidney yin deficiency or impaired kidney function.

[0087] The blood vessels of the middle finger are usually related to the cardiovascular and cerebrovascular system. The blood vessels of the middle finger are clearly visible and may be related to blood circulation. If the color of the blood vessels is abnormal (such as purple, black), it may indicate poor blood circulation or cardiovascular problems. The blood vessels of the middle finger mainly refer to the venous blood vessels on the front of the middle finger, which can usually be seen on both sides or the front of the middle finger.

[0088] The color of the nails can reflect the state of blood. Under normal circumstances, the nails should be red and shiny. If the color of the nails is pale, it may indicate insufficient blood; if the color of the nails is blue, it may indicate cold syndrome or blood stasis; if the color of the nails is purple, it may indicate poor blood circulation or hypoxia.

[0089] The texture of the nails can reflect the health of the body. Normally, the surface of the nails should be smooth and have no obvious texture. If there are vertical lines on the nails, it may indicate that the body has chronic diseases or is overworked; if there are white spots on the nails, it may indicate calcium deficiency or malnutrition; if there are horizontal lines on the nails, it may indicate that the body has a serious illness or has suffered a serious injury.

[0090] Step 3, construction of the deep learning model:

[0091] As shown in Figure 6 , a deep learning model based on ResNet50 is constructed for hand image feature extraction and classification. In this step, the transfer learning method is used to utilize the pre-trained model of ResNet50, and adjustments are made according to the requirements of the coronary heart disease tendency identification task. The reason for choosing ResNet50 is that its deep network structure (50 layers) and residual connection design can effectively extract complex features, while alleviating the gradient vanishing problem in deep network training, which is suitable for hand image feature extraction tasks.

[0092] Specifically, ResNet50 is an image classification algorithm based on deep convolutional neural network (CNN). It was proposed by Kaiming He et al. of Microsoft Research in 2015 and is an important member of the ResNet series. ResNet50 has a deeper network structure than traditional CNN models, and through the introduction of residual connection, it solves the gradient vanishing problem in the training process of deep network, effectively improving the performance of the model. The specific implementation process is as Figure 6 shown.

[0093] ResNet50 is divided into 5 stages (stages), among which the structure of Stage 0 is relatively simple and can be regarded as a preprocessing of the input (input). The last 4 stages are composed of Bottleneck and have similar structures. Stage 1 contains 3 Bottleneck, and the remaining 3 stages include 4, 6, and 3 Bottleneck respectively. Now Stage 0 and Stage 1 are described in detail, and the last 3 stages can be understood in the same way.

[0094] In Stage 0, (3, 224, 224) refers to the channel number (channel), height (height), and width (width) of the input, i.e. (C, H, W). Now assume that the height and width of the input are equal, so use (C, W, W) to represent.

[0095] The first layer in this stage includes three operations in sequence: ① CONV is the abbreviation of Convolution, 7x7 means the size of the convolution kernel, 64 means the number of the convolution kernel (i.e., the number of channels of the output of this convolution layer), and / 2 means the step of the convolution kernel is 2. ② BN is the abbreviation of Batch Normalization, i.e., the commonly said BN layer. ③ RELU means the ReLU activation function. The second layer in this stage is MaxPooling, i.e., the maximum pooling layer, with a kernel size of 3x3 and a step of 2. (64, 56, 56) is the number of channels (channel), height (height) and width (width) of the output of this stage, wherein 64 is equal to the number of convolution kernels in the first layer of the convolution layer in this stage, and 56 is equal to 224 / 2 / 2 (the step of 2 will make the input size halved).

[0096] In summary, in Stage 0, the input with a shape of (3, 224, 224) sequentially passes through a convolution layer, a BN layer, a ReLU activation function, and a MaxPooling layer to obtain an output with a shape of (64, 56, 56).

[0097] After understanding Stage 0 and being familiar with the meanings of various symbols in the figure, Stage 1 can be easily understood. The specific structure of the Bottleneck introduced in Stage 1 is shown on the right side. Figure 6

[0098] The two Bottlenecks correspond to two cases: the same number of input and output channels, which selects the BTNK2 method ("BTNK" is the abbreviation of BottleNeck), and different numbers of input and output channels, which selects the BTNK1 method.

[0099] BTNK2 has two variable parameters C and W, i.e., c and W in the shape (C, W, W) of the input. Let the input with a shape of (C, W, W) be X, and let the three convolution blocks (and related BN and ReLU) on the left side of BTNK2 be a function F(X). After adding (F(X)+X) and then passing through a ReLU activation function, the output of BTNK2 is obtained, and the shape of the output is still (C, W, W), i.e., the case of the same number of input and output channels corresponding to BTNK2.

[0100] BTNK1 has four variable parameters C, W, C1 and S. Compared with BTNK2, BTNK1 has an additional convolution layer on the right side, which is a function G(X). BTNK1 corresponds to the case of different numbers of input X and output F(X) channels, and it is this added convolution layer that changes X into G(X), which matches the difference in the dimensions of the input and output (G(X) and F(X) have the same number of channels), so that the sum G(X)+F(X) can be performed.​

[0101] After the output of the last convolutional layer of the last stage STAGE4 of ResNet50, a global average pooling layer, a dropout layer, a fully connected layer, a softmax layer, a categorical crossentropy layer, and an output layer are added.

[0102] The specific operations for improving the original ResNet50 model are as follows:

[0103] Step 3.1, load the pre-trained ResNet50 model: use ResNet50 as the base model, load the pre-trained weights on the ImageNet dataset, and use the general visual features learned on large-scale image data (such as edge contours, color textures, region shapes, and structural combinations) to improve feature extraction efficiency and generalization ability.

[0104] Set include_top = False to remove the top fully connected layer of ResNet50 and only keep the convolutional base part as a feature extractor. Because the ImageNet dataset has 1000 categories, the model weight trained on ImageNet is migrated in this embodiment, and the parameters of the last fully connected layer are suitable for 1000-class classification. However, the purpose of this embodiment is binary classification, so the fully connected layer is removed and replaced with a classification layer with 2 categories. The input image size of ResNet50 is 224x224x3, which is suitable for the preprocessed hand image.

[0105] The convolutional base of ResNet50 in each STAGE contains multiple residual blocks (such as conv2_block1 to conv5_block3), which extract features layer by layer through convolutional layers, batch normalization, and ReLU activation functions. For example, the output shape of the first convolutional layer (conv1_conv) is (None, 112, 112, 64), and the parameter amount is 9472; the output shape of the last convolutional layer (conv5_block3_out) is (None, 7, 7, 2048), which provides high-level features for subsequent classification.

[0106] Step 3.2, add a custom classification layer: after the output of the ResNet50 convolutional base, a global average pooling layer (GlobalAveragePooling2D) is added to compress the feature map from (None, 7, 7, 2048) to a 1-dimensional vector with a length of 2048, reducing the dimension and reducing the risk of overfitting.

[0107] Add a Dropout layer with a dropout rate of 0.3 to randomly drop 30% of the neurons, enhancing the model's generalization ability on hand image data.

[0108] Add a Dense (fully connected) layer with an output dimension of 2, corresponding to the two classes "no coronary heart disease tendency" and "coronary heart disease tendency", and use a softmax activation function to output probability distribution. The parameter quantity of the final layer is 12294.

[0109] Step 3.3, Model Fine-tuning Settings: Set the ResNet50 convolution base to be trainable (trainable=True), allowing all layers to be fine-tuned during training to adapt to the feature distribution of hand images (such as nail bed color, palm texture). During the fine-tuning process, the residual connection (Add layer) of ResNet50 ensures effective gradient transmission, alleviating the gradient vanishing problem in deep network training.

[0110] Step 3.4 Model Compilation: Set the loss function to the multi-class cross-entropy loss function categorical_crossentropy, suitable for multi-class classification tasks, the optimizer to Adam with an initial learning rate of 0.001, and the evaluation metric to accuracy.

[0111] The total parameter quantity of the model is 23,600,006, of which the trainable parameter quantity is 23,546,886 and the non-trainable parameter quantity is 53,120.

[0112] Step 4, Training of Deep Learning Model:

[0113] Train the model using the preprocessed hand image dataset and optimize the model parameters, with the following specific operations:

[0114] Step 4.1, Dataset Division: Divide the training data into training set and validation set according to the proportion of 80% and 20%, and keep the class balance of training set and validation set.

[0115] Step 4.2, Training Parameter Settings: Batch size is 32, and training rounds are 40 epochs. Use the following callback functions to optimize the training process:

[0116] ReduceLROnPlateau: Monitor the validation set accuracy (val_accuracy), patience value is 5, and learning rate reduction factor is 0.2;

[0117] EarlyStopping: Monitor the validation set accuracy, patience value is 20, if the accuracy does not improve for 20 consecutive epochs, stop training and save the best model.

[0118] Step 4.3, training process: input the training set and validation set into the model, calculate the loss and update the parameters. During the training process, the training loss decreased from 0.8136 to 0.0076, and the validation loss decreased from 3780.39 to 0.5293; the training accuracy improved from 0.2031 to 1.0, and the validation accuracy improved from 0.25 to 0.5625. The final model achieved an accuracy of 85.5% and an AUC value of 0.89 on the test set. As shown in FIGS. 1-3. Figure 7 、 Figure 8

[0119] Step 5, hand image feature extraction and classification:

[0120] The preprocessed hand image of the detected person is input into the trained ResNet50 model for feature extraction and classification.

[0121] Step 5.1, feature extraction: the image passes through the convolutional layers of ResNet50 to extract low-level features (such as edges, textures) and high-level features (such as nail bed color distribution, palm texture pattern). The global average pooling layer compresses the features into a 1-dimensional vector.

[0122] Step 5.2, classification prediction: the feature vector passes through the fully connected layer to output the probability distribution of two categories (no coronary heart disease tendency / coronary heart disease tendency). If the probability value is greater than 0.5, it is judged as having coronary heart disease tendency, otherwise as having no coronary heart disease tendency.

[0123] Step 6, output of coronary heart disease tendency prediction results:

[0124] The model prediction results are output in the form of probability value and category judgment, which is implemented as follows:

[0125] Step 6.1, result generation: through the software interface of the hand diagnostic instrument, the prediction results are displayed as "coronary heart disease tendency probability: X.XX" (such as "coronary heart disease tendency probability: 0.72") and "prediction result: coronary heart disease tendency / no coronary heart disease tendency" (such as "prediction result: coronary heart disease tendency").

[0126] Step 6.2, output method: the results are presented in real time through the display screen of the hand diagnostic instrument, and the time consumption of a single detection is about 2 seconds. The hand diagnostic instrument device integrates a high-definition camera (resolution 1920x1080) and an embedded computing module, supporting full-process automation operation from image acquisition to result output.

[0127] Example 2

[0128] A coronary heart disease tendency recognition system based on hand image features, the system is used to perform the recognition method described in example 1, comprising:

[0129] ​An image acquisition module: used for collecting the front and back image data of the hands of the detected person, the collected objects including healthy people and diagnosed coronary heart disease patients, and one image of each hand is collected for each person;

[0130] An image preprocessing module: used for adjusting, denoising, feature enhancing and normalizing the collected image data;

[0131] A deep learning model module: used for extracting and classifying the hand image features through the ResNet50 model;

[0132] A result output module: used for outputting the prediction result through the display screen and supporting real-time feedback.

[0133] Embodiment 3

[0134] A hand diagnosis instrument device, as shown in Figure 9 , comprises the system described in Embodiment 2, and further comprises a high-definition camera with a resolution of 1920x1080.

[0135] Embodiment 4

[0136] A coronary heart disease tendency recognition device based on hand image features, the device comprising: a memory and a processor, the memory storing a computer program, and the processor implementing the steps of the method described in Embodiment 1 when executing the computer program.

[0137] Embodiment 5

[0138] A computer-readable storage medium having a computer program stored thereon, the computer program being executed by a processor to implement the steps of the method described in Embodiment 1.

[0139] The above embodiments are only used to illustrate the present application and not to limit the technical solutions described in the present application. Although the present application has been described in detail with reference to the above embodiments, the present application is not limited to the above specific embodiments, and any modification or equivalent replacement of the present application; all technical solutions and improvements without departing from the spirit and scope of the present application are covered in the scope of claims of the present application.

Claims

1. A coronary heart disease predisposition identification device based on hand image features, the device comprising: The memory stores a computer program, and the processor implements the following steps when executing the computer program: The palm texture features and the nail bed color texture features are generated using the front and back hand image data of the left and right hands of the detected person, and the palm texture features and the nail bed color texture features are input into the trained ResNet50 model to identify the coronary heart disease tendency and the coronary heart disease tendency probability of the detected person; the palm texture includes: the blue vein distribution features including the thumb blue vein, the thenar blue vein, and the middle finger blood vessel; the thenar region features include the thenar shape, the thenar color, and the thenar wrinkle; the nail bed color includes the nail color and the nail texture; Before the ResNet50 model is trained, the front and back hand image data input into the ResNet50 model is enhanced in the YCrCb color space combined with the Otsu segmentation technology and the Clahe algorithm to enhance the hand features, and according to the clinical diagnosis result of the detected person, the class label is labeled as no coronary heart disease tendency and coronary heart disease tendency; The ResNet50 model includes five stages connected in sequence, the last convolution layer of the last stage is connected in sequence with a global average pooling layer, a Dropout layer, a full connection layer, a Softmax layer, a cross-entropy loss function layer, and an output layer, the top full connection layer of the ResNet50 model is removed, the output dimension of the full connection layer is 2, corresponding to "no coronary heart disease tendency" and "coronary heart disease tendency" two categories respectively, the probability distribution is output by the softmax activation function, the cross-entropy loss function layer outputs the difference between the probability distribution predicted by the ResNet50 model and the probability distribution of the real label, and is used to update the parameters of the ResNet50 model through back propagation; The global average pooling layer is used to compress the feature map output by the last convolution layer of the last stage into a 1-dimensional vector, reduce the dimension, and reduce the risk of overfitting; The ResNet50 model sets the ResNet50 convolution base as trainable, allows fine-tuning of all layers during the training process to adapt to the feature distribution of the hand image, and the residual connection Add layer of the ResNet50 ensures effective gradient transmission during the fine-tuning process; The hand features include the palm texture, and the enhanced hand features include: YCrCb color space conversion: using the cv2.cvtColor function of the OpenCV library, the input image in RGB format is converted into YCrCb color space to separate the brightness information and the chrominance information; Otsu threshold segmentation: in the Cr channel of the YCrCb image, the Otsu threshold segmentation algorithm is applied to automatically determine the best threshold to realize the binary segmentation of the hand region; Clahe algorithm vein enhancement: the Clahe algorithm is used to enhance the palm texture and improve the visibility of the vein distribution. 2.The hand image feature-based coronary heart disease tendency identification device according to claim 1, characterized in that, The Dropout layer sets the dropout rate to 0.3, and randomly discards 30% of the neurons to enhance the generalization ability of the model on the hand image data. 3.The hand image feature based coronary heart disease tendency identification device according to claim 1, characterized in that, The optimizer of the cross-entropy loss function layer is Adam, the initial learning rate is 0.001, and the evaluation index is accuracy. 4.The hand image feature-based coronary heart disease tendency identification device according to claim 1, characterized in that, The left and right hand front and back image data are collected by a high-definition camera, the shooting conditions are natural light, the shooting distance is 20 cm, the image storage format is JPEG, and after the collection is completed, the image resolution is uniformly adjusted to 224*224 pixels. 5.The hand image feature based coronary heart disease tendency identification device according to claim 1, characterized in that, The hand front and back image data input into the ResNet50 model is enhanced by using the YCrCb color space combined with the Otsu segmentation technology and the Clahe algorithm to enhance the hand features, and according to the clinical diagnosis result of the detected person, the class label is labeled as no coronary heart disease tendency and coronary heart disease tendency including: Step 2.1, image adjustment and denoising: convert the image to RGB format and use the Gaussian blur method to remove image noise; Step 2.2, feature enhancement: through the visible light and infrared image segmentation technology based on YCrCb color space and Otsu algorithm, and Clahe algorithm for vein image enhancement processing, the contrast of hand features including palm texture is enhanced; Step 2.3, data enhancement: randomly rotate the training set images by ±10°, translate by ±10 pixels, and horizontally flip them to increase data diversity; Step 2.4, normalization processing: normalize the image pixel value to the [0, 1] interval, and apply mean and standard deviation normalization to match the pre-trained input distribution of the ResNet50 model on the ImageNet dataset; Step 2.5, data labeling: according to the clinical diagnosis result of the detected person, label the class label for each image, 0 represents no coronary heart disease tendency, and 1 represents coronary heart disease tendency. 6.The device for recognizing the tendency of coronary heart disease based on hand image features according to claim 5, characterized in that, The Otsu threshold segmentation specifically includes: Step 2.221, extract the Cr channel data of the YCrCb image to generate a gray histogram; Step 2.222, use the cv2.threshold function of OpenCV to calculate the threshold T that maximizes the inter-class variance; Step 2.223, according to the threshold T, binarize the Cr channel image into foreground and background to generate a binary mask; the foreground includes the hand region and the value is 255; the background value is 0; Step 2.224, apply the binary mask to extract the hand region and filter out irrelevant background and noise. 7.The hand image feature based coronary heart disease tendency identification device according to claim 6, characterized in that, The Clahe algorithm vein enhancement specifically includes: Step 2.231, use the cv2.cvtColor function of OpenCV to convert the BGR format image to a grayscale image; Step 2.232, create a Clahe object and set the contrast limit parameter to 2.0 and the grid size to 8x8; use cv2.createCLAHE(clipLimit=2.0, tileGridSize=(8,8)); Step 2.233, apply the Clahe algorithm to the grayscale image to enhance the contrast of the vein texture and highlight the hand vein features.

Images