Image classification and recognition method for pulse wave signals

Abstract

The embodiment of the invention discloses a pulse wave signal imaging classification identification method, which comprises the following steps: collecting and preprocessing a pulse wave signal, and converting the pulse wave signal into a two-dimensional image by utilizing Markov conversion; an enhanced MTF-ImDenseNet169 network model is used to extract the image features of the two-dimensional image; the Grad-CAM is used for visualizing the image features, and then the pulse wave signals are subjected to image classification and recognition according to the visualized indication area. According to the pulse wave signal imagination classification and recognition method, single-part, two-part and multi-part pulse wave signals are coded into two-dimensional images based on a Markov migration field method, image fusion and enhancement are conducted on the two-dimensional images through the Mixup technology, a multi-scale parallel convolution layer, a feature channel attention block and an activation function Swsh function are introduced into an MTF-ImDenseNet169 network model, and therefore image fusion and recognition are conducted on the two-dimensional images. A class activation mapping visualization technology is adopted to visualize a discrimination area playing a decisive role in the pulse wave signals, and pulse wave signal imaging classification and recognition are achieved.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of data processing, and particularly relates to a method for image classification and recognition of pulse wave signals. BACKGROUND

[0002] The pulse wave signal contains rich physiological and pathological information of the human body and can be collected in a non-invasive manner. Due to the periodicity of the pulse wave, a large amount of effective information is presented through the waveform change within a single cycle. When collecting the pulse wave signal, a pressure-sensitive element is usually used for collection. However, due to the relatively weak pulse wave signal, it is extremely susceptible to external interference, which can cause the collected pulse wave signal to be distorted or to have a large abnormality in the later processing process, and cannot objectively reflect the true information.

[0004] In summary, the prior art has the technical problems of difficulty in mining different pulse signal characteristics, complexity in extraction, and low recognition rate. SUMMARY

[0005] Therefore, some embodiments disclose a method for image classification and recognition of pulse wave signals, comprising:

[0006] Collecting and preprocessing the pulse wave signal, and converting the pulse wave signal into a two-dimensional image using Markov conversion;

[0007] Extracting image features of the two-dimensional image using an enhanced MTF-ImDenseNet169 network model;

[0008] Visualizing the image features using Grad-CAM, and then classifying and recognizing the pulse wave signal according to the visualized indication area.

[0009] Further, some embodiments disclose a method for image classification and recognition of pulse wave signals, wherein the preprocessing of the pulse wave signal comprises:

[0010] Collecting and preprocessing the pulse wave signals of the three parts of the Cun, Guan and Chi;

[0011] Converting the single-part, double-part and three-part pulse wave signals into corresponding MTF pulse graphs using MTF;

[0012] Performing image fusion on the obtained MTF pulse graphs using Mixup technology.

[0013] Some embodiments disclose a method for image classification and recognition of pulse wave signals, wherein converting the pulse signal into a two-dimensional image using Markov conversion comprises:

[0014] The pulse wave signal is represented as a time series X=(x1, x2, …x i ), i=1, 2…n.

[0015] According to the value range, the time series is divided into Q bins, and each x i is mapped to a corresponding Q j (j = 1, 2, …, i).

[0016] Along each time step in a first-order Markov chain, q j The transition between them is used to construct a Markov transition matrix M, which is defined as follows:

[0017]

[0018] Where w ij represents the transition probability from q i to q j , and after normalization, it satisfies the condition

[0019] Arranging each transition probability in time sequence, the Markov transition matrix M is expanded into an n x n matrix M; the n x n matrix M is defined as follows:

[0020]

[0021] Where M ij (i, j ∈ 1, 2, … n) is the transition probability from the corresponding region of signal x i to the corresponding region of signal y i ; the value range of element M ij in matrix M is [0, 1];

[0022] The value of each element in matrix M is scaled to 0-255 using the following formula, so that the elements in matrix M correspond to the pixel values in the image, and a two-dimensional image is obtained.

[0023] I(i, j) = int(255M ij )

[0024] Where I(i, j) is the pixel value of the i-th row and j-th column of the image, and int(*) is the rounding function.

[0025] Some embodiments of the image classification and recognition method of the pulse wave signal are disclosed. The enhanced MTF-ImDenseNet169 model contains 171 learnable layers, of which 166 layers are located between the Dense Block and the first convolutional layer, and there is one FC layer and three transition layers, as listed in the following table:

[0026]

[0027]

[0028] The image classification method of the pulse wave signal disclosed by some embodiments includes the functions of the channel attention mechanism SE block, which includes:

[0029] For a given mapping input X (X∈R H'×W'×C' ) to a feature mapping U (U∈R H×W×C ), a corresponding SE block is constructed to perform feature recalibration; wherein the feature U first undergoes a compression operation Fsq, which compresses U into a 1×1×C feature, then excites the feature from Fsq through an excitation operation Fex, and finally obtains a recalibrated feature through Fscale

[0030] The image classification method of the pulse wave signal disclosed by some embodiments includes the functions of the channel attention mechanism SE block, which includes:

[0031] The two-dimensional pulse graph is first passed through three branch convolution kernels of 1×1, 3×3 and 5×5, then input into the ImDenseNet network through a 3×3 max pooling, and then the two-dimensional pulse wave signal image information is extracted through four Dense blocks, three SE blocks and three Transition layers.

[0032] The image classification method of the pulse wave signal disclosed by some embodiments includes the functions of the channel attention mechanism SE block, which includes:

[0033]

[0034] The image classification method of the pulse wave signal disclosed by some embodiments includes the functions of the channel attention mechanism SE block, which includes:

[0035] The gradient of the given class probability under the convolution layer feature map is calculated, and the weight calculation process of the kth feature map corresponding to class c in the convolutional neural network is as follows:

[0036]

[0037] Where y c represents the probability of the model's judgment of the class, represents the pixel value of the kth feature map at the (i, j) coordinate;

[0038] According to the obtained gradient, the contribution rate matrix is calculated by combining the weight, and the formula is as follows:

[0039]

[0040] Where, The weight coefficients of the corresponding feature maps are represented.

[0041] The pulse wave signal imaging classification method disclosed by the embodiment of the application encodes single, two and multi-pulse wave signals into two-dimensional images based on the Markov transition field method, and uses the Mixup technology to perform image fusion enhancement on the two-dimensional images, thereby improving the feature complementarity and data quality. Then, an improved MTF-ImDenseNet169 network model is proposed, and a multi-scale parallel convolution layer and a feature channel attention block are introduced, which respectively realize the capture of pulse wave signal features of different scales and the highlighting of information of different pulse waves in the feature map. In addition, a smoother activation function Swish function is introduced in the model, which improves the nonlinear ability of the network and makes the training process of the network more convergent and efficient. Finally, the class activation mapping visualization technology is used to visualize the discriminative region that plays a decisive role in the pulse wave signal, realizing the imaging classification of the pulse wave signal. The method establishes trust in the pulse wave signal recognition model and provides a beneficial classification method for the recognition of different pulses, which has a potential good application prospect. BRIEF DESCRIPTION OF DRAWINGS

[0042] Figure 1 , the flow chart of the pulse wave signal imaging classification method;

[0043] Figure 2 , the schematic diagram of the Markov transition process of the pulse wave signal;

[0044] Figure 3 , the schematic diagram of the feature fusion process of different parts of the pulse wave signal;

[0045] Figure 4 , the schematic diagram of the DenseNet neural network model structure;

[0046] Figure 5 , the schematic diagram of the dense block structure;

[0047] Figure 6 , the schematic diagram of the ImDenseNet169 neural network model structure;

[0048] Figure 7 , the schematic diagram of the SE model structure;

[0049] Figure 8 , the schematic diagram of the MTF-ImDenseNet169 model pulse wave recognition process;

[0050] Figure 9 , the training and test curves of the MTF-ImDenseNet169 model; (a) is the accuracy curve, and (b) is the loss curve;

[0051] Figure 10, Gradient-weighted class activation mapping flowchart;

[0052] Figure 11 , Pulse wave signal visualization of Grad-CAM;

[0053] Figure 12 , Typical pulse wave signal waveform diagram. DETAILED DESCRIPTION

[0054] The term "example" as used herein should not be construed as a preference or advantage of one embodiment over another embodiment. The performance indicators of the embodiments of the present application are tested by using conventional test methods in the art, unless otherwise specified. It should be understood that the terms described in the embodiments of the present application are only for the description of the specific embodiments, and are not used to limit the disclosure of the embodiments of the present application.

[0055] Unless otherwise specified, the technical and scientific terms used herein have the same meaning as generally understood by those skilled in the art to which the embodiments of the present application belong; the test methods and technical means not specifically noted in the embodiments of the present application refer to the experimental methods and technical means generally used by those skilled in the art.

[0056] The terms "substantially" and "approximately" as used herein are used to describe a small fluctuation. For example, they can mean less than or equal to ± 5%, such as less than or equal to ± 2%, such as less than or equal to ± 1%, such as less than or equal to ± 0.5%, such as less than or equal to ± 0.2%, such as less than or equal to ± 0.1%, such as less than or equal to ± 0.05%. Numerical data expressed or presented in the form of a range herein is used for convenience and brevity only, and should be construed as flexible to include not only the numerical values explicitly recited as the limits of the range, but also all individual numerical values or sub-ranges encompassed within the range. For example, a numerical range of "1-5%" should be interpreted to include not only the explicitly recited values of 1% and 5%, but also individual values and sub-ranges within the indicated range. Thus, included in this numerical range are individual values such as 2%, 3.5%, and 4%, and sub-ranges such as 1-3%, 2-4%, and 3-5%, etc. This same principle applies to ranges reciting only one numerical value. In addition, such interpretation applies regardless of the width of the range or the characteristics being described.

[0057] In this text, including the claims, conjunctions such as "comprising", "including", "with", "having", "containing", "relating to", "accommodating" and the like are understood to be open, i.e. to mean "including but not limited to". Only the conjunctions "consisting of" and "consisting essentially of" are closed conjunctions.

[0058] For a better understanding of the present application, numerous specific details are given in the following detailed description. One skilled in the art will understand, however, that the application can be practiced without certain specific details. In an embodiment, some methods, means, instruments, devices, etc. that are well known in the art are not described in detail in order to avoid obscuring the subject matter of the present application.

[0059] The technical features disclosed in the embodiments of the present application can be combined arbitrarily without conflict, and the resulting technical solutions belong to the disclosure of the present application.

[0060] In some embodiments, as shown in FIG. 1, the pulse wave signal image classification and recognition method includes: Figure 1

[0061] Collect and preprocess the pulse wave signal, and convert the pulse wave signal into a two-dimensional image using Markov conversion;

[0062] Extract image features of the two-dimensional image using an enhanced DenseNet169 network model;

[0063] Visualize the image features using Grad-CAM, and then classify and recognize the pulse wave signal according to the visualized indication area.

[0064] The following further illustrates the technical details with examples.

[0065] Construction of the pulse wave signal image classification and recognition model

[0066] Markov transition field

[0067] As a quasi-periodic signal, the pulse wave signal exhibits obvious time sequence structure, with rich dynamic change characteristics and spatio-temporal correlation. The Markov transition field (MTF) can effectively capture the correlation and dynamic changes in the time sequence, fully consider the time sequence changes and spatial structure between signals, and thus more effectively analyze and utilize the information of the pulse wave signal.

[0068] For the pulse wave signal time sequence X=(x1, x2, …x i ), i=1, 2…n, the time sequence is divided into Q bins according to the value range, so that each x i is mapped to the corresponding Q j (j=1, 2, …, i).

[0069] The Markov transition matrix M is calculated by calculating q j ​The transition matrix M is constructed by the transition between the two states. The Markov transition matrix M is defined by the following equation (1):

[0070]

[0071] In equation (1), w ij represents the transition probability from q i to q j , and after normalization, it satisfies the condition

[0072] Finally, the Markov transition matrix is expanded to an n x n matrix M by arranging each probability in time sequence. The definition of the n x n matrix M is as follows in equation (2):

[0073]

[0074] In equation (2), M ij (i, j ∈ 1, 2, … n) is the transition probability from the corresponding region of signal x i to the corresponding region of signal y i ; the value range of the element in matrix M is [0, 1].

[0075] In order to make the element in matrix M correspond to the pixel value in the image, the value of each element in matrix M is scaled to 0-255 using the following equation (3) to obtain a two-dimensional image.

[0076] I(i, j) = int(255M ij ) (3)

[0077] In equation (3), I(i, j) is the pixel value of the i-th row and j-th column of the image, and int(*) is the rounding function. The MTF pulse migration process of converting the pulse wave signal into the Markov matrix and then into the Markov transition field is shown in Figure 2 . Figure 2 In the figure, red represents a larger pixel value, and blue represents a smaller pixel value.

[0078] Fusion of input pulse wave signals

[0079] The pulse wave signal usually refers to the pulse signals of the three parts of the wrist, usually, only the pulse wave signal containing any one of the three parts of the wrist is a single-part pulse wave signal, the pulse wave signal containing any two of the three parts of the wrist is called a double-part pulse wave signal, and the pulse wave signal containing all three parts of the wrist is called a three-part pulse signal;

[0080] The inventors found that there are both differences and complementarities between the features of single-pulse wave signals, double-pulse wave signals and triple-pulse wave signals, and there are also autocorrelation and cross-correlation relationships between pulse waves of different parts. Artificially extracting and simplifying these features not only takes time and effort, but also may cause information loss and affect the accuracy of the analysis results.

[0081] Based on the above findings, in order to better utilize the features of pulse wave signals, the present application proposes a fusion method for pulse wave features of different parts. As shown in Figure 3 , first, the pulse wave signals of the three parts of Cun, Guan and Chi are preprocessed respectively to ensure the quality and reliability of the signals; then, the MTF is used to convert the single-part, double-part and triple-part pulse wave signals into corresponding MTF pulse diagrams, which can better capture the dynamic characteristics and correlation of the signals. Finally, the Mixup technology is used to enhance the image fusion of the three pulse diagrams to more comprehensively retain the key features of the pulse wave signals and further enhance the feature expression ability of the pulse diagrams. Through this comprehensive feature enhancement method, the features of the pulse wave signals can be more accurately described, and the complementarity and reliability between the signals can be improved.

[0082] DenseNet convolutional network

[0083] The pulse wave signal features are distributed in different frequency bands, with high dimensionality and complex characteristics. Compared with traditional CNN, DenseNet is directly connected with all previous layers at each layer, so that the DenseNet network can fully utilize the features of pulse wave signals in each frequency band, further improving the reuse rate of signal features and information transmission efficiency. For time series data such as pulse wave signals, the dense connection structure helps to better retain and utilize the time series information, improving the accuracy of classification. In addition, pulse wave signals may be affected by noise interference or data loss, and the dense connection structure of DenseNet also helps to improve the robustness and generalization ability of the model, which can better adapt to various data situations during training, reducing the need for a large amount of labeled data, and achieving good results even in small sample scenarios.

[0084] The DenseNet network model draws on the advantages of ResNet network and GoogLeNet network, and from the perspective of optimal features, it proposes a dense connection mechanism, the structure of which is shown in Figure 4 . The DenseNet network model enhances the propagation of features by designing dense blocks with dense connections, reduces the number of network parameters, and alleviates the complexity of the network and the "gradient dispersion" problem caused by too deep network model, thereby improving the recognition rate of deep neural networks. Compared with other networks, DenseNet breaks the traditional idea of improving network performance by deepening network layers and widening network structure.

[0085] The DenseNet network consists of multiple DenseLayer transition layers strung together. Each standard DenseLayer performs standard convolutions of BN+ReLU+1×1Conv and BN+ReLU+3×3Conv, which is the basic part of the dense convolutional network, such as... Figure 5 As shown. Assuming the current layer is layer l, H... l To represent the nonlinear transformation between networks, use x. l If we represent the network input, then the network input of the current layer is given by the following formula (4):

[0086] x l =H l ([x0,x1,…,x l-1 (4)

[0087] In equation (4), [x0,x1,…,x l-1 ] represents the concatenation of feature maps from the output of layer 1 to layer l, H l It is a comprehensive transformation function.

[0088] The transition layer is mainly used to connect two adjacent dense blocks. Its main function is to reduce the dimensionality of the feature vectors extracted from the dense blocks, taking into account both channel and spatial dimensions. Each transition layer mainly consists of convolutional layers and pooling layers. The convolutional layer acts as a filter, compressing the input feature vector in the channel dimension, while the pooling layer reduces the dimensionality of the feature vector in each channel in the spatial dimension. In some embodiments, the transition layer includes a batch normalization layer, a 1×1 convolutional layer, and a 2×2 average pooling layer with a stride of 2.

[0089] The first layer of DenseNet is a 7×7 convolutional layer with a large receptive field. In this embodiment, three branch convolutional kernels of 1×1, 3×3, and 5×5 are used to replace the 7×7 convolutional layer to better extract detailed features of the pulse wave signal at different scales. On the one hand, the three branch convolutional kernels of different scales can extract various waveform features of the pulse wave image at multiple scales; on the other hand, the use of three small-scale branch convolutional kernels limits the number of training parameters in the center of the entire network, reduces the network complexity, and helps to speed up the convolution calculation. The structure of the improved DenseNet169 (Improved DenseNet169, ImDenseNet169) network model is as follows: Figure 6 As shown.

[0090] Squeeze and Excitation Model Structure

[0091] The attention mechanism can be regarded as a resource allocation mechanism, and an embodiment of the present application selects a channel attention mechanism, i.e., a Squeeze and Excitation (SE) block. The core idea is to learn feature weights according to loss, so that effective features have larger weights. The SE block mainly consists of a squeeze and an excitation. The squeeze means compressing features along the spatial dimension and converting each two-dimensional feature channel into a real number, which has a global receptive field to some extent. The excitation is a mechanism similar to the gate in a recurrent neural network (RNN), which generates weights for each feature channel through parameters, and learns these weights to explicitly model the relationship between feature channels.

[0092] The graphical structure of the SE block is shown in Figure 7 For a given mapping input X (X ∈ R H'×W'×C' ) to a feature map U (U ∈ R H×W×C ), a corresponding SE block can be constructed to perform feature recalibration. The feature U first undergoes a compression operation Fsq, which compresses U into a 1 × 1 × C feature. Then the feature from Fsq is excited through an excitation operation Fex. Finally, through Fscale, the recalibrated feature is obtained, where Fscale represents the previous feature channel weights obtained by weighting the excitation output one by one, and the recalibration of the original feature in the channel dimension is completed.

[0093] MTF-ImDenseNet169 model structure

[0094] The MTF-ImDenseNet169 model is applied to extract two-dimensional pulse wave image features. The MTF-ImDenseNet169 model contains 171 learnable layers, of which 166 layers are located in the Dense Block and the first convolutional layer, the last FC layer and three transition layers, as listed in Table 1. In some embodiments, the fully connected layer of three neurons is randomly initialized, a Softmax layer and a new classification layer with only three classes.

[0095] The function of the SE block is to selectively enhance effective features, and the SE block is added in each dense block, so that different pulse feature information can be effectively enhanced. Since the SE block can only be placed after each convolution non-linear operation, the SE block is placed after the batch normalization layer after the 3 × 3 convolution.

[0096] Table 1 MTF-ImDenseNet169 model structure

[0097]

[0098] The MTF-ImDenseNet169 pulse wave recognition process is as shown in Figure 8

[0099] Smooth variable activation function

[0100] Activation functions play a crucial role in neural network models, as they introduce non-linear properties that enable the neural network model to learn and represent more complex function relationships. By mapping the input, the activation function determines the activation level of the neuron, thereby affecting the learning ability and expression ability of the neural network model. Therefore, when designing a neural network model, selecting appropriate activation functions is a crucial step, which has important influence on improving the performance, stability and convergence speed of the network.

[0101] In some embodiments, the activation function used is the Swish function. The Swish function has a smoother curve, which can better maintain continuity when processing pulse wave signals, and can help better capture subtle changes in pulse wave signals. In addition, the Swish function has low saturation in the negative number interval, and the derivative is always > 0, which helps to train deeper models, thereby better capturing complex patterns and features in pulse wave signals. The Swish function expression is as follows (5):

[0102]

[0103] Experimental conclusions and analysis

[0104] Source of experimental data

[0105] In this embodiment, the pulse wave signal experimental data of 865 subjects is used, and three types of pulse conditions are selected after being judged by TCM experts, namely flat pulse, slippery pulse and string pulse. The three types of pulse wave signals are preprocessed to obtain the pulse wave signals required for subsequent analysis.

[0106] Experimental data processing

[0107] The preprocessed pulse wave signals are segmented and pulse diagram expanded with a length of 1500 sampling points, resulting in 31305 two-dimensional pulse diagrams, of which 11862 are flat pulse diagrams, 10875 are slippery pulse diagrams, and 8568 are string pulse diagrams. The pulse wave signals of the three groups of people are further encoded by MTF to generate a two-dimensional matrix of 224x224, thereby obtaining the probability feature image of the pulse wave signal offset. The pulse wave signal images are randomly divided into two groups, 80% for training and 20% for testing. The experiment uses 5-fold cross-validation, and the average value of the performance evaluation results of the test set obtained by all 5 cross-validations is taken as the final performance indicator of the model. The experimental scheme is implemented by Python 3.7 on a Linux server.

[0108] ​Specifically, the two-dimensional pulse pattern is first passed through 1x1, 3x3, 5x5 three branch convolution kernels, then passed through a 3x3 max pooling into the MTF-ImDenseNet network, and then the two-dimensional pulse wave signal image information is extracted through 4 Dense blocks, 3 SE blocks and 3 Transition layers. The SE selectively enhances the important features of the pulse wave signal, improving the expression ability of the features. The feature map output by the last dense connection block of the network model is respectively passed through the pooling layer and the fully connected layer, and the feature map is reduced to one dimension. In order to avoid model overfitting, the Dropout layer with a dropout rate of 0.2 is used to ignore some neurons with a certain probability.

[0109] The MTF-ImDenseNet169 model is trained using the classification cross-entropy loss function and the stochastic gradient descent optimizer with a momentum of 0.9 and a learning rate of 0.0001 to ensure that the model can effectively converge and learn features. Finally, the extracted pulse wave feature information is input to the fully connected layer for pulse wave classification and recognition.

[0110] Performance of MTF-ImDenseNet169 model

[0111] The neural network model MTF-ImDenseNet169 is evaluated using accuracy, precision, recall and F1 score performance indicators. The three types of pulse are classified using this neural network model, and the efficiency and performance of the model in feature extraction and classification tasks are studied. As shown in Figure 9 The average recognition accuracy and loss value of the training and test data set in cross-validation tend to be stable after 200 iterations. During the training process, the loss decreases and tends to be stable, indicating that the model has learned to extract key features from pulse wave images and perform accurate recognition.

[0112] The MTF-ImDenseNet169 model performs well on different pulse wave detection, with an average accuracy, precision, recall and F1 score of 97.93%, 98.33%, 97.62% and 98.01% respectively. The accuracy of the model in identifying flat pulse is 98.42%, and the accuracy of sliding pulse and chord pulse is 96.81% and 98.74% respectively, indicating that MTF-ImDenseNet169 can construct different pulse wave signal recognition models.

[0113] Visualization and interpretation analysis of pulse wave signal based on Grad-CAM

[0114] The clinical interpretability of pulse diagnosis is fundamental to guiding physicians in following diagnostic logic and strategies. Therefore, making deep learning models more interpretable in pulse diagnosis is highly significant. The CAM method is a visualization tool for two-dimensional deep learning models, primarily utilizing feature map activation to understand the model's classification criteria. The last convolutional layer of a CNN contains the richest information; the idea behind CAM is to replace fully connected layers with global average pooling layers, calculate the neural network weights based on the target category, and finally display the resulting weight matrix as a heatmap on the original feature map.

[0115] In formula (6) F k This represents the result of accumulating the activation values ​​of the k-th neuron before the global average pooling layer.

[0116]

[0117] Among them, f k (x,y) represents the activation value of the latent space coordinates in the k-th neuron.

[0118] For a given category, the input value of the output layer is f represents the corresponding category c k The weights of the values ​​are determined, and therefore the CAM matrix is ​​defined as shown in formula (7).

[0119]

[0120] While the CAM method can visualize the classification process of deep learning models, it is not suitable for computationally intensive classification tasks because it requires retraining the model. The Grad-CAM method addresses this issue by improving upon CAM, using gradient averaging to calculate weights. This allows visualization of CNNs with arbitrary structures without requiring modification to the network architecture.

[0121] The principle of the Grad-CAM method is to first calculate the gradient of the probability of a given class under the feature map of the convolutional layer. This step replaces the GAP calculation in the CAM algorithm. The weight calculation process of the class c corresponding to the kth feature map in the convolutional neural network is shown in formula (8).

[0122]

[0123] In equation (8), y c This represents the probability of the model classifying a category. This represents the pixel value of the k feature map at coordinates (i, j).

[0124] The process of calculating the contribution rate matrix based on the obtained gradient and weights is as shown in formula (9).

[0125]

[0126] In formula (9), indicates the weight coefficient corresponding to the feature map.

[0127] As Figure 10 shown, Grad-CAM takes the pulse wave signal image as input, uses the proposed model, and visualizes the key parts through the feature reverse weight propagation technology.

[0128] Using Grad-CAM technology, the key feature performance area of different pulse waves is explored, aiming to provide a theoretical basis for better identification of different pulses. By analyzing the interpretation results of the three groups of pulse wave signals, Grad-CAM highlights the indicator area, which is usually used as the discriminant marker area for pulse wave image classification.

[0129] As Figure 11 shown, Figure 11 , (a) is flat pulse, (b) is slippery pulse, (c) is string pulse; according to Figure 11 the result analysis, the main identification parts of the three types of pulses are reflected in several areas, namely the main wave, the pre-wave of heavy pulse, the descending middle isthmus and the heavy pulse wave, and these key areas are reflected in features H1, W1, H2, H3 and other features. The color part indicates the main area for realizing the identification of different pulse wave signals, reflecting the identifiable features of the pulse wave. In the visualization result of Grad-CAM, the high-probability area activation map (red part) highlights the main difference part of the pulse wave, which is important for the classification and identification of the pulse wave. Correspondingly, the low-probability activation map (yellow part) shows the background area with less difference, which has less influence on the identification of the pulse wave.

[0130] Figure 12 is a typical waveform diagram of the pulse wave signal, and the characteristic data of the signal is marked in the diagram, such as H1, W1, H2, H3, etc.

[0131] The image classification and identification method for pulse wave signals disclosed in the embodiments of the application uses the Grad-CAM visualization method to provide important theoretical and practical support for pulse wave identification; through the visualization analysis of Grad-CAM, the discrimination model of the pulse diagnosis equipment can be evaluated, and the key areas and key features of the model in identifying different pulse wave signals can be understood. While supporting the Chinese medical classics, it also provides more intuitive diagnostic basis for the subsequent development of pulse diagnosis equipment, so that doctors can more intuitively and accurately understand the discrimination parts and feature changes of different pulse wave signals.

[0132] The image classification and recognition method for pulse wave signals disclosed by the embodiments of the present application encodes single, two and multi-pulse wave signals into two-dimensional images based on the Markov transition field method, and uses the Mixup technology to perform image fusion enhancement on the two-dimensional images, thereby improving the feature complementarity and data quality. Then, an improved MTF-ImDenseNet169 network model is proposed, and a multi-scale parallel convolution layer and a feature channel attention block are introduced to capture different scale pulse wave signal features and highlight the information of different pulse waves in the feature map. In addition, a smoother activation function Swish function is introduced in the model to improve the non-linear ability of the network and make the network training process more convergent and efficient. Finally, the class activation mapping visualization technology is used to visualize the discriminant area that plays a decisive role in the pulse wave signal, and the image classification and recognition of the pulse wave signal is realized. The method establishes the trust of the pulse wave signal recognition model and provides a beneficial classification and recognition method for the recognition of different pulses, which has a potential good application prospect.

[0133] The technical solutions disclosed in the embodiments of the present application and the technical details disclosed in the embodiments are only exemplary to illustrate the inventive concept of the present application, and do not constitute a limitation on the technical solutions of the embodiments of the present application. Any conventional change, replacement or combination of the technical details disclosed in the embodiments of the present application has the same inventive concept as the present application, and is within the protection scope of the claims of the present application.

Claims

1. A method for image-based classification and recognition of pulse wave signals, characterized in that, include: The pulse wave signal was collected and preprocessed, and then converted into a two-dimensional image using Markov transformation. Image features of two-dimensional images are extracted using an enhanced MTF-ImDenseNet169 network model; Grad-CAM is used to visualize image features, and then pulse wave signals are classified and identified based on the visualized indication areas.

2. The image-based classification and recognition method for pulse wave signals according to claim 1, characterized in that, Preprocessing of pulse wave signals includes: Pulse wave signals from the cun, guan, and chi positions were collected and preprocessed separately. MTF is used to convert single, dual, and triple pulse wave signals into corresponding MTF pulse maps; The Mixup technique was used to perform image fusion on the obtained MTF pulse map.

3. The image-based classification and recognition method for pulse wave signals according to claim 1, characterized in that, The conversion of pulse signals into two-dimensional images using Markov transform includes: The pulse wave signal is represented as a time series X = (x1, x2, ... x... i ), i = 1, 2, ..., n; The time series is divided into Q bins based on the range of values, such that each x i Mapping to the corresponding Q j (j = 1, 2, ..., i); Calculate q along each time step using a first-order Markov chain. j The transitions between these points construct a Markov transition matrix M, which is defined by the following equation: Among them, w ij Indicates from q i to q j The transition probability, and after normalization, satisfies the condition. Arrange each transition probability in chronological order, and expand the Markov transition matrix M into an n×n matrix M; the n×n matrix M is defined as follows: Among them, M ij (i,j∈1,2,…n) is the signal x i The corresponding region to signal y i The transition probability of the corresponding region; elements M in matrix M ij The value range is [0,1]; The following formula is used to scale the value of each element in matrix M to 0-255, so that the elements in matrix M correspond to the pixel values ​​in the image, thus obtaining a two-dimensional image; I(i,j)=int(255M ij ) Where I(i,j) is the pixel value in the i-th row and j-th column of the image, and int(*) is the rounding function.

4. The image-based classification and recognition method for pulse wave signals according to claim 1, characterized in that, The MTF-ImDenseNet169 model contains 171 learnable layers, of which 166 are located between the Dense Block and the first convolutional layer, along with one fully connected (FC) layer and three transition layers, as listed in the table below:

5. The image-based classification and recognition method for pulse wave signals according to claim 4, characterized in that, The functions of the channel attention mechanism SE block include: For a given mapping input X (X∈R) H'×W'×C' ) to feature mapping U(U∈R) H×W×C The corresponding SE block is constructed to perform feature recalibration; where feature U is first compressed into a 1×1×C feature by the compression operation Fsq, then the feature from Fsq is excited by the activation operation Fex, and finally the recalibrated feature is obtained by Fscale.

6. The image-based classification and recognition method for pulse wave signals according to claim 4, characterized in that, Image features extracted from two-dimensional images using the MTF-ImDenseNet169 network model include: The two-dimensional pulse image is first passed through three branch convolutional kernels of 1×1, 3×3, and 5×5, and then input into the ImDenseNet network through a 3×3 max pooling layer. The two-dimensional pulse wave signal image information is then extracted through four Dense blocks, three SE blocks, and three Transition layers.

7. The image-based classification and recognition method for pulse wave signals according to claim 1, characterized in that, The activation function of the MTF-ImDenseNet169 network model is the Swish function, and the expression of the Swish function is as follows:

8. The image-based classification and recognition method for pulse wave signals according to claim 1, characterized in that, Visualizing image features using Grad-CAM includes: The gradient of the probability of a given class under the feature map of the convolutional layer is calculated. The weight calculation process for the class c corresponding to the k-th feature map in the convolutional neural network is shown in the following formula: Among them, y c This represents the probability of the model classifying a category. This represents the pixel value of the k-feature map at coordinates (i, j); The contribution rate matrix is ​​calculated based on the obtained gradient and weights, as shown in the following formula: in, This represents the weight coefficients of the corresponding feature map.

Images