A pulse condition modeling enhancement system based on image recognition

Abstract

The application provides a pulse condition modeling enhancement system based on image recognition, comprising a pulse position labeling module, a pulse image acquisition module, a pulse wave tracking module, a pulse condition structure modeling module and a pulse condition modeling enhancement module. The application belongs to the technical field of image recognition pulse condition modeling, and specifically refers to a pulse condition modeling enhancement system based on image recognition, which acquires a pulse image sequence through three-pulse position pre-labeling and high-frame-rate image acquisition; adopts an image processing method combining motion amplification and phase coding to extract pulse wave propagation characteristics, and further constructs a structured pulse condition model, fuses a multi-dimensional feature vector, self-encoding compression and multi-label classification, and realizes automatic recognition and visual expression of common traditional Chinese medicine pulse conditions.

Description

Technical Field

[0001] The invention belongs to the technical field of image recognition pulse modeling, and in particular relates to a pulse modeling enhancement system based on image recognition. Background Art

[0002] The pulse modeling enhancement system based on image recognition is an intelligent auxiliary tool that combines modern computer vision technology. The system uses a high-precision camera or optical sensor to collect dynamic images of the radial artery area of ​​the patient's wrist (such as skin micro-vibrations), and uses image processing algorithms to extract the spatiotemporal characteristics of the pulse pulsation (such as waveform, frequency, amplitude). These visual features are then mapped into TCM pulse classifications through a machine learning model. Its function is to objectively quantify pulse modeling information, reduce reliance on subjective experience, and assist TCM practitioners in improving the consistency of pulse control. At the same time, it can realize the digital archiving of pulse data and promote the modernization of TCM.

[0003] However, in the existing pulse modeling process combined with image recognition technology, there are technical problems such as inaccurate pulse position positioning, insufficient image acquisition frame rate and poor spatial stability of the pulsation area; in the existing pulse wave feature analysis process, there are technical problems such as one-sided single pulse position signal analysis, easy interference of timing characteristics by noise and difficulty in accurately capturing phase propagation characteristics; in the existing pulse modeling process, there are technical problems such as fragmentation of pulse feature dimensions, lack of integration of indicators, unexplainable modeling results and unstable pulse interpretation. Summary of the Invention

[0004] In view of the above situation, in order to overcome the defects of the prior art, the present invention provides a pulse modeling enhancement system based on image recognition. In view of the technical problems that exist in the existing pulse modeling process combined with image recognition technology, such as inaccurate pulse position positioning, insufficient image acquisition frame rate and poor spatial stability of the pulsation area, this solution creatively adopts an integrated strategy combining three-part pulse position pre-labeling, high-frame rate image acquisition, dynamic response-driven local pulsation area extraction and image time series synchronous calibration, effectively ensuring the spatial and temporal consistency of the three pulse positions in the image sequence, greatly improving the availability of pulse image data and the capture accuracy of pulse dynamic characteristics, and providing a high-quality original data foundation for subsequent waveform modeling; in view of the technical problems that exist in the existing pulse wave feature analysis process, such as one-sided single pulse position signal analysis, easy interference of timing characteristics by noise and difficult accurate capture of phase propagation characteristics, this solution creatively adopts a three-pulse position pulse wave tracking method combined with motion amplification and phase encoding, The pulse wave propagation process is tracked, and the propagation process of the pulse wave between the three pulses of Cun, Guan and Chi is modeled through multi-scale wavelet decomposition, dynamic enhancement filtering and Hilbert transform. Key propagation characteristics such as propagation delay, amplitude change and waveform drift can be extracted, and structured modeling of the pulse wave propagation pattern is realized, with stronger ability to distinguish pathological states and interpret pulse conditions. In view of the technical problems of fragmentation of pulse feature dimensions, lack of indicator fusion, uninterpretable modeling results and unstable pulse interpretation in the existing pulse modeling process, this scheme creatively adopts a multi-indicator fusion and alignment pulse modeling method, constructs structured feature vectors of three pulse positions, extracts pulse center factor vectors by autoencoder compression, and introduces a fuzzy label multi-label classification mechanism, which significantly improves the robustness, nonlinear representation ability and clinical interpretability of pulse modeling, and can achieve accurate prediction and structured output of twelve common Chinese medicine pulse conditions, providing a key infrastructure for the intelligent modeling of Chinese medicine pulse conditions.

[0005] The technical solution adopted by the present invention is as follows: the present invention provides a pulse modeling enhancement system based on image recognition, comprising a pulse position marking module, a pulse image acquisition module, a pulse wave tracking module, a pulse structure modeling module and a pulse modeling enhancement module;

[0006] The pulse position marking module is used for pre-marking three pulse positions, obtaining three pulse position positioning reference data through the pre-marking of the three pulse positions, and sending the three pulse position positioning reference data to the pulse image acquisition module and the pulse wave tracking module;

[0007] The pulse image acquisition module is used for collecting pulse high frame rate images, obtaining three pulse position and pulse sequence data through the pulse high frame rate image acquisition, and sending the three pulse position and pulse sequence data to the pulse wave tracking module;

[0008] The pulse wave tracking module is used for tracking the pulse wave propagation process, obtaining three parts of pulse position pulsation waveform characteristic data through the pulse wave propagation process tracking, and sending the three parts of pulse position pulsation waveform characteristic data to the pulse structure modeling module and the pulse modeling enhancement module;

[0009] The pulse structure modeling module is used for pulse model modeling, obtains pulse position structured pulse characteristic data through pulse model modeling, and sends the structured pulse characteristic data to the pulse modeling enhancement module;

[0010] The pulse modeling enhancement module is used for pulse modeling visualization enhancement, and obtains pulse modeling enhancement visualization reference data through pulse modeling visualization enhancement.

[0011] Furthermore, the three pulse positions are pre-labeled to collect original images required for pulse modeling and label the positions of the Cun, Guan, and Chi pulses, specifically by acquiring original images to obtain wrist original images, and manually interactively labeling the wrist original images to obtain three pulse position positioning reference data;

[0012] The three pulse location reference data include the image position coordinates of the three pulse locations, shooting timestamps and annotated timestamp auxiliary information.

[0013] Furthermore, the pulsation high frame rate image acquisition is used to obtain a high frame rate pulsation image sequence, specifically after the three pulse positions are pre-labeled and the three pulse position positioning reference data are recorded, a synchronous acquisition and labeling method is used to perform high frame rate image acquisition of the three pulse positions to obtain the pulsation sequence data of the three pulse positions, including the following steps: equipment preset, pulse position area selection, image preprocessing and sensor calibration;

[0014] The three pulse position pulsation sequence data specifically include 500 frames of pulsation image sequence data of the Cun pulse, Guan pulse and Chi pulse.

[0015] Furthermore, the pulse wave propagation process tracking is used to extract the propagation characteristics of the pulse wave between the Cun, Guan, and Chi pulses. Specifically, based on the three-part pulse position pulsation sequence data, a three-part pulse wave tracking method combining motion amplification and phase encoding is used to track the pulse wave propagation process to obtain the three-part pulse position pulsation waveform characteristic data, including the following steps: local pulsation area extraction, time domain motion amplification, time sequence phase signal extraction, pulse wave propagation characteristic calculation, and three-part pulse waveform structured modeling;

[0016] The local pulsation region extraction is used to locate the dynamic pulsation core region within the three pulse positions from the image sequence. Specifically, a lightweight dynamic three-dimensional convolutional neural network is constructed, and the temporal standard deviation of each pixel in the pulse position image sequence is calculated. A dynamic response matrix is ​​constructed by combining spatial gradient weights. Based on the dynamic response, the center region of the local pulsation region is extracted to obtain the local pulsation region image data.

[0017] Extracting the central area of ​​the local pulsation area, specifically taking the central area of ​​the top 2% of the dynamic response as the local pulsation area image data;

[0018] The time domain motion amplification is specifically performed by performing multi-scale wavelet decomposition on the local pulsation area image data, and by constructing an improved adaptive time domain amplification formula, performing dynamic filtering enhancement to obtain the grayscale pixel value of the image after time domain adaptive amplification, and extracting the statistical curve of the grayscale pixel value of the image after time domain adaptive amplification over time to obtain the pulsation signal data;

[0019] The time series phase signal extraction is specifically based on the pulsation signal data, combined with the local pulsation area image data, by performing Hilbert transform on the grayscale time series in the local pulsation area image data, and by calculating the phase average trajectory and phase standard deviation index of each pulse position to obtain three pulse position phase code sequence data;

[0020] The pulse wave propagation characteristic calculation is specifically performed based on the three-part pulse phase code sequence data by sequentially calculating the wave propagation delay characteristic, the amplitude variation coefficient characteristic, and the waveform drift coefficient characteristic to obtain the pulse wave propagation characteristic data;

[0021] The three-pulse waveform structured modeling is specifically to construct a standardized pulse waveform characteristic structure based on the three-pulse position pulse wave propagation characteristic data to obtain the three-pulse position pulsation waveform characteristic data;

[0022] The three parts of pulse waveform characteristic data specifically include vibration main frequency characteristics, amplitude characteristics, wave propagation delay characteristics, amplitude variation coefficient characteristics and waveform drift coefficient characteristics.

[0023] Furthermore, the pulse model is used to establish a digital pulse model, specifically based on the three pulse waveform feature data, a pulse modeling method of multi-index fusion alignment is adopted to perform pulse modeling to obtain pulse position structured pulse feature data, including the following steps: pulsation feature vector construction, feature fusion compression, pulse category mapping and structured pulse model construction;

[0024] The pulsation feature vector construction is specifically to construct a standardized multidimensional feature vector for each pulse position based on the three-part pulse position pulsation waveform feature data to obtain the three-part pulse position pulsation multidimensional feature vector;

[0025] The pulse category mapping is specifically based on the pulse center factor vector data, by constructing a predefined pulse type label set, adopting a multi-label classifier, performing pulse category mapping, obtaining pulse category prediction data, and introducing a fuzzy label representation method to perform uncertainty optimization on the pulse category prediction data to obtain pulse category comprehensive modeling data;

[0026] The structured pulse model is constructed specifically by outputting structured pulse results based on the comprehensive modeling data of the pulse category to obtain structured pulse characteristic data of the pulse position.

[0027] Furthermore, the pulse modeling visualization enhancement is used to visualize the pulse and propagation process and provide an auxiliary basis for pulse modeling. Specifically, based on the three-part pulse waveform characteristic data and the pulse position structured pulse characteristic data, the pulse modeling visualization enhancement is performed by generating a pulse visual output and a timing comparison diagram to obtain pulse modeling enhancement visualization reference data, and pulse modeling assistance is performed based on the pulse modeling enhancement visualization reference data.

[0028] The beneficial effects achieved by the present invention using the above scheme are as follows:

[0029] (1) In view of the technical problems of inaccurate pulse position positioning, insufficient image acquisition frame rate and poor spatial stability of the pulsation area in the existing pulse modeling process combined with image recognition technology, this solution creatively adopts an integrated strategy that combines three-part pulse position pre-labeling, high-frame-rate image acquisition, dynamic response-driven local pulsation area extraction and image time series synchronization calibration, effectively ensuring the spatial and temporal consistency of the three pulse positions in the image sequence, greatly improving the availability of pulse image data and the accuracy of capturing pulse dynamic characteristics, and providing a high-quality original data foundation for subsequent waveform modeling;

[0030] (2) In view of the technical problems in the existing pulse wave feature analysis, such as the one-sided analysis of single pulse position signal, the susceptibility of time series characteristics to noise interference, and the difficulty in accurately capturing phase propagation characteristics, this scheme creatively adopts a three-pulse position pulse wave tracking method that combines motion amplification and phase encoding to track the pulse wave propagation process. By means of multi-scale wavelet decomposition, dynamic enhancement filtering and Hilbert transform, the propagation process of the pulse wave between the three pulses of Cun, Guan and Chi is modeled, and key propagation characteristics such as propagation delay, amplitude change and waveform drift can be extracted, thus realizing the structured modeling of the pulse wave propagation mode, and having stronger ability to distinguish pathological conditions and interpret pulse conditions;

[0031] (3) In view of the technical problems in the existing pulse modeling process, such as the fragmentation of pulse feature dimensions, lack of indicator integration, uninterpretable modeling results and unstable pulse interpretation, this scheme creatively adopts a multi-indicator fusion and alignment pulse modeling method. By constructing the structured feature vector of the three pulse positions, extracting the pulse center factor vector by autoencoder compression, and introducing a fuzzy label multi-label classification mechanism, the robustness, nonlinear representation ability and clinical interpretability of pulse modeling are significantly improved. It can achieve accurate prediction and structured output of twelve common TCM pulse conditions, and provide a key infrastructure for the intelligent modeling of TCM pulse conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 A schematic structural diagram of a pulse condition modeling enhancement system based on image recognition provided by the present invention;

[0033] Figure 2 A schematic flow chart of the steps performed by the system is provided for the present invention;

[0034] Figure 3 A schematic flow chart of the steps performed by the pulse image acquisition module;

[0035] Figure 4 A flowchart illustrating the steps performed by the pulse wave tracking module;

[0036] Figure 5 Flowchart showing the steps performed by the pulse structure modeling module.

[0037] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention. DETAILED DESCRIPTION

[0038] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments; based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0039] Example 1, see Figure 1 The technical solution adopted by the present invention is as follows: the present invention provides a pulse modeling enhancement system based on image recognition, including a pulse position marking module, a pulse image acquisition module, a pulse wave tracking module, a pulse structure modeling module and a pulse modeling enhancement module;

[0040] The pulse position marking module is used for pre-marking three pulse positions, obtaining three pulse position positioning reference data through the pre-marking of the three pulse positions, and sending the three pulse position positioning reference data to the pulse image acquisition module and the pulse wave tracking module;

[0041] The pulse image acquisition module is used for collecting pulse high frame rate images, obtaining three pulse position and pulse sequence data through the pulse high frame rate image acquisition, and sending the three pulse position and pulse sequence data to the pulse wave tracking module;

[0042] The pulse wave tracking module is used for tracking the pulse wave propagation process, obtaining three parts of pulse position pulsation waveform characteristic data through the pulse wave propagation process tracking, and sending the three parts of pulse position pulsation waveform characteristic data to the pulse structure modeling module and the pulse modeling enhancement module;

[0043] The pulse structure modeling module is used for pulse model modeling, obtains pulse position structured pulse characteristic data through pulse model modeling, and sends the structured pulse characteristic data to the pulse modeling enhancement module;

[0044] The pulse modeling enhancement module is used for pulse modeling visualization enhancement, and obtains pulse modeling enhancement visualization reference data through pulse modeling visualization enhancement.

[0045] By performing the above operations, in order to address the technical problems of inaccurate pulse position positioning, insufficient image acquisition frame rate and poor spatial stability of the pulsation area in the existing pulse modeling process combined with image recognition technology, this solution creatively adopts an integrated strategy that combines three-part pulse position pre-labeling, high-frame rate image acquisition, dynamic responsiveness-driven local pulsation area extraction and image time series synchronous calibration, effectively ensuring the spatial and temporal consistency of the three pulse positions in the image sequence, greatly improving the availability of pulse image data and the accuracy of capturing pulse dynamic characteristics, and providing a high-quality original data foundation for subsequent waveform modeling.

[0046] Example 2: This example is based on the above example. Figure 1 、 Figure 2 The three pulse positions are pre-marked for collecting the original images required for pulse modeling and marking the positions of the Cun, Guan, and Chi pulses. Specifically, the original image of the wrist is obtained by collecting the original image, and the three pulse position positioning reference data is obtained by manually interactively marking the original image of the wrist;

[0047] The original image is captured by using a standard visible light camera to capture images of the radial side of the patient's wrist when the wrist is placed horizontally, with the shooting angle perpendicular to the wrist plane, and with a fixed bracket or shooting platform to ensure image stability;

[0048] The manual interactive annotation specifically includes manually annotating the three pulse points of the Cun pulse, Guan pulse and Chi pulse in sequence, and recording the image pixel coordinates corresponding to the three pulse points;

[0049] Preferably, in the manual interactive annotation process, in order to ensure the consistency of the image coordinates between the pre-annotation of the three pulse positions and the pulse high frame rate image acquisition process, the system adopts a synchronous acquisition and annotation method, supplemented by a positioning physical template to constrain the wrist posture, to ensure the spatial consistency of the pulse position area in the image sequence;

[0050] The three pulse location reference data include the image position coordinates of the three pulse locations, shooting timestamps and annotated timestamp auxiliary information.

[0051] Example 3: This example is based on the above example. Figure 1 、 Figure 2 and Figure 3 The pulsation high frame rate image acquisition is used to obtain a high frame rate pulsation image sequence. Specifically, after the three pulse positions are pre-labeled and the three pulse position positioning reference data are recorded, a synchronous acquisition and labeling method is used to perform high frame rate image acquisition of the three pulse positions to obtain the pulsation sequence data of the three pulse positions, including the following steps: device preset, pulse position area selection, image preprocessing and sensor calibration;

[0052] The device is preset to specifically select a high frame rate image acquisition device, set the acquisition frame rate to 500 fps, set the acquisition time to 15 seconds, and record the image frame timestamp, and acquire images when the radial side of the patient's wrist is placed horizontally to obtain a high frame rate image sequence;

[0053] The pulse position region is selected by locating the ROI regions corresponding to the three pulse position positions in the high frame rate image sequence based on the three pulse position positioning reference data, and obtaining the three pulse position high frame rate image sequence by ROI cropping;

[0054] The image preprocessing is specifically to perform grayscale, denoising and contrast enhancement processing on the images in the three pulse position high frame rate image sequence to obtain optimized image data;

[0055] The sensor calibration specifically includes recording light intensity parameters during high frame rate image acquisition, and performing brightness normalization processing on the optimized image data according to the light intensity parameters to obtain three-part pulse position pulsation sequence data;

[0056] The three pulse position pulsation sequence data specifically include 500 frames of pulsation image sequence data of the Cun pulse, Guan pulse and Chi pulse.

[0057] Example 4: This example is based on the above example. Figure 1 and Figure 4The pulse wave propagation process tracking is used to extract the propagation characteristics of the pulse wave between the Cun, Guan, and Chi pulses. Specifically, based on the three-part pulse position pulsation sequence data, a three-part pulse wave tracking method combining motion amplification and phase encoding is used to track the pulse wave propagation process to obtain the three-part pulse position pulsation waveform characteristic data, including the following steps: local pulsation area extraction, time domain motion amplification, time sequence phase signal extraction, pulse wave propagation characteristic calculation, and three-part pulse waveform structured modeling;

[0058] The local pulsation region extraction is used to locate the dynamic pulsation core region within the three pulse positions from the image sequence. Specifically, a lightweight dynamic three-dimensional convolutional neural network is constructed, and the temporal standard deviation of each pixel in the pulse position image sequence is calculated. A dynamic response matrix is ​​constructed by combining spatial gradient weights. Based on the dynamic response, the center region of the local pulsation region is extracted to obtain the local pulsation region image data.

[0059] The lightweight dynamic three-dimensional convolutional neural network specifically refers to a lightweight dynamic recognition network that combines three-dimensional convolution, SE attention and temporal pooling;

[0060] The calculation formula of the time standard deviation is:

[0061] ;

[0062] Where, is the time standard deviation, p is the pulse position index, x is the horizontal pixel index, y is the vertical pixel index, N is the total number of image frames, specifically 500, t is the time index, and is used as the image frame index, is the grayscale value of the image pixel at the p-th pulse position in the t-th frame, is the image pixel grayscale value of all frames at the pth pulse position

[0063] The calculation formula of the dynamic response matrix is:

[0064] ;

[0065] Where, is the dynamic responsiveness, is the time standard deviation, Is the adjustment factor, the default value is 0.1, is the spatial gradient value of the pixel point of the initial frame image of the p-th pulse position, which is calculated by the Sobel operator;

[0066] Extracting the central area of ​​the local pulsation area, specifically taking the central area of ​​the top 2% of the dynamic response as the local pulsation area image data;

[0067] The time domain motion amplification is specifically performed by performing multi-scale wavelet decomposition on the local pulsation area image data, and by constructing an improved adaptive time domain amplification formula, performing dynamic filtering enhancement to obtain the grayscale pixel value of the image after time domain adaptive amplification, and extracting the statistical curve of the grayscale pixel value of the image after time domain adaptive amplification over time to obtain the pulsation signal data;

[0068] The multi-scale wavelet decomposition specifically decomposes the local pulsation image sequence into low-frequency structure and high-frequency detail parts according to the scale level, and extracts the pulsation component in the 0.8-2.5Hz frequency band;

[0069] The calculation formula of the improved adaptive time domain amplification formula is:

[0070] ;

[0071] Where, is the grayscale pixel value of the image after time-domain adaptive amplification, is the original image pixel value, It is an adaptive amplification factor, and the specific heart rate main frequency characteristics are dynamically adjusted. are the wavelet decomposition coefficients, is the wavelet decomposition average coefficient, p is the pulse position index, which is used to indicate the level of wavelet decomposition during the wavelet decomposition process, x is the horizontal pixel index, y is the vertical pixel index, t is the time index, and f is the frequency band representation, which is used to represent the 0.8–2.5 Hz frequency band;

[0072] The time series phase signal extraction is specifically based on the pulsation signal data, combined with the local pulsation area image data, by performing Hilbert transform on the grayscale time series in the local pulsation area image data, and by calculating the phase average trajectory and phase standard deviation index of each pulse position to obtain three pulse position phase code sequence data;

[0073] The calculation formula of the phase average trajectory is:

[0074] ;

[0075] Where, is the phase-encoded time series of the p-th pulse position, used to represent the phase average trajectory, R is the local pulsation area, x is the horizontal pixel index, y is the vertical pixel index, is the instantaneous phase value obtained by writing Baud transform;

[0076] The calculation formula of the phase standard deviation index is:

[0077] ;

[0078] Where, It is a phase standard deviation index, which is used to measure the pulsation phase synchronization in a local area;

[0079] The pulse wave propagation characteristic calculation is specifically performed based on the three-part pulse phase code sequence data by sequentially calculating the wave propagation delay characteristic, the amplitude variation coefficient characteristic, and the waveform drift coefficient characteristic to obtain the pulse wave propagation characteristic data;

[0080] The wave propagation delay characteristic is calculated using the maximum cross-correlation phase shift calculation method, and the calculation formula is:

[0081] ;

[0082] Where, is the wave propagation delay characteristic, p is the pulse position index, q is the propagation pulse position index, is the time offset index, Corr is the cross-correlation function, is the phase-encoded time series of the p-th pulse position, is the offset time The phase-coded time series of the qth propagation pulse position;

[0083] The calculation formula of the amplitude variation coefficient characteristic is:

[0084] ;

[0085] Where A p is the phase waveform amplitude of the p-th pulse position, is the amplitude variation coefficient characteristic, A q is the phase waveform amplitude of the qth propagation pulse position;

[0086] The calculation formula of the waveform drift coefficient characteristic is:

[0087] ;

[0088] Where, is the waveform drift coefficient characteristic, t is the time index, is the phase-coded time series after the propagation delay corresponding to the qth propagation pulse position is aligned;

[0089] The three-pulse waveform structured modeling is specifically to construct a standardized pulse waveform characteristic structure based on the three-pulse position pulse wave propagation characteristic data to obtain the three-pulse position pulsation waveform characteristic data;

[0090] The three parts of pulse waveform characteristic data specifically include vibration main frequency characteristics, amplitude characteristics, wave propagation delay characteristics, amplitude variation coefficient characteristics and waveform drift coefficient characteristics.

[0091] By performing the above operations, in order to address the technical problems in the existing pulse wave feature analysis, such as the one-sided analysis of single pulse position signals, the susceptibility of timing characteristics to noise interference, and the difficulty in accurately capturing phase propagation characteristics, this solution creatively adopts a three-pulse position pulse wave tracking method combining motion amplification and phase encoding to track the pulse wave propagation process. Through multi-scale wavelet decomposition, dynamic enhancement filtering and Hilbert transform, the propagation process of the pulse wave between the Cun, Guan and Chi pulses is modeled, and key propagation characteristics such as propagation delay, amplitude change and waveform drift can be extracted, thus realizing structured modeling of the pulse wave propagation pattern and having stronger ability to distinguish pathological conditions and interpret pulse conditions.

[0092] Example 5: This example is based on the above example. Figure 1 and Figure 5 The pulse model is used to establish a digital pulse model, specifically based on the three pulse waveform feature data, a multi-index fusion and alignment pulse modeling method is used to perform pulse modeling to obtain pulse position structured pulse feature data, including the following steps: pulsation feature vector construction, feature fusion compression, pulse category mapping and structured pulse model construction;

[0093] The pulsation feature vector construction is specifically to construct a standardized multidimensional feature vector for each pulse position based on the three-part pulse position pulsation waveform feature data to obtain the three-part pulse position pulsation multidimensional feature vector;

[0094] The calculation formula of the three-part pulse position pulsation multidimensional feature vector is:

[0095] ;

[0096] Where, F p is the multidimensional feature vector of the three pulse positions, p is the pulse position index, and the specific value range is {Cun pulse, Guan pulse, Chi pulse}, n is the total number of pulse features, and the specific value is 5, is the main frequency characteristic of vibration, is the amplitude characteristic, is the wave propagation delay characteristic, is the amplitude variation coefficient characteristic, is the waveform drift coefficient characteristic;

[0097] The feature fusion compression is specifically to connect the multi-dimensional feature vectors of the three pulse positions and pulsations in series to form a fusion vector, and adopt an autoencoder to perform nonlinear compression, use the reconstruction error as the loss function and extract the pulse center factor vector data; the pulse center factor vector is used to characterize the nonlinear comprehensive state of the three pulse positions;

[0098] The calculation formula of the fusion vector is:

[0099] ;

[0100] Where F is the fusion vector, is the multidimensional feature vector of Cun pulse, is the multidimensional feature vector of the pulse pulsation, is the multidimensional characteristic vector of radial pulse;

[0101] Preferably, the autoencoder comprises an input layer, an encoder hidden layer, a compression factor layer, a decoder hidden layer and an output layer, wherein the input of the input layer is a fusion vector F with a length of 15, the encoder comprises two layers of nonlinear activation units, the number of nodes in each layer being 12 and 8 respectively, the compression factor layer outputs a six-dimensional pulse center factor vector, and the decoder has a symmetrical structure and is used to output a reconstruction vector; preferably, the encoder uses a ReLU activation function, the training adopts an Adam optimizer, the learning rate is set to 0.001, and the number of iterations is 1000 rounds;

[0102] The pulse category mapping is specifically based on the pulse center factor vector data, by constructing a predefined pulse type label set, adopting a multi-label classifier, performing pulse category mapping, obtaining pulse category prediction data, and introducing a fuzzy label representation method to perform uncertainty optimization on the pulse category prediction data to obtain pulse category comprehensive modeling data;

[0103] Preferably, the predefined pulse type label set specifically includes 12 common TCM pulse types, and each pulse type is encoded as a one-hot encoding; the TCM pulse types specifically include floating, sinking, slow, rapid, slippery, astringent, stringy, slow, surging, fine, deficiency and excess;

[0104] The multi-label classifier specifically adopts a multi-layer perceptron model, including an input layer structure, a hidden layer structure and an output layer structure; the input layer structure dimension is set to 6, for receiving a six-dimensional pulse center factor vector; the hidden layer structure specifically has two hidden layers, the number of nodes in each layer is 16 and 12, and the activation function uses ReLU;

[0105] The output layer structure outputs a 12-dimensional output vector and uses softmax normalization to obtain the predicted label distribution. The calculation formula is:

[0106] ;

[0107] Where, is the pulse category prediction data, c i is the i-th pulse category, i is the pulse category index, Z is the pulse center factor vector, is the original unnormalized predicted data, j is the normalized pulse category index, is the predicted data used for normalization;

[0108] The fuzzy label representation method specifically adopts the Top-3 soft label retention mechanism, and for each pulse position, the pulse category prediction data is selected. The top three categories with the highest values ​​and their corresponding probabilities constitute the fuzzy label vector, and the comprehensive modeling data of the pulse category is obtained;

[0109] Preferably, the Top-3 soft label retention mechanism uses a fuzzy label loss function with a temperature control parameter for training optimization during the training phase, and the calculation formula is:

[0110] ;

[0111] Where, is the fuzzy label loss function with temperature control parameters, i is the pulse category index, is the fuzzy label value corresponding to the i-th pulse category, which represents the probability weight of the true pulse category on the pulse category i. is the original unnormalized prediction data, T is the temperature control parameter, and the default value is 1.5;

[0112] The structured pulse model is constructed specifically by outputting structured pulse results based on the comprehensive modeling data of the pulse category to obtain structured pulse characteristic data of the pulse position.

[0113] By performing the above operations, in order to address the technical problems in the existing pulse modeling process, such as the fragmentation of pulse feature dimensions, lack of indicator fusion, unexplainable modeling results and unstable pulse interpretation, this solution creatively adopts a multi-indicator fusion and alignment pulse modeling method. By constructing structured feature vectors of three pulse positions, extracting the pulse center factor vector by autoencoder compression, and introducing a fuzzy label multi-label classification mechanism, the robustness, nonlinear representation ability and clinical interpretability of pulse modeling are significantly improved, and the accurate prediction and structured output of twelve common TCM pulse conditions can be achieved, providing a key infrastructure for the intelligent modeling of TCM pulse conditions.

[0114] Example 6: This example is based on the above example. Figure 1 The pulse modeling visualization enhancement is used to visualize the pulse and propagation process and provide an auxiliary basis for pulse modeling. Specifically, based on the three-part pulse waveform characteristic data and the pulse position structured pulse characteristic data, the pulse modeling visualization enhancement is performed by generating a pulse visual output and a timing comparison diagram to obtain pulse modeling enhancement visualization reference data, and pulse modeling assistance is performed based on the pulse modeling enhancement visualization reference data.

[0115] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "include," "comprise," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process or method comprising a set of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process or method.

[0116] While the embodiments of the present invention have been shown and described, it will be apparent to those skilled in the art that various changes, modifications, substitutions, and alterations can be made to the embodiments without departing from the principles and spirit of the invention.

[0117] The present invention and its embodiments are described above. This description is not restrictive. The drawings show only one embodiment of the present invention, and the actual structure is not limited thereto. In short, if a person skilled in the art is inspired by this and, without departing from the purpose of the present invention, designs structures and embodiments similar to this technical solution without inventiveness, they shall fall within the scope of protection of the present invention.

Claims

1. A pulse condition modeling enhancement system based on image recognition, characterized in that: It includes pulse position marking module, pulse image acquisition module, pulse wave tracking module, pulse structure modeling module and pulse modeling enhancement module; The pulse position marking module is used for pre-marking three pulse positions, obtaining three pulse position positioning reference data through the pre-marking of the three pulse positions, and sending the three pulse position positioning reference data to the pulse image acquisition module and the pulse wave tracking module; The pulse image acquisition module is used for collecting pulse high frame rate images, obtaining three pulse position and pulse sequence data through the pulse high frame rate image acquisition, and sending the three pulse position and pulse sequence data to the pulse wave tracking module; The pulse wave tracking module is used for tracking the pulse wave propagation process, obtaining three parts of pulse position pulsation waveform feature data through the pulse wave propagation process tracking, and sending the three parts of pulse position pulsation waveform feature data to the pulse structure modeling module and the pulse modeling enhancement module, specifically comprising the following steps: pulsation feature vector construction, feature fusion compression, pulse category mapping and structured pulse model construction; The pulse structure modeling module is used for pulse model modeling, obtains pulse position structured pulse feature data through pulse modeling, and sends the structured pulse feature data to the pulse modeling enhancement module, specifically comprising the following steps: pulsation feature vector construction, feature fusion compression, pulse category mapping and structured pulse model construction; The pulse modeling enhancement module is used for pulse modeling visualization enhancement, and obtains pulse modeling enhancement visualization reference data through pulse modeling visualization enhancement.

2. a kind of pulse condition modeling enhancement system based on image recognition according to claim 1, is characterized in that: The three pulse position pre-marking is used to collect the original images required for pulse modeling and mark the positions of the Cun, Guan and Chi pulses, specifically by collecting the original images to obtain the wrist original image, and by manually interactively marking the wrist original image to obtain the three pulse position positioning reference data; The three pulse location reference data include the image position coordinates of the three pulse locations, shooting timestamps and annotated timestamp auxiliary information.

3. a kind of pulse condition modeling enhancement system based on image recognition according to claim 2, is characterized in that: The pulsation high-frame rate image acquisition is used to obtain a high-frame rate pulsation image sequence. Specifically, after the three pulse positions are pre-marked and the three pulse position positioning reference data are recorded, a synchronous acquisition and marking method is adopted to perform high-frame rate image acquisition of the three pulse positions to obtain the pulsation sequence data of the three pulse positions, including the following steps: equipment preset, pulse position area selection, image preprocessing and sensor calibration; the three pulse position pulsation sequence data specifically includes 500 frames of pulsation image sequence data of the Cun pulse, Guan pulse and Chi pulse.

4. a kind of pulse condition modeling enhancement system based on image recognition according to claim 3, is characterized in that: The pulse wave propagation process tracking is used to extract the propagation characteristics of the pulse wave between the Cun, Guan, and Chi pulses. Specifically, based on the three-part pulse position pulsation sequence data, a three-part pulse wave tracking method combining motion amplification and phase encoding is used to track the pulse wave propagation process to obtain the three-part pulse position pulsation waveform characteristic data, including the following steps: local pulsation area extraction, time domain motion amplification, time sequence phase signal extraction, pulse wave propagation characteristic calculation, and three-part pulse waveform structured modeling; The local pulsation region extraction is used to locate the dynamic pulsation core region within the three pulse positions from the image sequence. Specifically, a lightweight dynamic three-dimensional convolutional neural network is constructed, and the temporal standard deviation of each pixel in the pulse position image sequence is calculated. A dynamic response matrix is ​​constructed by combining spatial gradient weights. Based on the dynamic response, the center region of the local pulsation region is extracted to obtain the local pulsation region image data. The time domain motion amplification is specifically to perform multi-scale wavelet decomposition on the local pulsation area image data, and to construct an improved adaptive time domain amplification formula to perform dynamic filtering enhancement to obtain the grayscale pixel value of the image after time domain adaptive amplification, and to obtain the pulsation signal data by extracting the statistical curve of the grayscale pixel value of the image after time domain adaptive amplification changing with time.

5. a kind of pulse condition modeling enhancement system based on image recognition according to claim 4, is characterized in that: The time series phase signal extraction is specifically based on the pulsation signal data, combined with the local pulsation area image data, by performing Hilbert transform on the grayscale time series in the local pulsation area image data, and by calculating the phase average trajectory and phase standard deviation index of each pulse position to obtain three pulse position phase code sequence data; The pulse wave propagation characteristic calculation is specifically performed based on the three-part pulse phase code sequence data by sequentially calculating the wave propagation delay characteristic, the amplitude variation coefficient characteristic, and the waveform drift coefficient characteristic to obtain the pulse wave propagation characteristic data; The three-pulse waveform structured modeling is specifically to construct a standardized pulse waveform characteristic structure based on the three-pulse position pulse wave propagation characteristic data to obtain the three-pulse position pulsation waveform characteristic data.

6. a kind of pulse condition modeling enhancement system based on image recognition according to claim 5, is characterized in that: The three parts of pulse waveform characteristic data specifically include vibration main frequency characteristics, amplitude characteristics, wave propagation delay characteristics, amplitude variation coefficient characteristics and waveform drift coefficient characteristics.

7. a kind of pulse condition modeling enhancement system based on image recognition according to claim 6, is characterized in that: The pulse model is used to establish a digital pulse model, specifically based on the three pulse position pulsation waveform feature data, adopting a pulse modeling method of multi-index fusion alignment to perform pulse modeling to obtain pulse position structured pulse feature data, including the following steps: pulsation feature vector construction, feature fusion compression, pulse category mapping and structured pulse model construction; The pulsation feature vector construction is specifically to construct a standardized multidimensional feature vector for each pulse position based on the three-part pulse position pulsation waveform feature data to obtain the three-part pulse position pulsation multidimensional feature vector; The feature fusion compression is specifically to connect the multi-dimensional feature vectors of the three pulse positions and pulsations in series to form a fusion vector, and adopt an autoencoder to perform nonlinear compression, use the reconstruction error as the loss function and extract the pulse center factor vector data; the pulse center factor vector is used to characterize the nonlinear comprehensive state of the three pulse positions; The pulse category mapping is specifically based on the pulse center factor vector data, by constructing a predefined pulse type label set, adopting a multi-label classifier, performing pulse category mapping, obtaining pulse category prediction data, and introducing a fuzzy label representation method to perform uncertainty optimization on the pulse category prediction data to obtain pulse category comprehensive modeling data; The structured pulse model is constructed specifically by outputting structured pulse results based on the comprehensive modeling data of the pulse category to obtain structured pulse characteristic data of the pulse position.

8. a kind of pulse condition modeling enhancement system based on image recognition according to claim 7, it is characterized in that: The pulse modeling visualization enhancement is used to visualize the pulse and propagation process and provide an auxiliary basis for pulse modeling. Specifically, based on the three-part pulse waveform characteristic data and the pulse position structured pulse characteristic data, the pulse modeling visualization enhancement is performed by generating a pulse visual output and a timing comparison diagram to obtain pulse modeling enhancement visualization reference data, and pulse modeling assistance is performed based on the pulse modeling enhancement visualization reference data.

Images