Heart disease risk monitoring method and system based on multi-source information

Abstract

The invention relates to a heart disease risk monitoring method and system based on multi-source information, and the method comprises the following steps: obtaining a heart ultrasound image, electrocardiogram data and blood pressure data of a user, extracting corresponding features, carrying out the time sequence alignment, fusion and time sequence enhancement processing, obtaining an updated fusion feature, and carrying out the time sequence enhancement processing. Based on the trained heart disease risk monitoring model, obtaining a probability value of the heart disease risk of the user; wherein the heart disease risk monitoring model is subjected to sparse pruning processing during training, the sparse pruning processing comprises layer-by-layer pruning, block sparseness, greedy optimization weight reconstruction, fast approximate Hessian calculation and parallel pruning, and model compression and reasoning acceleration are achieved. The heart disease risk monitoring accuracy and real-time performance are effectively improved by performing lightweight processing on the monitoring model, and the heart disease risk monitoring method is suitable for edge device deployment and has good lightweight and practicability.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical information processing, in particular to a heart disease risk monitoring method and system based on multi-source information. BACKGROUND

[0002] The statements in this section merely provide background information related to the present application and do not necessarily constitute the prior art.

[0003] The onset of heart disease has the characteristics of suddenness, concealment and rapid progression. Traditional heart disease monitoring methods mainly rely on single modality medical imaging and signal data such as electrocardiogram, echocardiogram, nuclear magnetic resonance imaging and CT scan. Although these methods have high accuracy in professional medical environment, they usually rely on expensive equipment, complex operation process and professional doctor's interpretation, which is difficult to meet the growing demand for home health monitoring and remote medical care.

[0004] With the development of artificial intelligence, wearable devices and multi-modal data fusion technology, data-driven heart disease monitoring technology based on multi-source information can cover physiological signals, image information, behavior data and environmental information. These data reflect the individual's heart health status from different dimensions, and through fusion analysis, it helps to improve the accuracy and robustness of diagnosis.

[0005] However, the existing multi-modal heart disease monitoring system generally has the problems of complex data fusion algorithm, high resource consumption and difficult system deployment, which is difficult to deploy in resource-constrained edge computing devices or wearable terminals.

[0006] In addition, the existing heart disease monitoring model generally focuses on accuracy in design and ignores the lightweight and real-time of the model, which is difficult to meet the real-time demand of heart disease monitoring and early warning. SUMMARY

[0007] In order to solve the technical problems existing in the background art, the present application provides a heart disease risk monitoring method and system based on multi-source information, which can effectively fuse multi-modal data, has low computing resource consumption, is easy to deploy and has real-time monitoring capability, and is beneficial to realize early screening, early warning and personalized health management of heart disease.

[0008] In order to achieve the above purpose, the present application adopts the following technical solutions:

[0009] The first aspect of the present application provides a heart disease risk monitoring method based on multi-source information, comprising the following steps:

[0010] The cardiac ultrasound image, electrocardiogram data and blood pressure data of a user are acquired and preprocessed, and corresponding ultrasound image features, electrocardiogram features and blood pressure encoding features are extracted, and after time alignment, fusion and time enhancement processing, updated fusion features are obtained;

[0011] The updated fusion features are based on a trained heart disease risk monitoring model to obtain a probability value of the user having a heart disease risk;

[0012] During training, the heart disease risk monitoring model is subjected to sparse pruning processing, specifically: a layer-by-layer structured pruning strategy is used to independently sparsify each linear layer of the model, during which the weight matrix is divided into fixed-shaped blocks, and the pruned structure is constrained to satisfy block sparsity to adapt to hardware acceleration; by calculating the contribution of each column in the original weight matrix to the output, the columns with higher contribution are preferentially retained, and the remaining weights are optimized through reconstruction processing to minimize pruning error; through layer-by-layer iteration, a high-sparse block-structured weight matrix is obtained, which optimizes the heart disease risk monitoring model.

[0013] Further, the obtained ultrasound image features, electrocardiogram features and blood pressure encoding features are time-aligned through time sequence relationship to obtain initial features, the initial features are based on attention mechanism to obtain queries, the similarity between the query of the current time frame and the key value of the previous time frame is obtained by calculating the affinity matrix between the query and the key value, and the fusion features are obtained after normalization processing, the fusion features are subjected to time enhancement and time embedding processing to obtain updated fusion features.

[0014] Further, the heart disease risk monitoring model takes the updated fusion features as input, obtains initial state features through an encoder, calculates learnable parameters and forward and backward propagation encoding, and aggregates information in the forward and backward directions to obtain a probability value of the user having a heart disease risk.

[0015] Further, a layer-by-layer structured pruning strategy is used to independently sparsify each linear layer of the model, a pruning objective of minimizing output reconstruction error is constructed, and the optimization form is to minimize the difference between the output generated by the sparsified weight and the original weight under the condition that the input activation matrix remains unchanged, and the pruning optimization objective is as follows:

[0016]

[0017] wherein, WX represents the original output, represents the output after pruning, is a sparse matrix, represents a set of sparse structures that meet the sparsity requirement.

[0018] Further, the weight matrix is divided into fixed-shaped blocks, and the pruned structure is constrained to satisfy block sparsity to adapt to hardware acceleration; specifically, the weight matrix W is divided into multiple blocks, and a block mask M is defined ij ∈{0,1} represents whether the block of the ith row and the jth column is retained, and the sparse matrix is represented as The sparsity s is limited to the proportion of the total number of block units: Where N represents the total number of block division.

[0019] Further, during the sparsification pruning process, a column selection algorithm based on greedy reconstruction is used to calculate the contribution of each column in the original weight matrix to the output, preferentially retain the column with high contribution, and use the pseudo-inverse method to reconstruct it to minimize the error caused by pruning; specifically:

[0020] For the jth column W of the weight matrix j , the influence on the output is calculated, The k columns with the largest contribution are selected to form a subset S; the weights corresponding to the k columns are reconstructed, X represents the Moore-Penrose pseudo-inverse of the corresponding column in X, and the final sparse matrix is obtained.

[0021] The second aspect of the application provides a heart disease risk monitoring system based on multi-source information, comprising:

[0022] A data acquisition module configured to acquire and preprocess the user's echocardiogram, electrocardiogram data and blood pressure data;

[0023] A feature extraction and fusion module configured to extract corresponding ultrasound image features, electrocardiogram features and blood pressure encoding features, respectively, and to obtain updated fusion features through time alignment, fusion and time enhancement processing;

[0024] A risk monitoring module configured to obtain a probability value of the user's risk of heart disease based on the trained heart disease risk monitoring model;

[0025] Wherein, the heart disease risk monitoring model is subjected to sparsification pruning during training, specifically: a layer-by-layer structured pruning strategy is used to independently sparsify each linear layer of the model, during which the weight matrix is divided into fixed-shaped blocks, and the pruned structure is constrained to satisfy block sparsity to adapt to hardware acceleration; by calculating the contribution of each column in the original weight matrix to the output, preferentially retaining the column with higher contribution, and optimizing the remaining weights through reconstruction processing, the pruning error is minimized; through layer-by-layer iteration, a high-sparsity block-structured weight matrix is obtained, and the heart disease risk monitoring model is optimized.

[0026] The third aspect of the present application provides a computer program product comprising computer readable instructions which, when executed on an electronic device, cause the electronic device to implement the above-mentioned cardiac condition risk monitoring method based on multi-source information.

[0027] The fourth aspect of the present application provides an electronic device comprising at least one processor and a memory connected to the processor, the memory being configured to store a computer program; the processor being configured to execute the computer program, so that the electronic device can implement the above-mentioned cardiac condition risk monitoring method based on multi-source information.

[0028] The fifth aspect of the present application provides a computer storage medium carrying one or more computer programs, which, when executed by an electronic device, can enable the electronic device to implement the above-mentioned cardiac condition risk monitoring method based on multi-source information.

[0029] Compared with the prior art, the above one or more technical solutions have the following beneficial effects:

[0030] 1. By means of light-weight sparse pruning processing, the model complexity and inference overhead are significantly reduced. After the model is constructed, the invalid connections, redundant channels or convolution kernels in the model are compressed and simplified by introducing a structured sparsity constraint and pruning mechanism, thereby effectively reducing the number of model parameters and the amount of calculation. On the premise of ensuring that the performance does not decrease substantially, the model is lightened, which provides a feasible path for deployment in edge devices and resource-constrained environments.

[0031] 2. The designed cardiac condition risk monitoring model can effectively integrate data of multiple modalities such as electrocardiogram, cardiac ultrasound image, physiological parameter, etc. Through a multi-modal feature encoding, alignment and fusion mechanism, the model's recognition ability for cardiac conditions is enhanced, and the accuracy and stability of the diagnosis result are improved. It is especially suitable for heterogeneous data scenarios. Through multi-modal fusion modeling, the accuracy and robustness of cardiac condition risk monitoring are improved.

[0032] 3. The model architecture forms a modular design, which is convenient for integration and extension, and is convenient for subsequent integration of more data modalities or embedding of other auxiliary analysis algorithms. It has good engineering practicability and scalability, is helpful for popularization and application in different platforms and systems, and promotes the transition from "high-performance server centralized processing" to "edge terminal intelligent processing", effectively alleviating the dependence on professional medical resources, and providing an efficient, economical and intelligent solution for chronic disease management and early warning. BRIEF DESCRIPTION OF DRAWINGS

[0033] The accompanying drawings, which form a part of this application, are included to provide a further understanding of the application, and are incorporated in and constitute a part of this application. The embodiments of the present application and their descriptions are used to explain the present application, and do not constitute an improper limitation on the present application.

[0034] Figure 1 is a flowchart of a method for monitoring risk of heart disease based on multi-source information provided by one or more embodiments of the present application;

[0035] Figure 2 is a schematic diagram of a model network structure provided by one or more embodiments of the present application;

[0036] Figure 3 is a schematic diagram of a cardiac ultrasound image processing module network structure provided by one or more embodiments of the present application;

[0037] Figure 4 is a schematic diagram of an electrocardiogram processing module network structure provided by one or more embodiments of the present application;

[0038] Figure 5 is a schematic diagram of a blood pressure signal processing module network structure provided by one or more embodiments of the present application;

[0039] Figure 6 is a schematic diagram of a multi-modal feature fusion module network structure provided by one or more embodiments of the present application;

[0040] Figure 7 is a schematic diagram of a heart disease risk monitoring module network structure provided by one or more embodiments of the present application;

[0041] Figure 8 is a schematic diagram of a pruning and lightweight process provided by one or more embodiments of the present application. DETAILED DESCRIPTION

[0042] The present application will be further described below in conjunction with the accompanying drawings and embodiments.

[0043] It should be noted that the following detailed description is exemplary and is intended to provide further explanation of the present application. Unless otherwise indicated, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs.

[0044] Embodiment One:

[0045] As shown in Figure 1 , the method for monitoring risk of heart disease based on multi-source information includes the following steps:

[0046] S1, acquiring multi-modal data for heart health monitoring, including cardiac ultrasound images, electrocardiograms, blood pressure, etc.

[0047] In this embodiment, the acquired data is: ultrasound image where T represents the timestamp. Electrocardiogram ECG data where C represents the number of ECG leads and T represents the total timestamp. Blood pressure data wherein T represents a time stamp, and "2" respectively represents a systolic blood pressure SBP sequence and a diastolic blood pressure DBP sequence.

[0048] S2, constructing a model for intelligent monitoring of heart disease, the model comprising a heart ultrasound image processing module, an electrocardiogram signal processing module, a blood pressure signal processing module, a multi-modal feature fusion module, and a heart disease risk monitoring module.

[0049] The overall architecture of the model is shown in Figure 2 The heart ultrasound image processing module encodes the input heart ultrasound video stream, captures features within a single time frame and the relationship between time frames, and thereby captures comprehensive features from the ultrasound video stream that are conducive to heart disease monitoring.

[0050] The electrocardiogram processing module encodes the input electrocardiogram ECG signal and captures corresponding electrocardiogram ECG features in time series.

[0051] The blood pressure processing module extracts features with high correlation to heart disease from the input blood pressure sequence through a deep learning model.

[0052] The multi-modal feature fusion module fuses different features of different modalities to obtain aggregated features.

[0053] The heart disease risk monitoring module uses the idea of state space model to capture strong correlation in time series and monitor the risk probability of heart disease.

[0054] S3, designing a model sparsification pruning technique, including layer-by-layer pruning, block sparsification, greedy optimization weight reconstruction, fast approximate Hessian calculation, and parallel pruning.

[0055] The layer-by-layer pruning divides the sparsification problem of the overall model into local optimization of each layer, avoiding the complexity brought by global optimization. Block sparsification uses a structured sparsification method, which has good hardware compatibility, especially for CPU inference acceleration, and is more easily utilized by hardware than non-structured sparsification. Greedy optimization weight reconstruction performs "pruning-reconstruction" operation on each layer, maximally reduces errors by retaining key parameters and reconstructing outputs. The fast approximate Hessian calculation method realizes efficient parameter importance estimation, significantly reduces computational complexity, and avoids resource overhead caused by traditional methods due to complexity problems. Parallel pruning supports multi-core CPU parallel processing through independent calculation of each layer, effectively reducing memory occupation and time cost in the overall pruning process.

[0056] S4, collect medical images and related physiological index data of the patient's heart, including echocardiogram, electrocardiogram, blood pressure and other data, input into the trained heart disease monitoring model, use the model to process and analyze the input ultrasonic image and related physiological indicators, monitor the risk probability of heart disease, and make an early warning when a future heart disease risk is found.

[0057] The echocardiogram processing module in S2, as shown in Figure 3 includes the following steps:

[0058] S211, input a single time frame of ultrasonic image image t ∈data image , output features Where t represents the position of the current time frame in the total timestamp.

[0059] S212, down-sampling. The down-sampling layer uses a convolutional feedforward network as a feature extractor to construct an intermediate feature pyramid of different resolutions of the input image

[0060] Specifically, the output result of the previous layer of convolution is input into the convolution layer to obtain the down-sampling feature of the layer The formula is as follows:

[0061]

[0062] Where,

[0063] Where N represents the number of down-sampling layers.

[0064] S213, up-sampling. The up-sampling layer encoder block is composed of multi-head attention self-attention and multi-layer perceptron, wherein the multi-head attention layer is composed of window-based multi-head attention and shift window-based multi-head attention, and an up-sampling intermediate feature pyramid is obtained Specifically, for each intermediate up-sampling layer input feature, denoted as:

[0065]

[0066] Where,

[0067] The up-sampling layer is composed of window-based multi-head attention and shift window-based multi-head attention, and the formula is as follows:

[0068]

[0069] where WMSA denotes window multi-head self-attention, LN denotes a normalization layer, MLP denotes a multi-layer perceptron, and SWMSA denotes a shifted window attention.

[0070] All the above formulas are summarized as follows:

[0071]

[0072] where UP N denotes the Nth up-sampling layer.

[0073] By combining the up-sampling layer and the down-sampling layer, the up-sampling layer makes up for the information lost by the low-level down-sampling layer and restores the local spatial information, while the down-sampling layer pays more attention to the long-distance dependent information in the image. The up-sampling layer and the down-sampling layer complement each other, better capturing the global and local information in the ultrasound image. The segmentation layer fuses the cross-scale information through the cross-attention mechanism, effectively fusing the features of the up-sampling layer and the down-sampling layer while maintaining the consistency of the features. Finally, the fused features are decoded to obtain the final segmentation map.

[0074] S214, segmentation decoding. The segmentation layer fuses cross-scale information through the cross-attention mechanism, effectively fusing the features of the up-sampling layer and the down-sampling layer. The segmentation layer takes the minimum layer and the maximum layer, i.e., the first layer and the fourth layer, as input, and is represented as:

[0075]

[0076] where and denote the features of the first layer and the fourth layer as input, respectively, and P s and P l are the corresponding outputs of the segmentation layer.

[0077] Further, the class embedding is obtained, and the formula is as follows:

[0078]

[0079] where GAP denotes global average pooling, and Norm denotes a normalization layer.

[0080] Further, the features of each other are refined through the cross-attention mechanism to obtain Z s , Z l , and then the final feature is aggregated. The formula is as follows:

[0081] y s = f s (CLS s ) + MCA(LN([f s (CLS s ) || Pl ]));

[0082] Z s =[P s ||q s (y s )];

[0083] y l =f l (CLS l )+MCA(LN([f l (CLS l )||P s ]));

[0084] Z l =[P l ||q l (y l )];

[0085]

[0086] where ConvUp=ReLU(GN(BlinearUP(Conv 1x1 ())));

[0087] ConvBlk=ReLU(GN(Conv 1x1 ()));

[0088] where f s denotes the dimension projection layer, LN denotes the layer normalization, MCA denotes the cross attention layer, BlinearUP denotes the bilinear up-sampling layer, GN denotes the group normalization layer, and ReLU denotes the activation function.

[0089] The electrocardiogram processing module, as shown in Figure 4 , inputs the ECG electrocardiogram data data ECG to obtain electrocardiogram features , including the following steps:

[0090] S221, QRS wave recognition, process the I lead signal through a band-pass filter of 5-20Hz, then use Ricker wavelet to perform moving wave integration on the filtered signal, and save the square of the integrated signal. The formula is as follows:

[0091] x f (t)=BPF [5,20] (data ECG (t));

[0092] x w (t)=x f (t)*ψ(t);

[0093]

[0094] where BPF [5,20] represents a band-pass filter of 5-20 Hz, ψ(t) represents a Ricker wavelet convolution, and W represents a moving integration window.

[0095] S222, traverse the local maximum of the moving wave integral, when meeting: 1) its distance with the previous local maximum is greater than the "physiological refractory period"; 2) greater than the detection threshold, then consider it as QRS complex, then obtain the QRS wave sequence, the formula is as follows:

[0096] L={t i ∣x w (t i )>x w (t i -1)∧x w (t i )>x w (t i +1)};

[0097] Q={q i ∈L∣x w (q i )>θ∧(q i -q i-1 )>Δ r};

[0098] where Δ r represents the physiological refractory period, θ represents the detection threshold, L is the set of local maximum, and Q represents the QRS wave position index.

[0099] S223, ECG feature extraction. In order to further capture the structural information of the ECG signal, time embedding TE u , space embedding SE v , and position embedding PE are designed, where time embedding TE u represents that the 10-second ECG signal is divided into 10 time periods, and each time period is assigned an independent time embedding, space embedding SE v represents that an independent space embedding is assigned to each lead, and position embedding PE is the relative position information of the signal.

[0100] The ECG signal block obtained in the previous step is input into the convolution layer, then the time period position corresponding to each ECG signal block is mapped to a time embedding vector, the space embedding of each ECG signal block comes from the lead to which it belongs, and the position embedding comes from the relative position in the overall signal, the formula is as follows:

[0101] x t =Conv1x1 (x), x e Q;

[0102]

[0103] where TE u represents time embedding, SE v represents space embedding, and PE represents position embedding.

[0104] Further, by means of an encoder based on a Transformer architecture, the final electrocardio feature is obtained as follows:

[0105] Q = LN(x')w Q , K = LN(x')w K , V = LN(x')w V ;

[0106]

[0107] where d head represents the dimension of each head in multi-head attention, and LN represents layer normalization.

[0108] The blood pressure signal processing module, as shown in Figure 5 , obtains the blood pressure encoding feature , including the following steps:

[0109] S231, experience variable calculation. The blood pressure signal sequence can be calculated by a medical experience formula to obtain the pulse pressure PP and the mean arterial pressure MAP, and the formula is as follows:

[0110] PP = SBP - DBP;

[0111]

[0112] where SBP is the systolic pressure, and DBP is the diastolic pressure.

[0113] S232, feature encoding, to obtain the blood pressure encoding feature , the formula is as follows:

[0114] x = [SBP, DBP, PP, MAP];

[0115]

[0116] where MLP represents a multi-layer perceptron, and ReLU(x) = max(0, x) represents an activation function.

[0117] The time-series-based multi-modal feature fusion module, as shown in Figure 6 , includes the following steps:

[0118] S241, time alignment, for the above ultrasound image features ECG features and blood pressure features They are connected through the time sequence relationship. According to this characteristic, time alignment processing is performed.

[0119] Obtain initial features The formula is as follows:

[0120]

[0121] Among them, ultrasound image features, ECG features Blood pressure features.

[0122] Further, based on the attention mechanism, obtain the query q t The formula is as follows,

[0123]

[0124] Among them, projection layer represents mapping layer.

[0125] Further, by calculating the affinity matrix between the query and the key value, the similarity between the query q t and the key value k t-1 of the previous time frame is captured, and then normalized to finally obtain the aggregation feature F t The formula is as follows:

[0126] F t =v t-1 ·softmax(Sim(k t-1 ,q t ));

[0127] Among them, Sim represents similarity calculation, v t-1 , k t-1 represent the learnable parameters of the previous layer.

[0128] S242, time sequence enhancement, in order to further reduce the influence of noise, time sequence enhancement is adopted to improve the distinguishability of the features, the formula is as follows:

[0129] F t =F t ⊙softmax(Conv 1x1 (Conv 3x3 (F t )));

[0130] Among them, Conv 1x1denotes a 1*1 convolution layer, Conv 3x3 denotes a 3*3 convolution layer, denotes element-wise multiplication, denotes an activation function.

[0131] S243, feature update, after capturing the relationship between each time sequence, obtain time sequence embedding and update the factor h t Update the time sequence embedding, the formula is as follows:

[0132]

[0133]

[0134] where GRU denotes a gated recurrent unit, denotes matrix addition.

[0135] Heart disease risk monitoring module, such as Figure 7 As shown, based on the above steps, the time sequence embedding of each time frame can be obtained All time sequence embeddings are summarized as The steps are as follows:

[0136] S251, state feature acquisition, based on the above formula, the encoder obtains the initial state feature T0, the formula is as follows:

[0137]

[0138] where, denotes the t-th time sequence embedding obtained above, E pos denotes the position information of the time sequence embedding.

[0139] S252, learnable parameter calculation, the formula is as follows:

[0140] x = Linear x (T' l-1 ) ;

[0141] z = Linear z (T' l-1 ) ;

[0142] x'0 = SiLU(Conv1d((x)) ;

[0143] B f / b = Linear B (x' f / b ) ;

[0144] C f / b = Linear C (x' f / b);

[0145] Δ f / b = log(1 + exp(Linear B (x f / b )+ P Δ ));

[0146]

[0147] where Linear represents a linear layer, Conv1d represents a one-dimensional convolution, represents matrix multiplication, and represents an activation function.

[0148] S253, forward and backward propagation encoding, the formula is as follows:

[0149]

[0150] where f represents the forward propagation process, b represents the backward propagation process, and represents element-wise multiplication, represents a state space model.

[0151] S254, aggregate information in both forward and backward directions, and finally obtain the monitoring The formula is as follows:

[0152]

[0153] where, represents the first embedding of the last layer output, which contains more comprehensive information; Norm represents normalization operation; MLP represents multi-layer perception, represents the monitoring result, and represents the risk probability value of the occurrence of heart disease in the input current data.

[0154] The above process designs a fusion modeling method for multi-modal input for the heart disease monitoring task, which can effectively integrate data of multiple modalities such as electrocardiogram, cardiac ultrasound image, physiological parameters, etc. Through multi-modal feature encoding, alignment and fusion mechanism, the model's recognition ability for heart disease is enhanced, and the accuracy and stability of the diagnosis result are improved. It is especially suitable for heterogeneous data scenarios. Through multi-modal fusion modeling, the accuracy and robustness of heart disease risk monitoring are improved.

[0155] S3, model sparsification pruning processing, as Figure 8 shown, the technology includes five steps of layer-by-layer pruning, block sparsification, greedy optimization weight reconstruction, fast approximate Hessian calculation and parallel pruning.

[0156] The layer-by-layer pruning divides the sparsification problem of the overall model into local optimization of each layer, avoiding the complexity brought by global optimization.

[0157] The block sparsification adopts a structured sparse manner, has good hardware compatibility, and is particularly suitable for inference acceleration of CPUs, and is more easily utilized by hardware than non-structured sparsification.

[0158] The greedy optimization weight reconstruction performs a "pruning-reconstruction" operation on each layer, maximally reduces errors by retaining key parameters and reconstructing outputs.

[0159] The fast approximate Hessian calculation method realizes efficient parameter importance estimation, significantly reduces the computational complexity, and avoids the resource overhead brought by the traditional O(N 3 ) complexity;

[0160] The parallel pruning supports multi-core CPU parallel processing through independent calculation of each layer, effectively reduces the memory occupation and time cost in the overall pruning process.

[0161] The above sub-steps are used to achieve different lightweight purposes, and there is synergy and complementarity. Specifically: the layer-by-layer pruning decomposes the overall pruning task into local optimization, reduces the problem complexity, and improves controllability; the structured sparsification has good hardware compatibility, is particularly suitable for CPU acceleration, and is convenient for deployment; the greedy optimization and reconstruction method can maximize the retention of model performance after pruning, avoiding a significant decrease in precision; the fast approximate Hessian provides efficient parameter importance evaluation, significantly reducing the computational overhead; and the parallel pruning strategy further improves the overall pruning efficiency and reduces the memory and time cost.

[0162] These sub-steps are not isolated, but synergistic: the Hessian guides the greedy selection, the structured sparsification facilitates parallel deployment, the layer-by-layer decomposition reduces the risk of error accumulation, and is suitable for medical application scenarios that are sensitive to real-time performance and precision and have limited resources.

[0163] The model sparsification pruning in the embodiment has the following processing steps:

[0164] S301, independently sparsify each linear layer in the model using the layer-by-layer pruning method, construct a pruning objective that minimizes the output reconstruction error, and the optimization form is to minimize the difference between the output generated by the sparsified weight and the original weight under the condition that the input activation matrix remains unchanged. The pruning optimization objective is as follows:

[0165]

[0166] wherein WX represents the original output, represents the output after pruning, a sparse matrix, a sparse structure set satisfying the sparsity requirement.

[0167] S302, in order to improve the inference efficiency on general-purpose processors, the method introduces a column structure block sparsity strategy, which constrains the pruning structure to a block sparse mode with a fixed shape, thereby being compatible with hardware sparse acceleration libraries.

[0168] The goal of block sparsity is to control the sparse structure into a block form friendly to hardware, divide W into multiple blocks, and define a block mask M ij ∈{0,1} indicates whether the i-th row and j-th column block is retained, and the sparse matrix is represented as The sparsity s is limited to the proportion of the total number of block units:∑ i,j M ij ≤(1-s)·N, where N represents the total number of block divisions.

[0169] S303, in the pruning process, a column selection algorithm based on greedy reconstruction is used, which calculates the contribution of each column in the original weight matrix to the output, preferentially retains the column with high contribution, and uses the pseudo-inverse method to reconstruct it, to minimize the error caused by pruning. Specifically:

[0170] For each column W j , calculate its impact on the output, as follows:

[0171]

[0172] where W j is the j-th column of the weight matrix, i.e., all output weights of the j-th input channel, X j is all data in the j-th channel in the input data, x b is the b-th input sample.

[0173] Further, select the k columns with the largest contribution to form a subset S.

[0174] Further, reconstruct the weights corresponding to the k columns, as follows:

[0175]

[0176] where, represents the Moore-Penrose pseudo-inverse of the corresponding column in X.

[0177] Finally, the sparse matrix is obtained, as follows:

[0178] S304、To avoid the resource overhead brought by calculating the high-dimensional Hessian matrix, the method adopts a fast approximation scheme to directly estimate the output perturbation using the activation information obtained by forward propagation, without backward propagation and second-order derivative calculation. Let Using Taylor expansion, we get H represents Hessian.

[0179] By lightweight sparse pruning processing, the model complexity and inference overhead are significantly reduced. After the model is constructed, by introducing a structured sparsity constraint and pruning mechanism, invalid connections, redundant channels or convolution kernels in the model are compressed and simplified, effectively reducing the number of model parameters and computation, and realizing model lightweight on the premise of basically not reducing performance, providing a feasible path for deployment in edge devices and resource-constrained environments.

[0180] S4, after the lightweight processing model, collect the medical images and related physiological index data of the user's heart, including echocardiogram, electrocardiogram, blood pressure and other data, input into the trained heart disease risk monitoring model, use the model to process and analyze the input echocardiogram and related physiological indicators, and monitor the risk of heart disease. When the probability value of heart disease risk exceeds the set value, an early warning is made.

[0181] The risk monitoring model of the scheme is modularized in the overall architecture, facilitating integration and expansion. The proposed method has clear structure and high modularization, facilitating the integration of more data modalities or the embedding of other auxiliary analysis algorithms, and has good engineering practicability and scalability, which is helpful for popularization and application in different platforms and systems, and promotes the transition from "high-performance server centralized processing" to "edge terminal intelligent processing", effectively alleviating the dependence on professional medical resources, and providing an efficient, economical and intelligent solution for chronic disease management and early warning.

[0182] Embodiment two:

[0183] The heart disease risk monitoring system based on multi-source information comprises:

[0184] The data acquisition module is configured to acquire and preprocess the echocardiogram, electrocardiogram data and blood pressure data of the user;

[0185] The feature extraction and fusion module is configured to extract the corresponding echocardiogram features, electrocardiogram features and blood pressure coding features respectively, and obtain updated fusion features through time alignment, fusion and time enhancement processing.

[0186] The risk monitoring module is configured to update the fused features based on the trained heart disease risk monitoring model to obtain a probability value of the user having a heart disease risk.

[0187] During the training, the heart disease risk monitoring model is subjected to sparse pruning processing, specifically, a layer-by-layer structured pruning strategy is adopted to independently perform sparse processing on each linear layer of the model, during which the weight matrix is divided into blocks of fixed shape, and the structure after pruning is constrained to satisfy block sparsity to adapt to hardware acceleration; by calculating the contribution of each column in the original weight matrix to the output, the columns with higher contribution are preferentially retained, and the remaining weights are optimized through reconstruction processing to minimize the pruning error; through layer-by-layer iteration, a block-structured weight matrix with high sparsity is obtained, and the heart disease risk monitoring model is optimized.

[0188] Through the lightweight sparse pruning processing, the model complexity and inference overhead are significantly reduced. After the model is constructed, the structured sparsity constraint and pruning mechanism are introduced to compress and simplify the invalid connections, redundant channels or convolution kernels in the model, effectively reducing the number of model parameters and the amount of calculation, and realizing the lightweight of the model under the premise of basically not reducing the performance. It provides a feasible path for deployment in edge devices and resource-constrained environments.

[0189] The model architecture forms a modular design, which is convenient for integration and extension, and is convenient for subsequent integration of more data modalities or embedding of other auxiliary analysis algorithms. It has good engineering practicability and scalability, helps to promote the application in different platforms and systems, and promotes the transition from "high-performance server centralized processing" to "edge terminal intelligent processing", effectively alleviating the dependence on professional medical resources, and providing an efficient, economical and intelligent solution for chronic disease management and early warning.

[0190] Embodiment three:

[0191] A computer program product includes computer readable instructions that, when executed on an electronic device, cause the electronic device to implement the above-mentioned heart disease risk monitoring method based on multi-source information.

[0192] Embodiment four:

[0193] An electronic device includes at least one processor and a memory connected to the processor, the memory being used to store a computer program; the processor is used to execute the computer program, so that the electronic device can implement the above-mentioned heart disease risk monitoring method based on multi-source information.

[0194] Embodiment five:

[0195] A computer storage medium, the storage medium carries one or more computer programs, when the one or more computer programs are executed by the electronic device, can make the electronic device realize the above-mentioned cardiac disease risk monitoring method based on multi-source information.

[0196] The above only is the preferred embodiment of the present application, and is not used to limit the present application, for the person skilled in the art, the present application can have various changes and changes. Any modification, equivalent replacement, improvement, etc. within the spirit and principle of the present application should be included in the protection scope of the present application.

Claims

1. A method for monitoring the risk of heart disease based on multi-source information, characterized in that: The following steps are involved: Obtain and preprocess the user's cardiac ultrasound image, electrocardiogram (ECG) data, and blood pressure data, extract the corresponding ultrasound image features, ECG features, and blood pressure coding features, respectively, and obtain updated fusion features through time alignment, fusion, and time enhancement. The updated fusion features are based on the trained heart disease risk monitoring model to obtain the probability value of the user's heart disease risk; Among them, the heart disease risk monitoring model undergoes sparse pruning during training. Specifically, a layer-by-layer structured pruning strategy is used to independently sparse each linear layer of the model. During this period, the weight matrix is ​​divided into blocks of fixed shapes, and the pruned structure is constrained to meet block sparsity to adapt to hardware acceleration; by calculating the contribution of each column in the original weight matrix to the output, the columns with higher contributions are retained first, and the remaining weights are optimized through reconstruction to minimize pruning errors; through layer-by-layer iteration, a highly sparse block-structured weight matrix is ​​obtained to optimize the heart disease risk monitoring model.

2. The method for monitoring heart disease risk based on multi-source information according to claim 1, characterized in that: The obtained ultrasound image features, electrocardiogram features and blood pressure encoding features are time-series aligned through temporal relationships to obtain initial features. The initial features obtain queries based on the attention mechanism. By calculating the affinity matrix between the query and the key value, the similarity between the query of the current time frame and the key value of the previous time frame is obtained. The fused features are normalized and the fused features are time-series enhanced and time-series embedded to obtain updated fused features.

3. The method for monitoring heart disease risk based on multi-source information according to claim 1, characterized in that: The heart disease risk monitoring model takes the updated fusion features as input, obtains the initial state features through the encoder, calculates the learnable parameters and forward and backward propagation encoding, and aggregates the information in both the forward and backward propagation directions to obtain the probability value of the user's heart disease risk.

4. The method for monitoring heart disease risk based on multi-source information according to claim 1, wherein: A layer-by-layer structured pruning strategy is used to independently sparsify each linear layer of the model, and a pruning objective is constructed to minimize the output reconstruction error. The optimization form is to minimize the difference between the output produced by the sparsified weights and the original weights while keeping the input activation matrix unchanged. The pruning optimization objective is as follows: Among them, WX represents the original output, represents the output after pruning, is a sparse matrix, Represents a sparse structured set that meets the sparsity requirement.

5. The method for monitoring heart disease risk based on multi-source information according to claim 1, characterized in that: The weight matrix is ​​divided into blocks of fixed shape, and the pruned structure is constrained to meet the block sparsity to adapt to hardware acceleration; specifically: the weight matrix W is divided into multiple blocks, and the block mask M is defined. ij ∈{0,1} indicates whether the block in row i and column j is retained. The sparse matrix is ​​represented as The sparsity rate s is limited to the ratio of the total number of block-level units: ∑ i,j M ij ≤(1-s)·N, where N represents the total number of block partitions.

6. The method for monitoring heart disease risk based on multi-source information according to claim 1, characterized in that: During the sparse pruning process, a column selection algorithm based on greedy reconstruction is used. By calculating the contribution of each column in the original weight matrix to the output, the columns with high contribution are retained first and reconstructed using the pseudo-inverse method to minimize the error caused by pruning. Specifically: For the jth column of the weight matrix W j , calculate its impact on the output, Select the k columns with the largest contribution to form a subset S; reconstruct the weights corresponding to the k columns, represents the Moore-Penrose pseudoinverse of the corresponding column in X, resulting in the final sparse matrix.

7. A heart disease risk monitoring system based on multi-source information, characterized by: include: The data acquisition module is configured to: acquire and pre-process the user's cardiac ultrasound image, electrocardiogram data, and blood pressure data; The feature extraction and fusion module is configured to respectively extract corresponding ultrasound image features, electrocardiogram features, and blood pressure coding features, and obtain updated fusion features through time alignment, fusion, and time enhancement processing; The risk monitoring module is configured to obtain a probability value of the user's risk of developing a heart disease based on the updated fusion features and the trained heart disease risk monitoring model; Among them, the heart disease risk monitoring model undergoes sparse pruning during training. Specifically, a layer-by-layer structured pruning strategy is used to independently sparse each linear layer of the model. During this period, the weight matrix is ​​divided into blocks of fixed shapes, and the pruned structure is constrained to meet block sparsity to adapt to hardware acceleration; by calculating the contribution of each column in the original weight matrix to the output, the columns with higher contributions are retained first, and the remaining weights are optimized through reconstruction to minimize pruning errors; through layer-by-layer iteration, a highly sparse block-structured weight matrix is ​​obtained to optimize the heart disease risk monitoring model.

8. A computer program product, characterized in that The method comprises computer-readable instructions, which, when executed on an electronic device, enable the electronic device to implement the steps in the method for monitoring the risk of heart disease based on multi-source information as claimed in any one of claims 1 to 6.

9. An electronic device, characterized in that: The electronic device comprises at least one processor and a memory connected to the processor, wherein the memory is used to store a computer program; the processor is used to execute the computer program, so that the electronic device can implement the steps in the method for monitoring the risk of heart disease based on multi-source information as claimed in any one of claims 1 to 6.

10. A computer storage medium, characterized in that The storage medium carries one or more computer programs, which, when executed by an electronic device, enable the electronic device to implement the steps in any one of the multi-source information-based heart disease risk monitoring methods according to claims 1-6.

Images