Image-enhanced peripheral intravenous catheterization system based on multi-modal fusion

Abstract

This invention discloses a multimodal fusion-based image enhancement examination peripheral venous puncture vascular pressure monitoring system, belonging to the field of medical image-assisted technology. The system includes: acquiring patient image data, sensor data, and personalized data; generating image feature vectors, sensor feature vectors, and personalized feature vectors based on preprocessed multimodal data; performing feature interaction on multimodal feature lines; fusing representations through Transformer combined with contrastive learning to generate a fused representation vector; using a vascular pressure tolerance prediction model to predict and assess the maximum tolerable pressure, extravasation risk level, and recommended injection flow rate range at the target puncture site under the current injection flow rate; generating injection suggestions and injection alarms for the target puncture site; and automatically adjusting injection parameters. This invention accurately assesses the personalized safe pressure threshold for patients with different vascular conditions, significantly improving the safety and accuracy of peripheral venous puncture.

Description

Technical Field

[0001] This invention relates to the field of medical imaging-assisted technology, and more specifically, to a peripheral venous puncture vessel pressure monitoring system based on multimodal fusion for image enhancement examination. Background Technology

[0002] Contrast-enhanced CT scans are a commonly used clinical imaging technique. Using a high-pressure injector, contrast agents are injected through the patient's intravenous line to obtain standardized and regulated images. Of the approximately 76 million multi-slice spiral CT scans performed globally each year, 50% of patients require contrast-enhanced examinations, with an extravasation rate of 0.6%–1.2%. Medical imaging contrast-enhanced examinations often involve rapid, high-dose, high-concentration, high-osmolarity, and high-viscosity contrast agents injected intravenously under high pressure, with flow rates typically ranging from 2.0 to 6.0 ml / s, or even higher. The high pressure and rapid flow rate during injection can easily lead to contrast agent extravasation or even needle breakage, especially in patients with poor vascular conditions, mismatched catheter material and type with the injection flow rate, repeated punctures, and improper fixation. Due to poor peripheral vascular conditions after surgery and chemotherapy in malignant tumor patients, and the inherent physicochemical properties of contrast agents, adverse events such as vascular extravasation caused by rapid injection of contrast agents under high pressure remain a common clinical challenge that urgently needs to be addressed. Cancer patients undergoing chemotherapy and radiotherapy, critically ill patients, infants, and the elderly often have poor vascular conditions, which not only makes clinical puncture more difficult but also increases the incidence of adverse reactions to contrast agents, seriously affecting patients' lives and health, and in severe cases, leading to death. Therefore, designing an image-enhanced peripheral venous puncture vascular pressure monitoring system to achieve intelligent prediction of vascular pressure resistance and dynamic early warning of extravasation risk has become an urgent problem to be solved. Summary of the Invention

[0003] To address the aforementioned technical issues, this invention proposes a multimodal fusion-based image enhancement examination peripheral venous puncture vascular pressure monitoring system. This system enables intelligent prediction of vascular pressure resistance and dynamic early warning of extravasation risk, achieving precise quantification of contrast agent injection safety thresholds, improving puncture success rates, and preventing vascular complications. This addresses the problems of traditional vascular assessment relying on experience-based judgment and lacking objective quantitative standards.

[0004] The first aspect of the present invention provides a peripheral venous puncture vessel pressure monitoring system based on multimodal fusion for image enhancement examination, comprising: a patient data acquisition module, a personalized data representation module, an image data representation module, a sensor data representation module, a patient information representation fusion module, an intelligent prediction module, a prediction result output module, and a prediction training module;

[0005] The patient data acquisition module collects the patient's image data, sensor data, and personalized data, preprocesses the collected patient data, and inputs it into the characterization module according to preset format requirements;

[0006] The personalized data representation module generates a personalized feature vector of clinical risk based on the preprocessed structured patient personalized data.

[0007] The image data characterization module extracts vascular structure features and hemodynamic features based on the preprocessed image data, fixes the dimension of the characterization, and generates an image feature vector.

[0008] The sensor data characterization module processes sensor data in both the time and frequency domains to extract tissue mechanical response features, fixes the characterization dimensions, and generates sensor feature vectors.

[0009] The patient information representation fusion module interacts with personalized feature vectors, image feature vectors, and sensor feature vectors, and performs representation fusion through Transformer combined with contrastive learning to generate a fused representation vector.

[0010] The intelligent prediction module constructs a vascular pressure resistance prediction model, inputs the fused representation vector into the pre-trained vascular pressure resistance prediction model for prediction, and evaluates the maximum tolerance pressure of the target puncture point and the extravasation risk level and recommended injection flow rate range under the current injection flow rate.

[0011] The prediction result output module encrypts and desensitizes the prediction data, and returns and displays the injection suggestion and injection alarm for the target puncture point according to the preset format and transmission mode.

[0012] The prediction training module trains the vascular pressure resistance prediction model through multi-task learning by comparing and learning the three heterogeneous representations after fusion.

[0013] In this solution, the personalized data representation module generates a personalized feature vector of clinical risk based on the preprocessed structured patient personalized data, specifically as follows:

[0014] Obtain the structured data corresponding to the preprocessed peripheral venous access three-level vascular evaluation table, group the features of each clinical indicator in the three-level vascular evaluation table according to their physiological meaning, create a low-dimensional embedding representation for each group of features, and map the original features to a unified dimensional space.

[0015] The embedded features are imported into a multilayer perceptron, and the hidden layer is used to learn the nonlinear relationship between the clinical indicators in the three-level vascular evaluation table. After feature interaction learning, the feature vector is output, and a self-attention mechanism is introduced to calculate the attention score of each feature vector.

[0016] Several parallel attention heads are used to capture different types of feature vector dependencies. Attention weights are generated through attention scores. The attention weights are used to identify the most important clinical indicators for vascular pressure resistance. The attention weights are then used to weight the importance of each clinical indicator and dynamically adjust the contribution of each clinical indicator to the final prediction.

[0017] The output of the multi-head attention layer is residually concatenated with the feature vector output of the multilayer perceptron to generate an attention-enhanced feature vector. The attention-enhanced feature vector is then mapped to the final feature space to output a fixed-dimensional personalized feature vector.

[0018] In this solution, the image data representation module extracts vascular structure features and hemodynamic features based on the preprocessed image data, fixes the representation dimensions, and generates image feature vectors, specifically as follows:

[0019] Preprocessed ultrasound and infrared dual-modal image data were acquired. A vascular structure feature extraction branch was constructed based on the improved U-Net structure. In the encoder part of the vascular structure feature extraction branch, a deep multi-scale dense block was established. Each dense block contained 4 convolutional layers, and the output of each layer was connected to the input of all subsequent layers.

[0020] Cross-layer residual connections are introduced inside the dense block, the block input is directly added to the block output, and multi-scale convolution is used in parallel to obtain feature maps. The feature maps output by the deep multi-scale dense block are divided into 3 groups, and the SK attention mechanism is introduced to multiply the learned weights with the original feature map channel by channel to represent the importance of vascular structure features.

[0021] In the decoder part of the vascular structure feature extraction branch, high-resolution feature maps of each stage of the encoder part are saved. After unifying the number of channels through one-dimensional convolution, they are concatenated with the corresponding layer features of the decoder part to perform feature compensation, establish lateral connections from shallow to deep layers, use deformable convolution to adapt to changes in vascular morphology, and adopt a gating mechanism to control the intensity of information flow.

[0022] Deep features are used as gating signals to generate spatial attention maps through convolution and sigmoid. The spatial attention maps represent the importance of different vascular regions. The feature contributions are dynamically adjusted based on the importance of vascular regions. Based on the feature contributions, shallow features and deep features are fused to obtain vascular structure features.

[0023] In this solution, the image data representation module extracts vascular structure features and hemodynamic features based on the preprocessed image data, fixes the representation dimensions, and generates image feature vectors, specifically as follows:

[0024] Preprocessed ultrasound and infrared dual-modal image data are acquired, spatiotemporal features are extracted to construct spatiotemporal image blocks, and the spatiotemporal image blocks are input into a 3D convolutional neural network. Hierarchical structure and dilated convolution are used to obtain local blood flow patterns and global hemodynamic states as low-level features and high-level features, respectively.

[0025] The low-level and high-level features are processed by spatial pyramid and temporal pyramid, and cross-dimensional interaction is performed using spatial pyramid attention and temporal pyramid attention. The interactive features are then imported into the maximum flow velocity regression head and turbulence intensity classification head for physical parameter prediction.

[0026] The physical parameter prediction structure is fused with low-level and high-level features to obtain hemodynamic features. A cross-modal attention mechanism is introduced to fuse vascular structure features and hemodynamic features to obtain a fixed-dimensional image feature vector.

[0027] In this solution, the sensor data characterization module processes sensor data jointly in the time and frequency domains to extract tissue mechanical response features, fixes the characterization dimensions, and generates sensor feature vectors, specifically as follows:

[0028] The preprocessed flexible sensor data is acquired, the puncture area is dynamically segmented, and the maximum pressure value, pressure gradient and contact area of ​​the puncture area are collected as dynamic mechanical parameters. The time domain characteristics are obtained based on the dynamic mechanical parameters.

[0029] The preprocessed flexible sensor data is subjected to continuous wavelet transform to collect energy distribution characteristics and phase coupling characteristics as frequency domain features.

[0030] The sensor array is constructed as a graph structure, with nodes representing sensors and margins reflecting spatial adjacency. Feature propagation and neighbor aggregation are performed through a graph attention network to generate mechanical response features and obtain sensor feature vectors of fixed dimensions.

[0031] In this solution, the patient information representation fusion module interacts with personalized feature vectors, image feature vectors, and sensor feature vectors, and performs representation fusion through Transformer combined with contrastive learning to generate a fused representation vector, specifically:

[0032] Personalized feature vectors, image feature vectors, and sensor feature vectors are specifically embedded and projected, and position information is injected to construct a multimodal feature sequence, which is then imported into Transformer encoding and hierarchical interactive attention is set.

[0033] Feature interaction is achieved through intramodal self-attention and cross-modal cross-attention, and the output of each Transformer layer is obtained through gated feature update;

[0034] The training is carried out through a contrastive learning mechanism. The original three-modal complete view, the virtual view with only personalized feature vectors and image feature vectors, and the virtual view with only personalized feature vectors and sensor feature vectors are generated for the same patient sample. Positive sample pairs are constructed based on the different view features of the same patient, and negative sample pairs are constructed based on the random view features of different patients. The NT-Xent loss function is used to perform contrastive learning enhancement based on the positive and negative sample pairs.

[0035] After iterative updates through 6 layers of Transformer, a global fusion representation is obtained. The global fusion representation is then weighted and concatenated with the mean feature vectors of each modality to generate a fusion representation vector.

[0036] In this solution, the intelligent prediction module constructs a vascular pressure resistance prediction model, specifically as follows:

[0037] A vascular pressure resistance prediction model is constructed using a multi-task learning framework, with parallel pressure regression, risk classification, and safety threshold branches set up. The vascular pressure resistance prediction model is then pre-trained through transfer learning.

[0038] The fused representation vector is imported into the vascular pressure resistance prediction model, and the fused representation vector is subjected to feature distillation using a 3-layer bottleneck MLP. Key dimensions with a preset proportion are retained through feature importance analysis.

[0039] Based on the fusion characterization vector after feature distillation and the current injection flow rate at the target puncture point, the risk critical point is identified by the maximum tolerable pressure at the target puncture point and the slope change point of the pressure-flow rate curve. The risk level is then determined using risk standards based on the risk critical point.

[0040] Establish a blood vessel characteristic-safe flow rate mapping table, read the mapping features from the blood vessel characteristic-safe flow rate mapping table according to the current blood vessel status, use the particle swarm optimization algorithm to optimize the flow rate safety threshold according to the mapping features, and output the recommended injection flow rate range.

[0041] Obtain the maximum tolerable pressure, extravasation risk level, and recommended injection flow rate range at the target puncture site under the current injection flow rate, and generate injection suggestions and injection alarms.

[0042] A second aspect of the present invention provides a method for monitoring peripheral venous puncture vessel pressure in image enhancement examination based on multimodal fusion, comprising the following steps:

[0043] Collect patients' imaging data, sensor data, and personalized data, and preprocess the patients' multimodal data;

[0044] Based on the preprocessed multimodal data, image feature vectors, sensor feature vectors, and personalized feature vectors are generated. The multimodal feature lines are subjected to feature interaction, and the representation is fused by Transformer combined with contrastive learning to generate a fused representation vector.

[0045] The fused representation vector is input into the pre-trained vascular pressure resistance prediction model for prediction, and the maximum tolerance pressure of the target puncture point, the extravasation risk level and the recommended injection flow rate range are evaluated under the current injection flow rate.

[0046] Based on the maximum tolerable pressure, extravasation risk level, and recommended injection flow rate range, injection suggestions and injection alarms are generated for the target puncture point, and injection parameters are automatically adjusted.

[0047] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0048] The multimodal fusion-based image enhancement examination peripheral venous puncture vascular pressure monitoring system provided by this invention employs dual-modal simultaneous acquisition technology of ultrasound imaging and infrared vascular detection, combined with a high-sensitivity flexible pressure sensor array, to achieve comprehensive acquisition of vascular structure and functional information. The multimodal intelligent analysis module, through a deep learning architecture, achieves deep fusion of imaging features and mechanical features, constructing a high-precision vascular pressure resistance prediction model capable of accurately assessing personalized safe pressure thresholds for patients with different vascular conditions.

[0049] This invention overcomes the limitations of traditional vascular assessment relying on subjective experience, providing a scientific basis for clinical procedures through objective quantitative indicators, significantly improving the safety and accuracy of peripheral venous puncture. The dynamic risk warning function monitors pressure fluctuations in real time, issuing warnings before extravasation occurs, effectively avoiding contrast agent injection-related complications. It is suitable for patient groups with special vascular conditions, such as elderly patients, children, and patients after chemotherapy, ensuring the safety of examinations for high-risk groups through personalized pressure limit recommendations. Collaborative analysis of multimodal data not only enhances the comprehensiveness of vascular assessment but also strengthens the system's adaptability to different physiological states. Attached Figure Description

[0050] To more clearly illustrate the technical solutions in the embodiments or examples of the present invention, the drawings used in the embodiments or examples will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained according to these drawings without creative effort.

[0051] Figure 1 A framework diagram of an image enhancement examination peripheral venous puncture vessel pressure monitoring system based on multimodal fusion is shown.

[0052] Figure 2 A flowchart for generating the fusion representation vector is shown;

[0053] Figure 3 The flowchart illustrates the process of constructing a vascular pressure resistance prediction model for prediction.

[0054] Figure 4 A flowchart of a method for monitoring vascular pressure in peripheral venous punctures based on multimodal fusion image enhancement is shown. Detailed Implementation

[0055] To better understand the above-mentioned objectives, features, and advantages of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in these embodiments can be combined with each other.

[0056] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and therefore the scope of protection of the invention is not limited to the specific embodiments disclosed below.

[0057] Figure 1 A framework diagram of an image-enhanced examination peripheral venous puncture vessel pressure monitoring system based on multimodal fusion is shown.

[0058] The first embodiment of the present invention provides a peripheral venous puncture vessel pressure monitoring system based on multimodal fusion for image enhancement examination, including: a patient data acquisition module, a personalized data representation module, an image data representation module, a sensor data representation module, a patient information representation fusion module, an intelligent prediction module, a prediction result output module, and a prediction training module;

[0059] The patient data acquisition module collects the patient's image data, sensor data, and personalized data, preprocesses the collected patient data, and inputs it into the characterization module according to preset format requirements;

[0060] The personalized data representation module generates a personalized feature vector of clinical risk based on the preprocessed structured patient personalized data.

[0061] The image data characterization module extracts vascular structure features and hemodynamic features based on the preprocessed image data, fixes the dimension of the characterization, and generates an image feature vector.

[0062] The sensor data characterization module processes sensor data in both the time and frequency domains to extract tissue mechanical response features, fixes the characterization dimensions, and generates sensor feature vectors.

[0063] The patient information representation fusion module interacts with personalized feature vectors, image feature vectors, and sensor feature vectors, and performs representation fusion through Transformer combined with contrastive learning to generate a fused representation vector.

[0064] The intelligent prediction module constructs a vascular pressure resistance prediction model, inputs the fused representation vector into the pre-trained vascular pressure resistance prediction model for prediction, and evaluates the maximum tolerance pressure of the target puncture point and the extravasation risk level and recommended injection flow rate range under the current injection flow rate.

[0065] The prediction result output module encrypts and desensitizes the prediction data, and returns and displays the injection suggestion and injection alarm for the target puncture point according to the preset format and transmission mode.

[0066] The prediction training module trains the vascular pressure resistance prediction model through multi-task learning by comparing and learning the three heterogeneous representations after fusion.

[0067] It should be noted that the wearable probe bracket simultaneously acquires vascular cross-sectional structure data from ultrasound imaging modality and blood flow distribution data from infrared imaging modality. The matrix capacitive sensor array acquires the mechanical response signals of the tissues surrounding the puncture point in real time and records the patient's personalized clinical data, including the peripheral venous access three-level vascular evaluation form for image enhancement examination.

[0068] Categorical variables such as vascular elasticity grading and vascular diameter grading in the three-level vascular evaluation table are encoded using one-hot encoding. Continuous variables such as vascular wall thickness and blood flow velocity are standardized using Z-scores, and missing values ​​are filled using the mean of data from similar patients. The preprocessed structured data corresponding to the three-level peripheral venous access vascular evaluation table is obtained. The features of each clinical indicator in the three-level vascular evaluation table are grouped according to their physiological significance, such as vascular morphology features, hemodynamic features, and patient baseline features. A low-dimensional embedding representation is created for each group of features, mapping the original features to a unified dimensional space.

[0069] The multilayer perceptron (MLP) consists of an input layer, hidden layers, and an output layer. The input layer receives preprocessed structured data, with dimensionality depending on the number of fields in the evaluation table. The hidden layer contains 2-3 fully connected layers, each followed by a ReLU activation function and batch normalization. Embedded features are imported into the MLP, and the hidden layers learn the nonlinear relationships between clinical indicators in the three-level vascular evaluation table. Feature vectors are output after feature interaction learning, with particular attention to the interaction effects of vascular elasticity, diameter, and fundamental features such as patient age and BMI. A self-attention mechanism is introduced, using the feature vectors output by the MLP as the query, key, and value inputs to the attention layer. Attention scores are calculated for each feature vector, and several parallel attention heads capture different types of feature vector dependencies, with each head focusing on different aspects of clinical feature interactions. Attention weights are generated by attention scores. These attention weights are then used to identify the most important clinical indicators for vascular pressure resistance. The attention weights are used to weight the importance of each clinical indicator and dynamically adjust the contribution of each clinical indicator to the final prediction. The output of the multi-head attention layer is residually connected with the feature vector output by the multilayer perceptron to generate an attention-enhanced feature vector. This attention-enhanced feature vector is then mapped to the final feature space to output a fixed-dimensional personalized feature vector.

[0070] It should be noted that the ultrasound and infrared images are enhanced by noise reduction and contrast enhancement. The timestamps and spatial markers of the hardware synchronization device are used to synchronize the ultrasound and infrared images in a time and space. A feature-based registration algorithm is used to achieve pixel-level alignment of the dual-modal images. The region of interest is determined based on the infrared blood vessel detection results, and the corresponding region of the ultrasound image is cropped.

[0071] Preprocessed ultrasound and infrared dual-modal image data were acquired. A vascular structure feature extraction branch was constructed based on an improved U-Net architecture. Deep multi-scale dense blocks were established in the encoder part of this branch, with each block containing four convolutional layers. The output of each layer was connected to the input of all subsequent layers, enabling feature reuse. Cross-layer residual connections were introduced within the dense blocks, directly adding the block input to the block output to alleviate gradient vanishing. Multi-scale convolutions were designed within the deep multi-scale dense blocks, and feature maps were obtained in parallel using multi-scale convolutions. Feature maps at different scales were fused through channel concatenation. Batch normalization and LeakyReLU activation were added after each convolutional layer. The feature maps output from the deep multi-scale dense blocks were divided into three groups. An SK attention mechanism was introduced, and global average pooling was used to generate channel statistical descriptors. Attention weights for each channel were learned through fully connected layers, and softmax was used to normalize the weights. The learned weights were multiplied by the original feature map channel by channel to represent the importance of vascular structure features. The SK attention mechanism dynamically adjusted the weights of each channel, focusing the network on key vascular regions and improving the sensitivity for identifying small vessels.

[0072] In the decoder part of the vascular structure feature extraction branch, high-resolution feature maps from each stage of the encoder part are saved. After unifying the number of channels through one-dimensional convolution, they are concatenated with the corresponding layer features of the decoder part for feature compensation, establishing lateral connections from shallow to deep layers. Deformable convolution is used to adapt to changes in vascular morphology, and a gating mechanism is adopted to control the intensity of information flow to achieve detail restoration. Deep features are used as gating signals, and spatial attention maps are generated through convolution and sigmoid. Feature maps from different decoding stages are input into the attention gate. The spatial attention map represents the importance of different vascular regions, and the feature contribution is dynamically adjusted based on the importance of vascular regions. Based on the feature contribution, shallow features and deep features are fused to obtain vascular structure features. For example, 80% of high-level features and 20% of low-level details are retained during the final fusion.

[0073] It should be noted that, after acquiring preprocessed ultrasound and infrared dual-modal image data, the ultrasound and infrared sequences are segmented into cardiac cycle phases. Spatiotemporal features are extracted based on the vessel center to construct spatiotemporal image blocks. These spatiotemporal image blocks are then input into a 3D convolutional neural network. A hierarchical structure and dilated convolution are used to obtain local blood flow patterns and global hemodynamic states as low-level and high-level features, respectively. Dilated convolution is introduced to expand the receptive field while maintaining resolution. Low-level features capture local blood flow patterns and can be used to extract vessel wall motion trajectories, while high-level features encode the global hemodynamic state and can be used to identify abnormal turbulent regions. Enhanced by introducing a PSA module, spatial pyramids capture blood flow heterogeneity in different vascular regions, while temporal pyramids model the dynamic characteristics of each phase of the cardiac cycle. The low-level and high-level features are processed using spatial and temporal pyramids, and cross-dimensional interaction is achieved through spatial and temporal pyramid attention. The spatial attention map is broadcast along the time axis, and the temporal weights are expanded along the spatial dimension. The interactive features are imported into the maximum velocity regression head and turbulence intensity classification head for physical parameter prediction. The physical parameter prediction structure is then fused with low-level and high-level features to obtain hemodynamic features. Explicit quantification parameters and implicit deep features complement each other to improve the model's robustness under low-quality images. A cross-modal attention mechanism is introduced to fuse vascular structural features and hemodynamic features to obtain a fixed-dimensional image feature vector.

[0074] It should be noted that a hardware synchronization triggering mechanism is used to ensure the time alignment of the data from the capacitive sensor array, and baseline correction is performed on each channel to eliminate individual sensor offset differences. Preprocessed flexible sensor data is acquired, the puncture area is dynamically segmented, and the maximum pressure value, pressure gradient, and contact area of ​​the puncture area are collected as dynamic mechanical parameters. Based on these dynamic mechanical parameters, time-domain characteristics are obtained, such as pulse wave propagation time, waveform steepness, signal entropy, and autocorrelation attenuation coefficient.

[0075] The preprocessed flexible sensor data undergoes continuous wavelet transform, with the Morlet function selected as the mother wavelet, corresponding to the 0.1-50Hz physiological frequency band. Five characteristic frequency bands are extracted: ultra-low frequency (0.1-0.5Hz), vascular pulsation frequency (0.6-2Hz), muscle tone frequency (2-5Hz), and high-frequency noise (>5Hz). Energy distribution characteristics and phase coupling characteristics are collected as frequency domain features. For example, the proportion of energy in the vascular pulsation frequency band to the total energy is calculated to obtain the energy distribution ratio; the coherence coefficient of sensors at different locations in the characteristic frequency bands is analyzed to obtain the frequency domain coupling degree. The sensor array is constructed as a graph structure, with nodes representing sensors and margins reflecting spatial adjacency relationships. Feature propagation and neighbor aggregation are performed through a graph attention network to generate mechanical response features, obtaining sensor feature vectors of fixed dimensions. Each layer of the graph attention network uses four attention heads to learn different mechanical interaction patterns, and a margin gating mechanism is used to control the information propagation intensity.

[0076] Figure 2 A flowchart for generating the fusion representation vector is shown;

[0077] According to an embodiment of the present invention, the patient information representation fusion module performs feature interaction between personalized feature vectors, image feature vectors, and sensor feature vectors, and performs representation fusion through Transformer combined with contrastive learning to generate a fused representation vector, specifically as follows:

[0078] S202 involves specifically embedding and projecting personalized feature vectors, image feature vectors, and sensor feature vectors, injecting location information, constructing a multimodal feature sequence, importing it into Transformer encoding, and setting hierarchical interactive attention.

[0079] S204 uses intramodal self-attention and cross-modal cross-attention for feature interaction, and obtains the output of each Transformer layer through gated feature update;

[0080] S206 is trained through a contrastive learning mechanism to generate the original three-modal complete view, the virtual view with only personalized feature vectors and image feature vectors, and the virtual view with only personalized feature vectors and sensor feature vectors for the same patient sample. Positive sample pairs are constructed based on the different view features of the same patient, and negative sample pairs are constructed based on the random view features of different patients. The NT-Xent loss function is used to perform contrastive learning enhancement based on the positive and negative sample pairs.

[0081] S208. After iterative updates through 6 layers of Transformer, a global fusion representation is obtained. The global fusion representation is then weighted and concatenated with the mean feature vectors of each modality to generate a fusion representation vector.

[0082] It should be noted that the three feature vectors are mapped to a unified feature space through independent linear projection layers. Layer normalization is then applied to the projected features to eliminate distribution differences between modalities, generating spatial location codes for image and sensor features. Personalized features are marked with all-one-padded location markers. Intramodal self-attention restricts interactions between modalities to internal interactions only through attention masks, strengthening the consistency of features within each modality. In cross-modal attention, the query comes from the target modality, and the key-value pairs come from the other two modalities. Each Transformer layer performs a residual connection between the intramodal self-attention output, the cross-modal attention output, and the input as its output. Adaptive feature fusion is achieved through gated cross-modal attention. The NT-Xent loss function is used for contrastive learning enhancement based on positive and negative sample pairs. The virtual perspective constructed by contrastive learning enhancement allows the model to work reliably even when modalities are missing. The loss function is expressed as:

[0083] ,

[0084] in The anchor sample represents the feature vector, while the target sample needs to be optimized. The feature vector representing a positive sample. The feature vector representing the negative sample. This represents the temperature coefficient, used to adjust the sharpness of the probability distribution. The similarity function is usually cosine similarity.

[0085] Figure 3 The flowchart for constructing a vascular pressure resistance prediction model is shown.

[0086] According to an embodiment of the present invention, the intelligent prediction module constructs a vascular pressure resistance prediction model, specifically as follows:

[0087] S302 constructs a vascular pressure resistance prediction model through a multi-task learning framework, sets up parallel pressure regression branches, risk classification branches and safety threshold branches, and pre-trains the vascular pressure resistance prediction model through transfer learning;

[0088] S304, The fused representation vector is imported into the vascular pressure resistance prediction model, and the fused representation vector is subjected to feature distillation using a 3-layer bottleneck MLP. Key dimensions with a preset proportion are retained through feature importance analysis.

[0089] S306, based on the fusion characterization vector after feature distillation and the current injection flow rate of the target puncture point, the risk critical point is identified by the maximum tolerance pressure of the target puncture point and the slope change point of the pressure-flow rate curve is used to determine the risk level according to the risk critical point;

[0090] S308, Establish a blood vessel characteristic-safe flow rate mapping table, read the mapping features from the blood vessel characteristic-safe flow rate mapping table according to the current blood vessel status, use the particle swarm algorithm to optimize the flow rate safety threshold according to the mapping features, and output the recommended injection flow rate range;

[0091] S310: Obtain the maximum tolerable pressure, extravasation risk level, and recommended injection flow rate range at the target puncture site under the current injection flow rate, and generate injection suggestions and injection alarms.

[0092] It should be noted that a vascular pressure resistance prediction model is constructed using a multi-task learning framework. The pressure regression branch predicts the maximum tolerable pressure using the Huber loss function; the risk classification branch outputs the extravasation risk level using a weighted cross-entropy loss function; and the safety threshold head branch outputs the recommended injection flow rate range. The vascular pressure resistance prediction model is pre-trained using a prediction training module through transfer learning. The fusion representation vector extraction part of the vascular pressure resistance prediction model, pre-trained on 100,000 cases of vascular imaging data, is used to pre-train the basic prediction branch on 50,000 cases of in vitro vascular model pressure test data. Physiological constraints such as vascular rupture pressure are added during the pre-training process.

[0093] A quantitative relationship between vascular physiological parameters and safe injection flow rate was established, including vascular elastic modulus, blood flow resistance index, vessel wall stress distribution, vessel diameter, and viscoelastic attenuation coefficient. The vascular elastic modulus reflects the mechanical properties of the vessel wall and is negatively correlated with the maximum permissible flow rate. The blood flow resistance index, calculated based on Doppler ultrasound, influences the recommended acceleration threshold. The vessel wall stress distribution is modeled using finite element analysis, determining the pressure equalization coefficient. The vessel diameter is directly related to the critical flow rate, and the viscoelastic attenuation coefficient, extracted from time-frequency domain sensor signals, affects the dynamic adjustment sensitivity. Real-time parameters are obtained based on the current vascular state. The closest combination of vascular characteristics is matched in a mapping table. Bilinear interpolation is used for discrete data to calculate the initial value of the continuous safe flow rate. A particle swarm optimization algorithm optimized by simulated annealing is used to find the maximum safe flow rate. The maximum safe flow rate is solved under the conditions that the pressure is less than the maximum tolerable pressure and the risk of extravasation is less than a preset risk threshold.

[0094] The particle swarm is initialized based on the initial safe flow rate and random perturbation. The particle swarm algorithm is improved using simulated annealing to set the probability of accepting inferior solutions. Particle fitness is evaluated during iterative optimization. The calculation formula is:

[0095] ,

[0096] in Indicates the maximum tolerable pressure of blood vessels. Indicates the flow rate Real-time vascular pressure values, Indicates flow rate The normalized value, , Indicates the weighting coefficient;

[0097] Obtaining individual optimality and global optimal Update particle velocity:

[0098] ,

[0099] in Indicates the first The particles in the next iteration Update speed at that time Indicates the first The particles in the iteration number The velocity vector at time, Indicates inertia weight, , Indicates the acceleration coefficient. , Represents a random number. Indicates the first The particles in the iteration number The current position at that time.

[0100] Every 10 iterations, the temperature is decreased, accepting some suboptimal solutions to escape local optima. The maximum safe flow rate is obtained when the maximum number of iterations is reached or the fitness rate of change is less than a preset threshold. The optimal solution is then corrected based on real-time risk prediction results. The safe flow rate threshold is determined by this process. Represented as:

[0101] ,

[0102] in This represents the initial baseline flow rate obtained from the vascular characteristics-safe flow rate mapping table. This indicates a dynamic adjustment factor based on the real-time risk level.

[0103] A recommended injection flow rate range is generated based on the aforementioned flow rate safety threshold. :

[0104] ,

[0105] in These represent the minimum and maximum flow velocities allowed for this type of blood vessel in the mapping table, respectively.

[0106] Based on the maximum tolerable pressure, extravasation risk level, and recommended injection flow rate range, injection suggestions and alarms are generated for the target puncture point. A patient-specific pressure-flow rate database is established, and personalized prediction model parameters are updated according to preset time steps. Inverse feature analysis is performed on cases of prediction errors to achieve adaptive optimization. Preferably, in the prediction result output module, real-time pressure distribution is overlaid on ultrasound images, displaying a risk prediction curve for the next 3 minutes. Injection protocol recommendations are generated based on different color distributions of the pressure heatmap; for example, the green zone corresponds to the recommended safe flow rate, the yellow zone corresponds to the critical value requiring careful operation, and the red zone corresponds to the contraindication range. In addition, for special conditions such as arteriosclerosis after chemotherapy in cancer patients and increased vascular fragility in diabetic patients, the risk judgment threshold is automatically adjusted.

[0107] Figure 4 A flowchart of a method for monitoring vascular pressure in peripheral venous punctures based on multimodal fusion image enhancement is shown.

[0108] The second embodiment of the present invention provides a method for monitoring peripheral venous puncture vessel pressure in image enhancement examination based on multimodal fusion, comprising the following steps:

[0109] S402 collects the patient's image data, sensor data, and personalized data, and preprocesses the patient's multimodal data;

[0110] S404 generates image feature vectors, sensor feature vectors, and personalized feature vectors based on preprocessed multimodal data. It performs feature interaction on multimodal feature lines and performs representation fusion through Transformer combined with contrastive learning to generate a fused representation vector.

[0111] S406, The fused representation vector is input into the pre-trained vascular pressure resistance prediction model for prediction, and the maximum tolerance pressure of the target puncture point, the extravasation risk level and the recommended injection flow rate range are evaluated under the current injection flow rate.

[0112] S408 generates injection suggestions and injection alarms for the target puncture point based on the maximum tolerable pressure, extravasation risk level, and recommended injection flow rate range, and automatically adjusts injection parameters.

[0113] A third embodiment of the present invention provides a computer-readable storage medium, which includes a program for a method of monitoring vascular pressure in peripheral venous puncture based on multimodal fusion image enhancement examination. When the program for monitoring vascular pressure in peripheral venous puncture based on multimodal fusion image enhancement examination is executed by a processor, it implements the steps of the method for monitoring vascular pressure in peripheral venous puncture based on multimodal fusion image enhancement examination.

[0114] In the several embodiments provided in this application, it should be understood that the disclosed methods and systems can be implemented in other ways. The system embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, and can be electrical, mechanical, or other forms. Furthermore, in the various embodiments of the present invention, all functional units can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0115] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0116] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A peripheral venous puncture vessel pressure monitoring system based on multimodal fusion for image enhancement examination, characterized in that, include: The system includes a patient data acquisition module, a personalized data representation module, an image data representation module, a sensor data representation module, a patient information representation fusion module, an intelligent prediction module, a prediction result output module, and a prediction training module. The patient data acquisition module collects the patient's image data, sensor data, and personalized data, preprocesses the collected patient data, and inputs it into the characterization module according to preset format requirements; The personalized data representation module generates a personalized feature vector of clinical risk based on the preprocessed structured patient personalized data. The image data characterization module extracts vascular structure features and hemodynamic features based on the preprocessed image data, fixes the dimension of the characterization, and generates an image feature vector. The sensor data characterization module processes sensor data in both the time and frequency domains to extract tissue mechanical response features, fixes the characterization dimensions, and generates sensor feature vectors. The patient information representation fusion module interacts with personalized feature vectors, image feature vectors, and sensor feature vectors, and performs representation fusion through Transformer combined with contrastive learning to generate a fused representation vector. The intelligent prediction module constructs a vascular pressure resistance prediction model, inputs the fused representation vector into the pre-trained vascular pressure resistance prediction model for prediction, and evaluates the maximum tolerance pressure of the target puncture point and the extravasation risk level and recommended injection flow rate range under the current injection flow rate. The prediction result output module encrypts and desensitizes the prediction data, and returns and displays the injection suggestion and injection alarm for the target puncture point according to the preset format and transmission mode. The prediction training module trains the vascular pressure resistance prediction model through multi-task learning by comparing and learning the three heterogeneous representations after fusion.

2. The image enhancement examination peripheral venous puncture vessel pressure monitoring system based on multimodal fusion according to claim 1, characterized in that, The personalized data representation module generates a personalized feature vector of clinical risk based on the preprocessed structured patient personalized data, specifically: Obtain the structured data corresponding to the preprocessed peripheral venous access three-level vascular evaluation table, group the features of each clinical indicator in the three-level vascular evaluation table according to their physiological meaning, create a low-dimensional embedding representation for each group of features, and map the original features to a unified dimensional space. The embedded features are imported into a multilayer perceptron, and the hidden layer is used to learn the nonlinear relationship between the clinical indicators in the three-level vascular evaluation table. After feature interaction learning, the feature vector is output, and a self-attention mechanism is introduced to calculate the attention score of each feature vector. Several parallel attention heads are used to capture different types of feature vector dependencies. Attention weights are generated through attention scores. The attention weights are used to identify the most important clinical indicators for vascular pressure resistance. The attention weights are then used to weight the importance of each clinical indicator and dynamically adjust the contribution of each clinical indicator to the final prediction. The output of the multi-head attention layer is residually concatenated with the feature vector output of the multilayer perceptron to generate an attention-enhanced feature vector. The attention-enhanced feature vector is then mapped to the final feature space to output a fixed-dimensional personalized feature vector.

3. The image enhancement examination peripheral venous puncture vessel pressure monitoring system based on multimodal fusion according to claim 1, characterized in that, The image data representation module extracts vascular structural features and hemodynamic features from the preprocessed image data, fixes the representation dimensions, and generates image feature vectors, specifically: Preprocessed ultrasound and infrared dual-modal image data were acquired. A vascular structure feature extraction branch was constructed based on the improved U-Net structure. In the encoder part of the vascular structure feature extraction branch, a deep multi-scale dense block was established. Each dense block contained 4 convolutional layers, and the output of each layer was connected to the input of all subsequent layers. Cross-layer residual connections are introduced inside the dense block, the block input is directly added to the block output, and multi-scale convolution is used in parallel to obtain feature maps. The feature maps output by the deep multi-scale dense block are divided into 3 groups, and the SK attention mechanism is introduced to multiply the learned weights with the original feature map channel by channel to characterize the importance of vascular structure features. In the decoder part of the vascular structure feature extraction branch, high-resolution feature maps of each stage of the encoder part are saved. After unifying the number of channels through one-dimensional convolution, they are concatenated with the corresponding layer features of the decoder part to perform feature compensation, establish lateral connections from shallow to deep layers, use deformable convolution to adapt to changes in vascular morphology, and adopt a gating mechanism to control the intensity of information flow. Deep features are used as gating signals to generate spatial attention maps through convolution and sigmoid. The spatial attention maps represent the importance of different vascular regions. The feature contributions are dynamically adjusted based on the importance of vascular regions. Based on the feature contributions, shallow features and deep features are fused to obtain vascular structure features.

4. The image enhancement examination peripheral venous puncture vessel pressure monitoring system based on multimodal fusion according to claim 3, characterized in that, The image data representation module extracts vascular structural features and hemodynamic features from the preprocessed image data, fixes the representation dimensions, and generates image feature vectors, specifically: Preprocessed ultrasound and infrared dual-modal image data are acquired, spatiotemporal features are extracted to construct spatiotemporal image blocks, and the spatiotemporal image blocks are input into a 3D convolutional neural network. Hierarchical structure and dilated convolution are used to obtain local blood flow patterns and global hemodynamic states as low-level features and high-level features, respectively. The low-level and high-level features are processed by spatial pyramid and temporal pyramid, and cross-dimensional interaction is performed using spatial pyramid attention and temporal pyramid attention. The interactive features are then imported into the maximum flow velocity regression head and turbulence intensity classification head for physical parameter prediction. The physical parameter prediction structure is fused with low-level and high-level features to obtain hemodynamic features. A cross-modal attention mechanism is introduced to fuse vascular structure features and hemodynamic features to obtain a fixed-dimensional image feature vector.

5. The image enhancement examination peripheral venous puncture vessel pressure monitoring system based on multimodal fusion according to claim 1, characterized in that, The sensor data characterization module processes sensor data jointly in the time and frequency domains, extracts tissue mechanical response features, fixes the characterization dimensions, and generates sensor feature vectors, specifically: The preprocessed flexible sensor data is acquired, the puncture area is dynamically segmented, and the maximum pressure value, pressure gradient and contact area of ​​the puncture area are collected as dynamic mechanical parameters. The time domain characteristics are obtained based on the dynamic mechanical parameters. The preprocessed flexible sensor data is subjected to continuous wavelet transform to collect energy distribution characteristics and phase coupling characteristics as frequency domain features. The sensor array is constructed as a graph structure, with nodes representing sensors and margins reflecting spatial adjacency. Feature propagation and neighbor aggregation are performed through a graph attention network to generate mechanical response features and obtain sensor feature vectors of fixed dimensions.

6. The image enhancement examination peripheral venous puncture vessel pressure monitoring system based on multimodal fusion according to claim 1, characterized in that, The patient information representation fusion module interacts with personalized feature vectors, image feature vectors, and sensor feature vectors, and performs representation fusion through Transformer combined with contrastive learning to generate a fused representation vector, specifically: Personalized feature vectors, image feature vectors, and sensor feature vectors are specifically embedded and projected, and position information is injected to construct a multimodal feature sequence, which is then imported into Transformer encoding and hierarchical interactive attention is set. Feature interaction is achieved through intramodal self-attention and cross-modal cross-attention, and the output of each Transformer layer is obtained through gated feature update; The training is carried out through a contrastive learning mechanism. The original three-modal complete view, the virtual view with only personalized feature vectors and image feature vectors, and the virtual view with only personalized feature vectors and sensor feature vectors are generated for the same patient sample. Positive sample pairs are constructed based on the different view features of the same patient, and negative sample pairs are constructed based on the random view features of different patients. The NT-Xent loss function is used to perform contrastive learning enhancement based on the positive and negative sample pairs. After iterative updates through 6 layers of Transformer, a global fusion representation is obtained. The global fusion representation is then weighted and concatenated with the mean feature vectors of each modality to generate a fusion representation vector.

7. The image enhancement examination peripheral venous puncture vessel pressure monitoring system based on multimodal fusion according to claim 1, characterized in that, The intelligent prediction module constructs a vascular pressure resistance prediction model, specifically as follows: A vascular pressure resistance prediction model is constructed using a multi-task learning framework, with parallel pressure regression, risk classification, and safety threshold branches set up. The vascular pressure resistance prediction model is then pre-trained through transfer learning. The fused representation vector is imported into the vascular pressure resistance prediction model, and the fused representation vector is subjected to feature distillation using a 3-layer bottleneck MLP. Key dimensions with a preset proportion are retained through feature importance analysis. Based on the fusion characterization vector after feature distillation and the current injection flow rate at the target puncture point, the risk critical point is identified by the maximum tolerable pressure at the target puncture point and the slope change point of the pressure-flow rate curve. The risk level is then determined using risk standards based on the risk critical point. Establish a blood vessel characteristic-safe flow rate mapping table, read the mapping features from the blood vessel characteristic-safe flow rate mapping table according to the current blood vessel status, use the particle swarm optimization algorithm to optimize the flow rate safety threshold according to the mapping features, and output the recommended injection flow rate range. Obtain the maximum tolerable pressure, extravasation risk level, and recommended injection flow rate range at the target puncture site under the current injection flow rate, and generate injection suggestions and injection alarms.

Images