Image-enhanced peripheral vein puncture blood vessel pressure monitoring system based on multi-modal fusion

Abstract

The invention discloses a peripheral vein puncture blood vessel pressure monitoring system for image enhancement examination based on multi-modal fusion, and relates to the technical field of medical image assistion.The peripheral vein puncture blood vessel pressure monitoring system comprises the steps that image data, sensor data and personalized data of a patient are collected; generating an image feature vector, a sensor feature vector and a personalized feature vector based on the preprocessed multi-modal data, carrying out feature interaction on a multi-modal feature line, carrying out representation fusion by combining Transform with contrast learning, and generating a fusion representation vector; the maximum tolerance pressure, the extravasation risk level and the recommended injection flow rate range of the target puncture point under the current injection flow rate are predicted and evaluated in the blood vessel pressure resistance prediction model, injection suggestions and injection alarms of the target puncture point are generated, and injection parameters are automatically adjusted. According to the method, the personalized safety pressure threshold values of patients with different blood vessel conditions are accurately evaluated, and the safety and accuracy of peripheral vein puncture are remarkably improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical image auxiliary, more particularly, to an image enhancement examination peripheral vein puncture blood vessel pressure monitoring system based on multi-modal fusion. BACKGROUND

[0002] CT enhancement scanning is a commonly used imaging examination method in clinic. The use of high-pressure injectors through the patient's vein channel for contrast agent injection can obtain standard and normative image images. Among the 76 million multi-slice spiral CT examinations performed worldwide each year, 50% of the patients need to do enhancement examination, and the extravasation rate is 0.6% to 1.2%. Medical image enhancement examination is mostly through high-pressure rapid bolus injection of large-dose high-concentration high-osmotic pressure high-viscosity contrast agent, and the examination flow rate is usually 2.0 to 6.0 ml / s, or even higher. The pressure is large and the instantaneous flow rate is fast during injection. If the patient's blood vessel condition is poor, the type and model of the indwelling needle do not match the injection flow rate, repeated puncture, and improper fixation, it is easy to cause contrast agent extravasation and even indwelling needle rupture. Due to the poor peripheral blood vessel condition of patients with malignant tumors after surgery and chemotherapy and the physical and chemical properties of the contrast agent itself, the extravascular extravasation caused by rapid injection of the contrast agent under high pressure is still a common and urgent problem in clinic. The blood vessel condition of tumor patients after chemotherapy and radiotherapy, critically ill patients and infants, and the elderly is poor, which not only brings difficulty to clinical puncture, but also increases the adverse reaction events of the contrast agent, seriously affecting the life and health of the patients, and even leading to death. Therefore, it is urgent to design an image enhancement examination peripheral vein puncture blood vessel pressure monitoring system to realize intelligent prediction of blood vessel pressure resistance and dynamic early warning of extravasation risk, so as to achieve the effects of accurately quantifying the safety threshold of contrast agent injection, improving the puncture success rate, and preventing blood vessel complications, so as to solve the problem that the traditional blood vessel evaluation relies on experience and lacks objective quantitative standards. SUMMARY

[0003] In order to solve the above technical problems, the present application provides an image enhancement examination peripheral vein puncture blood vessel pressure monitoring system based on multi-modal fusion, which realizes intelligent prediction of blood vessel pressure resistance and dynamic early warning of extravasation risk, accurately quantifies the safety threshold of contrast agent injection, improves the puncture success rate, and prevents blood vessel complications, so as to solve the problem that the traditional blood vessel evaluation relies on experience and lacks objective quantitative standards.

[0004] The first aspect of the present application provides an image enhancement examination peripheral vein puncture blood vessel pressure monitoring system based on multi-modal fusion, which comprises: a patient data acquisition module, a personalized data representation module, an image data representation module, a sensor data representation module, a patient information representation fusion module, an intelligent prediction module, a prediction result output module and a prediction training module. The patient data acquisition module acquires image data, sensor data and personalized data of the patient, pre-processes the acquired patient data, and inputs the pre-processed patient data into the representation module according to the preset format requirements. The personalized data characterization module generates a personalized feature vector of clinical risk based on the pre-processed structured patient personalized data; The image data characterization module extracts vascular structural features and hemodynamic features based on the preprocessed image data, fixes the representation dimensions, and generates an image feature vector; The sensor data characterization module processes sensor data in the time domain and frequency domain jointly, extracts tissue mechanical response characteristics, fixes the dimension of the characterization, and generates a sensor feature vector; The patient information representation fusion module performs feature interaction on the personalized feature vector, the image feature vector, and the sensor feature vector, and performs representation fusion through Transformer combined with contrast learning to generate a fused representation vector; The intelligent prediction module constructs a vascular pressure tolerance prediction model, inputs the fused representation vector into the pre-trained vascular pressure tolerance prediction model for prediction, and evaluates the maximum tolerable pressure and extravasation risk level of the target puncture point at the current injection flow rate, as well as the recommended injection flow rate range; The prediction result output module encrypts and desensitizes the prediction data, and returns and displays the injection suggestion and injection alarm of the target puncture point according to the preset format and transmission mode; The prediction training module trains the vascular pressure resistance prediction model through multi-task learning of the three types of heterogeneous representations after comparative learning fusion.

[0005] In this solution, the personalized data characterization module generates a personalized feature vector of clinical risk based on the pre-processed structured patient personalized data, specifically: Obtaining structured data corresponding to the preprocessed three-level peripheral venous access vascular evaluation table, grouping the features of each clinical indicator in the three-level vascular evaluation table according to physiological significance, creating a low-dimensional embedding representation for each group of features, and mapping the original features to a unified dimensional space; The embedded features are imported into a multi-layer perceptron, and the hidden layer is used to learn the nonlinear relationship between the clinical indicators in the three-level vascular evaluation table. After interactive learning of features, the feature vector is output. The self-attention mechanism is introduced to calculate the attention score of each feature vector. Several parallel attention heads are used to capture different types of feature vector dependencies. Attention weights are generated from attention scores. These weights are used to identify the most important clinical indicators for vascular pressure tolerance. The attention weights are then used to weight the importance of each clinical indicator and dynamically adjust the contribution of each clinical indicator's features to the final prediction. The output of the multi-head attention layer is residually connected with the feature vector output by the multi-layer perceptron to generate an attention-enhanced feature vector, which is then mapped to the final feature space to output a personalized feature vector of fixed dimension.

[0006] In this solution, the image data representation module extracts vascular structural features and hemodynamic features based on the preprocessed image data, fixes the representation dimensions, and generates an image feature vector, specifically: Preprocessed ultrasound and infrared dual-modality imaging data are obtained. A vascular structure feature extraction branch is constructed based on an improved U-Net structure. Deep multi-scale dense blocks are established in the encoder part of the vascular structure feature extraction branch. Each dense block contains four convolutional layers, and the output of each layer is connected to the input of all subsequent layers. A cross-layer residual connection is introduced within the dense block, the block input is directly added to the block output, and multi-scale convolution is used in parallel to obtain feature maps. The feature maps output by the deep multi-scale dense block are divided into three groups. The SK attention mechanism is introduced to multiply the learned weights with the original feature maps channel by channel to represent the importance of vascular structure features. In the decoder part of the vascular structure feature extraction branch, the high-resolution feature maps of each stage of the encoder are saved. After unifying the number of channels through one-dimensional convolution, they are spliced ​​with the features of the corresponding layers of the decoder to perform feature compensation. Horizontal connections are established from shallow to deep layers. Deformable convolution is used to adapt to changes in vascular morphology, and a gating mechanism is used to control the intensity of information flow. Deep features are used as gating signals, and a spatial attention map is generated through convolution and sigmoid. The importance of different vascular regions is represented according to the spatial attention map. The feature contribution is dynamically adjusted according to the importance of the vascular regions. The shallow features and deep features are fused according to the feature contribution to obtain the vascular structure characteristics.

[0007] In this solution, the image data representation module extracts vascular structural features and hemodynamic features based on the preprocessed image data, fixes the representation dimensions, and generates an image feature vector, specifically: Obtain preprocessed ultrasound and infrared dual-modality image data, extract spatiotemporal features to construct spatiotemporal image blocks, input the spatiotemporal image blocks into a 3D convolutional neural network, and use a hierarchical structure and dilated convolution to obtain local blood flow patterns and global hemodynamic states as low-level features and high-level features, respectively; Performing spatial pyramid and temporal pyramid processing on the low-level features and high-level features, using spatial pyramid attention and temporal pyramid attention for cross-dimensional interaction, and importing the interactive features into the maximum velocity regression head and the turbulence intensity classification head for physical parameter prediction; The physical parameter prediction structure is fused with low-level features and high-level features to obtain hemodynamic features. A cross-modal attention mechanism is introduced to fuse vascular structure features and hemodynamic features to obtain a fixed-dimensional image feature vector.

[0008] In this solution, the sensor data characterization module processes sensor data in the time domain and frequency domain, extracts tissue mechanical response characteristics, fixes the dimension of the characterization, and generates a sensor feature vector, specifically: Acquiring preprocessed flexible sensor data, dynamically segmenting the puncture area, collecting the maximum pressure value, pressure gradient, and contact area of ​​the puncture area as dynamic mechanical parameters, and acquiring time domain features based on the dynamic mechanical parameters; The pre-processed flexible sensor data is subjected to continuous wavelet transform to collect energy distribution characteristics and phase coupling characteristics as frequency domain features; The sensor array is constructed as a graph structure, with nodes as sensors and margins reflecting spatial adjacency. Feature propagation and neighbor aggregation are performed through a graph attention network to generate mechanical response features and obtain a fixed-dimensional sensor feature vector.

[0009] In this solution, the patient information representation fusion module performs feature interaction on the personalized feature vector, the image feature vector, and the sensor feature vector, and performs representation fusion through Transformer combined with contrastive learning to generate a fused representation vector. Specifically: The personalized feature vector, image feature vector, and sensor feature vector are specifically embedded and projected, and position information is injected into them to construct a multimodal feature sequence, which is imported into the Transformer encoding and hierarchical interactive attention is set. Feature interaction is performed through intra-modal self-attention and cross-modal cross-attention, and the output of each Transformer layer is obtained through gated feature updates; Through contrastive learning, training was performed to generate the original trimodal full view, a virtual view consisting of only personalized feature vectors and imaging feature vectors, and a virtual view consisting of only personalized feature vectors and sensor feature vectors for the same patient sample. Positive sample pairs were constructed based on the different viewpoint features of the same patient, while negative sample pairs were constructed based on the randomly paired viewpoint features of different patients. The NT-Xent loss function was used to perform contrastive learning enhancement based on the positive and negative sample pairs. After iterative updating through the 6-layer Transformer, a global fusion representation is obtained, and the global fusion representation is weightedly concatenated with the mean feature vector of each modality to generate a fusion representation vector.

[0010] In this solution, the intelligent prediction module constructs a vascular pressure tolerance prediction model, specifically: A vascular pressure tolerance prediction model was constructed using a multi-task learning framework. Parallel pressure regression, risk classification, and safety threshold branches were set up. The vascular pressure tolerance prediction model was pre-trained using transfer learning. The fusion feature vector is input into a blood vessel pressure tolerance prediction model, a 3-layer bottleneck MLP is used for feature distillation of the fusion feature vector, and a preset proportion of key dimensions is reserved through feature importance analysis; According to the calculation of the fusion feature vector after feature distillation and the current injection flow rate of the target puncture point, the risk critical point is identified based on the slope change point of the pressure-flow rate curve through the maximum tolerable pressure of the target puncture point, and the risk level is determined according to the risk critical point using the risk standard; A blood vessel characteristic-safety flow rate mapping table is established, mapping features are read from the blood vessel characteristic-safety flow rate mapping table according to the current blood vessel state, and a particle swarm algorithm is used to optimize the flow rate safety threshold according to the mapping features, and a recommended injection flow rate range is output; The maximum tolerable pressure of the target puncture point, the extravasation risk level and the recommended injection flow rate range under the current injection flow rate are obtained, and injection suggestions and injection alarms are generated.

[0011] The second aspect of the application provides a multi-modal fusion-based image-enhanced examination peripheral vein puncture blood vessel pressure monitoring method, comprising the following steps: Image data, sensor data and personalized data of a patient are collected, and multi-modal data of the patient are preprocessed; Image feature vectors, sensor feature vectors and personalized feature vectors are generated based on the preprocessed multi-modal data, multi-modal feature lines are interacted, representation fusion is performed through Transformer combined with contrast learning, and a fusion feature vector is generated; The fusion feature vector is input into a pre-trained blood vessel pressure tolerance prediction model for prediction, and the maximum tolerable pressure of the target puncture point, the extravasation risk level and the recommended injection flow rate range under the current injection flow rate are evaluated; Injection suggestions and injection alarms of the target puncture point are generated based on the maximum tolerable pressure, the extravasation risk level and the recommended injection flow rate range, and injection parameters are automatically adjusted.

[0012] Compared with the prior art, the beneficial effects of the application are: The multi-modal fusion-based image-enhanced examination peripheral vein puncture blood vessel pressure monitoring system provided by the application adopts dual-mode synchronous acquisition technology of ultrasonic images and infrared blood vessel detection, combined with a high-sensitivity flexible pressure sensing array, to realize all-around acquisition of blood vessel structure and function information. The multi-modal intelligent analysis module realizes deep fusion of image features and mechanical features through a deep learning architecture, constructs a high-precision blood vessel pressure tolerance prediction model, and can accurately evaluate the individualized safety pressure threshold of patients with different blood vessels.

[0013] The application breaks through the limitation of traditional blood vessel evaluation relying on subjective experience, provides a scientific basis for clinical operation through objective quantitative indicators, and significantly improves the safety and accuracy of peripheral vein puncture. The dynamic risk warning function can monitor pressure fluctuations in real time and issue warnings before extravasation occurs, effectively avoiding contrast agent injection-related complications. It is suitable for patient groups with special vascular conditions, such as elderly patients, children and patients after chemotherapy, and individual pressure limit recommendations ensure the safety of high-risk groups. The collaborative analysis of multi-modal data not only improves the comprehensiveness of blood vessel evaluation, but also enhances the adaptability of the system to different physiological states. BRIEF DESCRIPTION OF DRAWINGS

[0014] In order to more clearly illustrate the technical solutions in the embodiments or examples of the present application, the drawings needed to be used in the embodiments or examples will be briefly introduced as follows. Obviously, the drawings in the following description only constitute some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor.

[0015] Figure 1 A framework diagram of the image-enhanced examination peripheral vein puncture blood vessel pressure monitoring system based on multi-modal fusion is shown. Figure 2 A flowchart for generating a fusion feature vector is shown. Figure 3 A flowchart for constructing a blood vessel pressure tolerance prediction model for prediction is shown. Figure 4 A flowchart of the image-enhanced examination peripheral vein puncture blood vessel pressure monitoring method based on multi-modal fusion is shown. DETAILED DESCRIPTION

[0016] In order to more clearly illustrate the technical solutions in the embodiments or examples of the present application, the drawings needed to be used in the embodiments or examples will be briefly introduced as follows. Obviously, the drawings in the following description only constitute some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor.

[0017] In the following description, many specific details are set forth in order to provide a thorough understanding of the present application, however, the present application can also be implemented in other ways different from those described herein, therefore, the scope of protection of the present application is not limited by the specific embodiments disclosed below.

[0018] Figure 1 A framework diagram of the image-enhanced examination peripheral vein puncture blood vessel pressure monitoring system based on multi-modal fusion is shown.

[0019] A first embodiment of the present invention provides an image-enhanced inspection system for peripheral vein puncture vascular pressure monitoring based on multimodal fusion, comprising: a patient data acquisition module, a personalized data representation module, an image data representation module, a sensor data representation module, a patient information representation fusion module, an intelligent prediction module, a prediction result output module, and a prediction training module; The patient data acquisition module collects the patient's imaging data, sensor data, and personalized data, pre-processes the collected patient data, and inputs it into the characterization module according to the preset format requirements; The personalized data characterization module generates a personalized feature vector of clinical risk based on the pre-processed structured patient personalized data; The image data characterization module extracts vascular structural features and hemodynamic features based on the preprocessed image data, fixes the representation dimensions, and generates an image feature vector; The sensor data characterization module processes sensor data in the time domain and frequency domain jointly, extracts tissue mechanical response characteristics, fixes the dimension of the characterization, and generates a sensor feature vector; The patient information representation fusion module performs feature interaction on the personalized feature vector, the image feature vector, and the sensor feature vector, and performs representation fusion through Transformer combined with contrast learning to generate a fused representation vector; The intelligent prediction module constructs a vascular pressure tolerance prediction model, inputs the fused representation vector into the pre-trained vascular pressure tolerance prediction model for prediction, and evaluates the maximum tolerable pressure and extravasation risk level of the target puncture point at the current injection flow rate, as well as the recommended injection flow rate range; The prediction result output module encrypts and desensitizes the prediction data, and returns and displays the injection suggestion and injection alarm of the target puncture point according to the preset format and transmission mode; The prediction training module trains the vascular pressure resistance prediction model through multi-task learning of the three types of heterogeneous representations after comparative learning fusion.

[0020] It should be noted that the wearable probe holder synchronously collects the vascular cross-sectional structure data of the ultrasound imaging modality and the blood flow distribution data of the infrared imaging modality, and the mechanical response signal of the tissue around the puncture point is obtained in real time through the matrix capacitive sensor array, and the patient's personalized clinical data is recorded, including the image-enhanced examination of the three-level vascular evaluation form of the peripheral venous access.

[0021] Categorical variables such as vascular elasticity and diameter grades in the three-level vascular evaluation table were one-hot encoded. Continuous variables such as vascular wall thickness and blood flow velocity were Z-score standardized, and missing values ​​were filled using the mean of similar patient data. The preprocessed structured data corresponding to the three-level vascular evaluation table for peripheral venous access were obtained. The characteristics of each clinical indicator in the three-level vascular evaluation table were grouped according to physiological significance, such as vascular morphology feature group, hemodynamic feature group, and patient baseline feature group. A low-dimensional embedding representation was created for each feature group, and the original features were mapped to a unified dimensional space.

[0022] The multilayer perceptron is composed of an input layer, a hidden layer, and an output layer. The input layer receives preprocessed structured data, whose dimensionality depends on the number of fields in the evaluation form. The hidden layer consists of two to three fully connected layers, each followed by a Relu activation function and batch normalization. The embedded features are imported into the multilayer perceptron, and the hidden layer is used to learn the nonlinear relationships between clinical indicators in the three-level vascular evaluation form. After learning through feature interactions, a feature vector is output, with particular attention paid to the interaction effects between vascular elasticity and diameter and basic features such as patient age and BMI. A self-attention mechanism is introduced, and the feature vector output by the MLP is input into the attention layer as the query, key, and value. The attention score of each feature vector is calculated, and several parallel attention heads are used to capture different types of feature vector dependencies, with each head focusing on different aspects of clinical feature interactions. Attention weights are generated through attention scores, and the attention weights are used to identify the most important clinical indicators for vascular pressure resistance. The attention weights are then used to weight the importance and dynamically adjust the contribution of the features of each clinical indicator to the final prediction. The output of the multi-head attention layer is residually connected with the feature vector output by the multi-layer perceptron to generate an attention-enhanced feature vector. The attention-enhanced feature vector is mapped to the final feature space to output a personalized feature vector of fixed dimension.

[0023] It should be noted that the ultrasound images and infrared images are subjected to denoising, contrast enhancement and other enhancement processing, the timestamps and spatial markers of the hardware synchronization device are used to synchronize the ultrasound and infrared images for temporal and spatial consistency, and a feature-based registration algorithm is used to achieve pixel-level alignment of the dual-modality images. The region of interest is determined based on the infrared blood vessel detection results, and the corresponding area of ​​the ultrasound image is cropped.

[0024] The pre-processed ultrasonic and infrared dual-mode image data are acquired, a blood vessel structure feature extraction branch is constructed based on an improved U-Net structure, a deep multi-scale dense block is established in the encoder part of the blood vessel structure feature extraction branch, each dense block contains four convolution layers, the output of each layer is connected to the input of all subsequent layers, and feature reuse is realized; a cross-layer residual connection is introduced in the dense block, the block input is directly added to the block output, and gradient disappearance is alleviated. In the deep multi-scale dense block, a multi-scale convolution is designed, and a feature map is obtained by using the multi-scale convolution in parallel, and each scale feature map is fused by channel splicing. Batch normalization and LeakyReLU activation are added after each convolution. The feature maps output by the deep multi-scale dense block are divided into three groups, an SK attention mechanism is introduced, a channel statistical descriptor is generated by global average pooling, the attention weights of each channel are learned through a fully connected layer, and weight normalization is realized by using softmax. The learned weights are multiplied with the original feature map channel by channel to represent the importance of the blood vessel structure features. The SK attention mechanism dynamically adjusts the channel weights, so that the network focuses on the key areas of blood vessels and improves the recognition sensitivity of small blood vessels.

[0025] In the decoder part of the blood vessel structure feature extraction branch, the high-resolution feature maps of each stage of the encoder part are saved, and after the channel number is unified by one-dimensional convolution, the corresponding layer features of the decoder part are spliced, feature compensation is performed, transverse connections from shallow to deep layers are established, deformable convolution is used to adapt to the changes of blood vessel morphology, a gating mechanism is used to control the intensity of information flow, and detail recovery is realized; the deep features are used as gating signals, a spatial attention map is generated by convolution and sigmoid, the feature maps of different decoding stages are input into the attention gate, the importance of different blood vessel regions is represented according to the spatial attention map, the feature contribution is dynamically adjusted according to the importance of the blood vessel regions, the shallow features and the deep features are fused according to the feature contribution, and the blood vessel structure features are obtained, for example, 80% high-level features and 20% low-level details are reserved when finally fused.

[0026] It is important to note that the preprocessed ultrasound and infrared dual-modality imaging data is acquired, and the ultrasound and infrared sequences are segmented into cardiac cycle phases. Spatiotemporal features are extracted based on the vessel center to construct spatiotemporal image blocks. These spatiotemporal image blocks are then fed into a 3D convolutional neural network. A hierarchical structure and dilated convolutions are used to extract local blood flow patterns and global hemodynamic states as low-level features and high-level features, respectively. Dilated convolutions are introduced to expand the receptive field while maintaining resolution. Low-level features capture local blood flow patterns and can be used to extract vessel wall motion trajectories, while high-level features encode the global hemodynamic state and can be used to identify areas of abnormal turbulence. The PSA module is enhanced by introducing a spatial pyramid to capture blood flow heterogeneity in different vascular regions, and a temporal pyramid to model the dynamic characteristics of each phase of the cardiac cycle. The low-level and high-level features are processed using spatial and temporal pyramids, and spatial and temporal pyramid attention is used for cross-dimensional interaction. The spatial attention map is broadcast along the time axis, and the temporal weight is expanded along the spatial dimension. The interactive features are imported into the maximum velocity regression head and the turbulence intensity classification head for physical parameter prediction. The physical parameter prediction structure is fused with low-level and high-level features to obtain hemodynamic characteristics. The explicit quantitative parameters and implicit deep features complement each other to improve the model's robustness in low-quality images. A cross-modal attention mechanism is introduced to fuse vascular structural features and hemodynamic features to obtain a fixed-dimensional image feature vector.

[0027] It should be noted that a hardware synchronization trigger mechanism ensures time alignment of data from the capacitive sensor array, performs baseline correction on each channel, and eliminates individual sensor offset differences. Preprocessed flexible sensor data is acquired, and the puncture area is dynamically segmented. The maximum pressure value, pressure gradient, and contact area in the puncture area are collected as dynamic mechanical parameters. Based on these dynamic mechanical parameters, time-domain features such as pulse wave transit time, waveform steepness, signal entropy, and autocorrelation attenuation coefficient are derived.

[0028] After preprocessing, the flexible sensor data was transformed using a continuous wavelet transform (CWT). The mother wavelet used was the Morlet function, scaled to correspond to the physiological frequency band of 0.1-50 Hz. Five characteristic frequency bands were extracted: ultra-low frequency (0.1-0.5 Hz), vascular pulsation frequency (0.6-2 Hz), muscle tension frequency (2-5 Hz), and high-frequency noise (>5 Hz). Energy distribution and phase coupling characteristics were collected as frequency domain features. For example, the energy distribution ratio was calculated by calculating the proportion of the vascular pulsation frequency band energy to the total energy. The frequency domain coupling degree was analyzed by analyzing the coherence coefficients of sensors at different locations in the characteristic frequency bands. The sensor array was constructed as a graph structure, with nodes representing sensors and margins reflecting spatial adjacency. A graph attention network was used to propagate features and aggregate neighbors, generating mechanical response features and obtaining fixed-dimensional sensor feature vectors. The graph attention network used four attention heads per layer to learn different mechanical interaction patterns, and a margin gating mechanism was used to control the intensity of information propagation.

[0029] Figure 2 A flow chart for generating a fused representation vector is shown; According to an embodiment of the present invention, the patient information representation fusion module performs feature interaction on the personalized feature vector, the image feature vector, and the sensor feature vector, and performs representation fusion through Transformer combined with contrastive learning to generate a fused representation vector. Specifically, S202: The personalized feature vector, the image feature vector, and the sensor feature vector are specifically embedded and projected and position information is injected to construct a multimodal feature sequence, which is imported into the Transformer encoding and hierarchical interactive attention is set; S204, perform feature interaction through intra-modal self-attention and cross-modal cross-attention, and obtain the output of each Transformer layer through gated feature update; S206, training through a contrastive learning mechanism to generate, for the same patient sample, the original three-modal complete view, a virtual view consisting of only the personalized feature vector and the image feature vector, and a virtual view consisting of only the personalized feature vector and the sensor feature vector. Positive sample pairs are constructed based on the different view features of the same patient, and negative sample pairs are constructed based on the random view pairing features of different patients. NT-Xent loss function is used to perform contrastive learning enhancement based on the positive and negative sample pairs. S208 , obtaining a global fusion representation after iterative updating through a 6-layer Transformer, performing weighted concatenation of the global fusion representation and the mean feature vectors of each modality to generate a fusion representation vector.

[0030] It should be noted that the three feature vectors are mapped to a unified feature space through independent linear projection layers, and the projected features are layer-normalized to eliminate distribution differences between modalities. Spatial position encoding is generated for image and sensor features, and personalized features are added with position markers filled with all 1s. Intra-modal self-attention restricts each modality to internal interaction through attention masks, strengthening the consistency of each modality's own features. The query (Query) in cross-modal cross-attention comes from the target modality, and the key-value (Key-Value) comes from the other two modalities. Each Transformer layer uses a residual connection of the intra-modal self-attention output, the cross-modal cross-attention output, and the input as the output, and adaptive fusion of features is achieved through gated cross-modal attention. The NT-Xent loss function is used to perform contrastive learning enhancement based on positive and negative sample pairs. The virtual perspective construction of contrastive learning enhancement enables the model to work reliably even when the modality is missing. The loss function is expressed as: , in Represents the feature vector of the anchor sample, which is the target sample that needs to be optimized. represents the feature vector of the positive sample, Represents the feature vector of negative samples, Represents the temperature coefficient, which is used to adjust the sharpness of the probability distribution. The similarity function is usually cosine similarity.

[0031] Figure 3 A flowchart for constructing a vascular pressure resistance prediction model for prediction is shown.

[0032] According to an embodiment of the present invention, the intelligent prediction module constructs a vascular pressure resistance prediction model, specifically: S302, constructing a vascular pressure tolerance prediction model using a multi-task learning framework, setting up a parallel pressure regression branch, a risk classification branch, and a safety threshold branch, and pre-training the vascular pressure tolerance prediction model through transfer learning; S304, importing the fused representation vector into the vascular pressure resistance prediction model, performing feature distillation on the fused representation vector using a three-layer bottleneck MLP, and retaining a preset proportion of key dimensions through feature importance analysis; S306, calculating the current injection flow rate of the target puncture point based on the fused representation vector after feature distillation and the maximum tolerable pressure of the target puncture point and the slope change point of the pressure-flow rate curve, and determining the risk level based on the risk criteria according to the risk critical point; S308, establishing a vascular characteristic-safe flow rate mapping table, reading mapping features from the vascular characteristic-safe flow rate mapping table based on the current vascular state, optimizing the flow rate safety threshold using a particle swarm algorithm based on the mapping features, and outputting a recommended injection flow rate range; S310, obtaining the maximum tolerable pressure, extravasation risk level, and recommended injection flow rate range of the target puncture point at the current injection flow rate, and generating an injection suggestion and an injection alarm.

[0033] It is important to note that the vascular pressure tolerance prediction model was constructed using a multi-task learning framework. The pressure regression branch predicts the maximum tolerable pressure using the Huber loss function; the risk classification branch outputs the extravasation risk level using a weighted cross-entropy loss; and the safety threshold branch outputs the recommended injection flow rate range. The prediction training module pre-trained the vascular pressure tolerance prediction model through transfer learning. The fusion representation vector extraction component of the vascular pressure tolerance prediction model, pre-trained on 100,000 vascular imaging data, was used to pre-train the basic prediction branch on 50,000 in vitro vascular model pressure test data. Physiological constraints such as vascular rupture pressure were incorporated into the pre-training process.

[0034] A quantitative relationship between vascular physiological parameters and safe injection flow rate was established, including vascular elastic modulus, blood flow resistance index, vessel wall stress distribution, vessel diameter, and viscoelastic attenuation coefficient. The vascular elastic modulus reflects the mechanical properties of the vessel wall and is negatively correlated with the maximum allowable flow rate. The blood flow resistance index is calculated based on Doppler ultrasound and influences the recommended acceleration threshold. The vessel wall stress distribution is modeled using finite element analysis and determines the pressure equalization coefficient. Vessel diameter is directly related to the critical flow rate, and the viscoelastic attenuation coefficient is extracted from sensor signals in the time-frequency domain and influences the dynamic adjustment sensitivity. Real-time parameters are obtained based on the current vascular state. The closest combination of vascular characteristics is matched in a mapping table. Bilinear interpolation is used for discrete data to calculate the initial value of the continuous safe flow rate. A particle swarm algorithm optimized with simulated annealing is used to optimize the maximum safe flow rate. The maximum safe flow rate is determined under the conditions that the pressure is less than the maximum tolerable pressure and the extravasation risk is less than a preset risk threshold.

[0035] Initialize the particle swarm based on the initial value of the safe flow rate combined with random disturbance, improve the particle swarm algorithm through simulated annealing to set the probability of accepting inferior solutions, evaluate the particle fitness in iterative optimization, and The calculation formula is: , in Indicates the maximum tolerable pressure of the blood vessels. Indicates flow rate Real-time vascular pressure value under Indicates flow rate The normalized value of 、 represents the weight coefficient; Get the individual optimal and global optimal Update particle velocity: , in Indicates the Particles in the next iteration The updated speed is Indicates the Particles in the number of iterations The velocity vector at time , represents the inertia weight, 、 represents the acceleration factor, 、 represents a random number, Indicates the Particles in the number of iterations Current location at the time.

[0036] The temperature is lowered every 10 iterations, and some inferior solutions are accepted to jump out of the local optimum. When the number of iterations reaches the maximum number of iterations or the fitness change rate is less than the preset threshold, the maximum safe flow rate is obtained. The optimal solution is corrected according to the real-time risk prediction fine-tuning results. The flow rate safety threshold Expressed as: , in It represents the initial value of the reference flow rate obtained from the blood vessel characteristics-safe flow rate mapping table. Represents a dynamic adjustment factor based on real-time risk level; Generate a recommended injection flow rate range based on the flow rate safety threshold : , in They respectively represent the minimum flow rate and maximum flow rate allowed for this type of blood vessel in the mapping table.

[0037] Based on the maximum tolerable pressure, extravasation risk level and recommended injection flow rate range, injection recommendations and injection alarms for the target puncture point are generated, a patient-specific pressure-flow rate database is established, personalized prediction model parameters are updated according to the preset time step, and reverse feature analysis is performed on prediction error cases to achieve adaptive optimization. Preferably, in the prediction result output module, the real-time pressure distribution is superimposed on the ultrasound image to display the risk prediction curve for the next 3 minutes. Injection plan recommendations are generated based on the different color distributions of the pressure thermogram. For example, the green interval corresponds to the recommended safe flow rate, the yellow interval corresponds to the critical value that requires careful operation, and the red interval corresponds to the contraindication range. In addition, the risk judgment threshold is automatically adjusted for special conditions such as vascular sclerosis in cancer patients after chemotherapy and increased vascular fragility in diabetic patients.

[0038] Figure 4 A flowchart of a method for monitoring vascular pressure during peripheral venous puncture based on multimodal fusion image enhancement is shown.

[0039] A second embodiment of the present invention provides a method for monitoring blood pressure during peripheral venous puncture based on image enhancement and multimodal fusion, comprising the following steps: S402, collecting the patient's imaging data, sensor data, and personalized data, and preprocessing the patient's multimodal data; S404: Generate an image feature vector, a sensor feature vector, and a personalized feature vector based on the preprocessed multimodal data, perform feature interaction on the multimodal feature circuit, perform representation fusion through Transformer combined with contrastive learning, and generate a fused representation vector; S406, inputting the fused representation vector into the pre-trained vascular pressure tolerance prediction model to perform prediction, and evaluating the maximum tolerable pressure, extravasation risk level, and recommended injection flow rate range of the target puncture point at the current injection flow rate; S408: Generate an injection suggestion and an injection alarm for the target puncture point based on the maximum tolerable pressure, extravasation risk level, and recommended injection flow rate range, and automatically adjust injection parameters.

[0040] The third embodiment of the present invention provides a computer-readable storage medium, which includes a program for a method for monitoring the pressure of peripheral vein puncture vessels by image enhancement inspection based on multimodal fusion. When the program for monitoring the pressure of peripheral vein puncture vessels by image enhancement inspection based on multimodal fusion is executed by a processor, the steps of the method for monitoring the pressure of peripheral vein puncture vessels by image enhancement inspection based on multimodal fusion are implemented.

[0041] In the several embodiments provided in this application, it should be understood that the disclosed methods and systems can be implemented in other ways. The system embodiments described above are merely illustrative. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored, or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms. In addition, the functional units in the various embodiments of the present invention can all be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the above-mentioned integrated units can be implemented in the form of hardware or in the form of hardware plus software functional units.

[0042] Those skilled in the art will understand that all or part of the steps of the above-mentioned method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above-mentioned method embodiment; and the aforementioned storage medium includes: mobile storage devices, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disks or optical disks, and other media that can store program codes.

[0043] The above description is only a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any technician familiar with this technical field can easily think of changes or replacements within the technical scope disclosed by the present invention, which should be covered by the scope of protection of the present invention.

Claims

1. An image-enhanced inspection system for peripheral vein puncture vascular pressure monitoring based on multimodal fusion, characterized in that: include: Patient data acquisition module, personalized data representation module, image data representation module, sensor data representation module, patient information representation fusion module, intelligent prediction module, prediction result output module and prediction training module; The patient data acquisition module collects the patient's imaging data, sensor data, and personalized data, pre-processes the collected patient data, and inputs it into the characterization module according to the preset format requirements; The personalized data characterization module generates a personalized feature vector of clinical risk based on the pre-processed structured patient personalized data; The image data characterization module extracts vascular structural features and hemodynamic features based on the preprocessed image data, fixes the representation dimensions, and generates an image feature vector; The sensor data characterization module processes sensor data in the time domain and frequency domain jointly, extracts tissue mechanical response characteristics, fixes the dimension of the characterization, and generates a sensor feature vector; The patient information representation fusion module performs feature interaction on the personalized feature vector, the image feature vector, and the sensor feature vector, and performs representation fusion through Transformer combined with contrast learning to generate a fused representation vector; The intelligent prediction module constructs a vascular pressure tolerance prediction model, inputs the fused representation vector into the pre-trained vascular pressure tolerance prediction model for prediction, and evaluates the maximum tolerable pressure and extravasation risk level of the target puncture point at the current injection flow rate, as well as the recommended injection flow rate range; The prediction result output module encrypts and desensitizes the prediction data, and returns and displays the injection suggestion and injection alarm of the target puncture point according to the preset format and transmission mode; The prediction training module trains the vascular pressure resistance prediction model through multi-task learning of the three types of heterogeneous representations after comparative learning fusion.

2. The image-enhanced inspection system for peripheral vein puncture and vascular pressure monitoring based on multimodal fusion according to claim 1 is characterized in that: The personalized data characterization module generates a personalized feature vector of clinical risk based on the pre-processed structured patient personalized data, specifically: Obtaining structured data corresponding to the preprocessed three-level peripheral venous access vascular evaluation table, grouping the features of each clinical indicator in the three-level vascular evaluation table according to physiological significance, creating a low-dimensional embedding representation for each group of features, and mapping the original features to a unified dimensional space; The embedded features are imported into a multi-layer perceptron, and the hidden layer is used to learn the nonlinear relationship between the clinical indicators in the three-level vascular evaluation table. After interactive learning of features, the feature vector is output. The self-attention mechanism is introduced to calculate the attention score of each feature vector. Several parallel attention heads are used to capture different types of feature vector dependencies. Attention weights are generated from attention scores. These weights are used to identify the most important clinical indicators for vascular pressure tolerance. The attention weights are then used to weight the importance of each clinical indicator and dynamically adjust the contribution of each clinical indicator's features to the final prediction. The output of the multi-head attention layer is residually connected with the feature vector output by the multi-layer perceptron to generate an attention-enhanced feature vector, which is then mapped to the final feature space to output a personalized feature vector of fixed dimension.

3. The image-enhanced inspection system for peripheral vein puncture and vascular pressure monitoring based on multimodal fusion according to claim 1 is characterized in that: The image data characterization module extracts vascular structural features and hemodynamic features based on the preprocessed image data, fixes the representation dimensions, and generates an image feature vector, specifically: Preprocessed ultrasound and infrared dual-modality imaging data are obtained. A vascular structure feature extraction branch is constructed based on an improved U-Net structure. Deep multi-scale dense blocks are established in the encoder part of the vascular structure feature extraction branch. Each dense block contains four convolutional layers, and the output of each layer is connected to the input of all subsequent layers. A cross-layer residual connection is introduced within the dense block, the block input is directly added to the block output, and multi-scale convolution is used in parallel to obtain feature maps. The feature maps output by the deep multi-scale dense block are divided into three groups. The SK attention mechanism is introduced to multiply the learned weights with the original feature maps channel by channel to represent the importance of vascular structure features. In the decoder part of the vascular structure feature extraction branch, the high-resolution feature maps of each stage of the encoder are saved. After unifying the number of channels through one-dimensional convolution, they are spliced ​​with the features of the corresponding layers of the decoder to perform feature compensation. Horizontal connections are established from shallow to deep layers. Deformable convolution is used to adapt to changes in vascular morphology, and a gating mechanism is used to control the intensity of information flow. Deep features are used as gating signals, and a spatial attention map is generated through convolution and sigmoid. The importance of different vascular regions is represented according to the spatial attention map. The feature contribution is dynamically adjusted according to the importance of the vascular regions. The shallow features and deep features are fused according to the feature contribution to obtain the vascular structure characteristics.

4. The image-enhanced inspection system for peripheral vein puncture and vascular pressure monitoring based on multimodal fusion according to claim 3 is characterized in that: The image data characterization module extracts vascular structural features and hemodynamic features based on the preprocessed image data, fixes the representation dimensions, and generates an image feature vector, specifically: Obtain preprocessed ultrasound and infrared dual-modality image data, extract spatiotemporal features to construct spatiotemporal image blocks, input the spatiotemporal image blocks into a 3D convolutional neural network, and use a hierarchical structure and dilated convolution to obtain local blood flow patterns and global hemodynamic states as low-level features and high-level features, respectively; Performing spatial pyramid and temporal pyramid processing on the low-level features and high-level features, using spatial pyramid attention and temporal pyramid attention for cross-dimensional interaction, and importing the interactive features into the maximum velocity regression head and the turbulence intensity classification head for physical parameter prediction; The physical parameter prediction structure is fused with low-level features and high-level features to obtain hemodynamic features. A cross-modal attention mechanism is introduced to fuse vascular structure features and hemodynamic features to obtain a fixed-dimensional image feature vector.

5. The image-enhanced inspection system for peripheral vein puncture and vascular pressure monitoring based on multimodal fusion according to claim 1 is characterized in that: The sensor data characterization module processes sensor data in the time domain and frequency domain, extracts tissue mechanical response characteristics, fixes the dimension of the characterization, and generates a sensor feature vector, specifically: Acquiring preprocessed flexible sensor data, dynamically segmenting the puncture area, collecting the maximum pressure value, pressure gradient, and contact area of ​​the puncture area as dynamic mechanical parameters, and acquiring time domain features based on the dynamic mechanical parameters; The pre-processed flexible sensor data is subjected to continuous wavelet transform to collect energy distribution characteristics and phase coupling characteristics as frequency domain features; The sensor array is constructed as a graph structure, with nodes as sensors and margins reflecting spatial adjacency. Feature propagation and neighbor aggregation are performed through a graph attention network to generate mechanical response features and obtain sensor feature vectors of fixed dimensions.

6. The image-enhanced inspection system for peripheral vein puncture and vascular pressure monitoring based on multimodal fusion according to claim 1 is characterized in that: The patient information representation fusion module performs feature interaction on the personalized feature vector, the image feature vector, and the sensor feature vector, and performs representation fusion through Transformer combined with contrast learning to generate a fused representation vector. Specifically, The personalized feature vector, image feature vector, and sensor feature vector are specifically embedded and projected, and position information is injected into them to construct a multimodal feature sequence, which is imported into the Transformer encoding and hierarchical interactive attention is set. Feature interaction is performed through intra-modal self-attention and cross-modal cross-attention, and the output of each Transformer layer is obtained through gated feature updates; Through contrastive learning, training was performed to generate the original trimodal full view, a virtual view consisting of only personalized feature vectors and imaging feature vectors, and a virtual view consisting of only personalized feature vectors and sensor feature vectors for the same patient sample. Positive sample pairs were constructed based on the different viewpoint features of the same patient, while negative sample pairs were constructed based on the randomly paired viewpoint features of different patients. The NT-Xent loss function was used to perform contrastive learning enhancement based on the positive and negative sample pairs. After iterative updating through the 6-layer Transformer, a global fusion representation is obtained, and the global fusion representation is weightedly concatenated with the mean feature vector of each modality to generate a fusion representation vector.

7. The image-enhanced inspection system for peripheral vein puncture and vascular pressure monitoring based on multimodal fusion according to claim 1 is characterized in that: The intelligent prediction module constructs a vascular pressure tolerance prediction model, specifically: A vascular pressure tolerance prediction model was constructed using a multi-task learning framework. Parallel pressure regression, risk classification, and safety threshold branches were set up. The vascular pressure tolerance prediction model was pre-trained using transfer learning. Importing the fused representation vector into the vascular pressure resistance prediction model, performing feature distillation on the fused representation vector using a three-layer bottleneck MLP, and retaining a preset proportion of key dimensions through feature importance analysis; Calculate the current injection flow rate of the target puncture point based on the fusion representation vector after feature distillation and the maximum tolerable pressure of the target puncture point, identify the risk critical point based on the slope change point of the pressure-flow rate curve, and determine the risk level based on the risk standard according to the risk critical point; Establishing a vascular characteristic-safe flow rate mapping table, reading mapping features from the vascular characteristic-safe flow rate mapping table based on the current vascular state, using a particle swarm algorithm to optimize the flow rate safety threshold based on the mapping features, and outputting a recommended injection flow rate range; Obtain the maximum tolerable pressure, extravasation risk level, and recommended injection flow rate range of the target puncture point at the current injection flow rate, and generate injection recommendations and injection alarms.

8. A method for monitoring blood pressure during peripheral venous puncture based on multimodal fusion image enhancement, characterized in that: The image enhancement inspection peripheral vein puncture vascular pressure monitoring system based on multimodal fusion as described in any one of claims 1 to 7 comprises the following steps: Collect patient imaging data, sensor data, and personalized data, and pre-process the patient's multimodal data; Generate image feature vectors, sensor feature vectors, and personalized feature vectors based on preprocessed multimodal data. Perform feature interaction on multimodal feature circuits, perform representation fusion through Transformer combined with contrastive learning, and generate a fused representation vector. Inputting the fused representation vector into a pre-trained vascular pressure tolerance prediction model for prediction, and evaluating the maximum tolerable pressure, extravasation risk level, and recommended injection flow rate range of the target puncture point at the current injection flow rate; Based on the maximum tolerable pressure, extravasation risk level and recommended injection flow rate range, injection suggestions and injection alarms for the target puncture point are generated, and injection parameters are automatically adjusted.

Images