Embolic agent infusion control method, system and equipment based on reinforcement learning and medium

Abstract

The invention relates to an embolic agent infusion control method, system and equipment based on reinforcement learning and a medium. The method comprises the following steps: constructing an intraoperative blood vessel dynamic topology model in real time, and fusing multi-modal sensing data to generate a standardized state vector; adaptively deciding a pressure adjustment and target branch selection instruction by using a reinforcement learning agent according to the vascular morphological change; curvature compensation flow calculation is carried out in combination with geometric characteristics of the blood vessel, and a precise injection control signal is generated; and monitoring the embolism risk of the non-target area in real time based on the diffusion state of the embolism agent and feeding back an optimization decision strategy. According to the method, the problem of non-targeted embolism caused by static parameter setting and open-loop control in the traditional embolism is solved, the intraoperative dynamic sensing of vascular variation and precise embolism agent delivery of closed-loop feedback regulation are realized, and the clinical operation risk is remarkably reduced.

Description

Technical Field

[0001] This invention relates to the field of intelligent medical device control technology, specifically to a method, system, device, and medium for controlling embolic agent infusion based on reinforcement learning. Background Technology

[0002] Transcatheter arterial chemoembolization (TACE) is an important interventional procedure for treating solid tumors such as liver cancer. This method uses a microcatheter to precisely deliver embolic agents and chemotherapy drugs to the tumor-feeding arteries, blocking blood flow and releasing the drugs locally. In clinical practice, physicians need to adjust the embolic agent infusion strategy in real time based on intraoperative digital subtraction angiography (DSA) images to ensure the embolic agent accurately reaches the target area in the complex vascular network, while avoiding accidental entry into high-risk branches that could damage non-target organs. This process heavily relies on the physician's experience, judgment, and hand-eye coordination, requiring the simultaneous processing of multidimensional information such as dynamic changes in vascular morphology, hemodynamic parameters, and the diffusion status of the embolic agent.

[0003] However, existing embolization infusion control technologies have significant limitations: on the one hand, traditional open-loop control relies on preset fixed infusion parameters and cannot dynamically adjust the infusion strategy based on real-time morphological changes in vascular branches during surgery (such as abnormal bifurcation angles and abrupt changes in vascular curvature); on the other hand, rule-based control systems struggle to quantify the nonlinear coupling between vascular geometry and hemodynamics, especially when dealing with complex anatomical structures such as hepatic artery variations, often resulting in problems such as excessive injection of embolization agents into high-risk branches or insufficient embolization of the target area. These deficiencies lead to a persistently high incidence of non-target embolization during surgery. More fundamentally, existing technologies lack a real-time feedback mechanism for the diffusion effect of embolization agents, making it impossible to establish a closed-loop control chain of "infusion-monitoring-optimization". Summary of the Invention

[0004] Based on this, the purpose of the present invention is to provide a reinforcement learning-based embolic agent infusion control method, system, device and medium that can perceive dynamic changes in vascular morphology in real time, adaptively adjust infusion parameters and realize closed-loop feedback control.

[0005] The objective of this invention is achieved through the following solution:

[0006] In a first aspect, the present invention provides a reinforcement learning-based method for controlling embolic agent infusion, comprising the following steps:

[0007] S1: Perform vascular topology modeling on the acquired intraoperative digital subtraction angiography sequence, construct the vascular network diagram structure, calculate the bifurcation risk and curvature parameters, and generate a dynamic vascular topology diagram.

[0008] S2: Acquire real-time measurement data from pressure sensors and flow meters, combine them with dynamic vascular topology maps for multimodal fusion, align timestamps and normalize vascular features and sensor readings to generate standardized state vectors.

[0009] S3: Perform reinforcement learning decision-making on the standardized state vector, call the forward propagation state vector of the neural network to generate a continuous action space mapping, decompose the action vector into pressure adjustment coefficient and branch selection probability distribution, and output control commands containing pressure adjustment amount and target branch selection instruction;

[0010] S4: Perform curvature compensation flow calculation processing on the pressure adjustment amount, target branch selection instruction and dynamic vascular topology map in the control command, and apply the curvature attenuation factor to correct the Poiseuille equation in the target branch in combination with the preset vascular geometry characteristics, calculate the target flow value and convert it into a voltage control waveform, generate the infusion pump control signal and send the infusion pump control signal to the infusion pump actuator.

[0011] S5: Obtain digital subtraction angiography images after embolization, combine them with dynamic vascular topology maps for false embolization monitoring, segment non-target regions and calculate the overlap with embolic agent distribution, and update the policy gradient reward value of the neural network.

[0012] In one embodiment, S1 of the reinforcement learning-based embolic agent infusion control method provided by the present invention specifically includes the following steps:

[0013] S11: Based on the intraoperative digital subtraction angiography sequence acquired by medical imaging equipment, perform vascular segmentation processing on the digital subtraction angiography sequence, distinguish vascular tissue from background areas in each frame of image, and generate a binary mask image.

[0014] S12: Perform skeletonization processing on the binary mask image, extract the centerline structure of a single pixel width in the binary mask, and generate a blood vessel centerline map;

[0015] S13: Perform graph structure transformation on the vascular centerline diagram, mark the bifurcation points and endpoints in the centerline network and connect adjacent nodes to generate the initial vascular topology diagram;

[0016] S14: Calculate the geometric parameters of the initial vascular topology map, measure the diameter, length and curvature of each vascular segment, and generate a vascular topology map with parameters.

[0017] S15: Perform risk factor calculation on the parameterized vascular topology map, quantify the difference in sub-vascular diameter at bifurcation points and calculate the risk value of angle abnormalities, and generate a dynamic vascular topology map.

[0018] In one embodiment, S2 of the reinforcement learning-based embolic agent infusion control method provided by the present invention specifically includes the following steps:

[0019] S21: Perform feature extraction processing on the dynamic vascular topology map, select the diameter, length, curvature and risk factors of the current target branch, and generate a vascular feature vector;

[0020] S22: Acquire real-time measurement data from pressure sensors and flow meters, time-align the real-time measurement data, synchronously sample sensor readings within the time window, and generate timestamp-aligned sensor data;

[0021] S23: Normalize the sensor data, scale the pressure and flow values ​​to a uniform range, and generate a normalized sensor vector.

[0022] S24: Perform feature concatenation processing on the vascular feature vector and the normalized sensor vector, combine the multimodal data into a single vector, and generate a standardized state vector.

[0023] In one embodiment, step S3 of the reinforcement learning-based embolic agent infusion control method provided by the present invention specifically includes the following steps:

[0024] S31: Process the standardized state vector with a hidden layer of a neural network, and extract the high-order features in the nonlinear combination of the state vector through a fully connected layer to generate a hidden feature vector;

[0025] S32: Perform action space mapping processing on the hidden feature vector, output a multi-dimensional vector in the continuous action space within a preset range of continuous values, and generate the original action vector;

[0026] S33: Perform instruction parsing processing on the original action vector, separate the pressure adjustment coefficient of the first dimension component and the branch selection probability of the second and third dimension components, linearly map the pressure coefficient to the physical pressure range and convert the branch probability into discrete index values, and generate control instructions containing pressure adjustment amount and target branch selection instructions.

[0027] In one embodiment, S4 of the reinforcement learning-based embolic agent infusion control method provided by the present invention specifically includes the following steps:

[0028] S41: Perform branch location processing on the target branch selection instruction of the control command, index the geometric parameters of the target branch in the dynamic vascular topology map, and generate the target branch parameter set;

[0029] S42: Perform curvature compensation calculation on the pressure adjustment amount and target branch parameter set of the control command, apply the attenuation factor to correct the Poiseuille flow equation, and generate the target flow value;

[0030] S43: Perform control conversion processing on the target flow rate value, calculate the flow deviation and output proportional-integral-derivative control quantity, and generate the injection pump control voltage signal;

[0031] S44: Perform waveform modulation processing on the injection pump control voltage signal to generate a pulse width modulation signal, and generate the injection pump control signal.

[0032] In one embodiment, the formula for calculating the target flow rate of the embolic agent infusion control method based on reinforcement learning provided by the present invention is as follows:

[0033]

[0034] in, The target branch's theoretical flow rate, i.e., the target flow rate value. For the target branch pipe diameter, For pressure adjustment amount, For blood viscosity, For the target branch length, The curvature attenuation coefficient, The target branch curvature.

[0035] In one embodiment, step S5 of the reinforcement learning-based embolic agent infusion control method provided by the present invention specifically includes the following steps:

[0036] S51: Perform high-risk region identification processing on the dynamic vascular topology map, screen out vascular branches in the vascular branch set whose risk factors exceed the preset risk threshold, and generate a non-target region coordinate set.

[0037] S52: Perform embolization agent segmentation processing on the acquired digital subtraction angiography images after embolization, identify the diffusion area of ​​the contrast agent in the three-dimensional vascular space, and generate an embolization area mask.

[0038] S53: Perform overlap analysis on the coordinate set of the non-target area and the mask of the embolization area, calculate the proportion of the embolic agent coverage area in the non-target area, and generate the false embolization index.

[0039] S54: Perform target embolization calculation on the embolization area mask, quantify the embolization agent coverage in the target area, and generate the target embolization increment;

[0040] S55: Perform reward function calculation on the false embolism index, target area embolism increment, and pressure adjustment in the control command, and weighted sum the safety penalty term, efficiency reward term, and stability penalty term to generate the policy gradient reward value for updating the neural network.

[0041] Secondly, the present invention provides a reinforcement learning-based embolic agent infusion control system, which is configured with the following modules:

[0042] The vascular topology modeling module is used to perform vascular topology modeling on the acquired intraoperative digital subtraction angiography sequence, construct the vascular network diagram structure, calculate the bifurcation risk and curvature parameters, and generate a dynamic vascular topology diagram.

[0043] The multimodal data fusion module is used to acquire real-time measurement data from pressure sensors and flow meters, combine them with dynamic vascular topology maps for multimodal fusion, align timestamps and normalize vascular features and sensor readings to generate standardized state vectors.

[0044] The reinforcement learning decision module is used to perform reinforcement learning decisions on the standardized state vector. It calls the forward propagation state vector of the neural network to generate a continuous action space mapping and decomposes the action vector into a pressure adjustment coefficient and a branch selection probability distribution. It outputs control commands containing pressure adjustment amount and target branch selection instructions.

[0045] The curvature-compensated flow calculation and control signal generation module is used to perform curvature-compensated flow calculation processing on the pressure adjustment amount, target branch selection instruction and dynamic vascular topology map in the control command. It combines the preset vascular geometric characteristics, applies the curvature attenuation factor to correct the Poiseuille equation in the target branch, calculates the target flow value and converts it into a voltage control waveform, generates the infusion pump control signal and sends the infusion pump control signal to the infusion pump actuator.

[0046] The module for detecting and updating embolism is used to acquire digital subtraction angiography images after embolization, combine them with dynamic vascular topology maps to detect embolism, segment non-target areas and calculate the overlap with the distribution of embolic agents, and update the policy gradient reward value of the neural network.

[0047] Thirdly, this application provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement any of the above-mentioned reinforcement learning-based embolization agent infusion control methods.

[0048] Fourthly, this application provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements any of the above-mentioned reinforcement learning-based embolic agent infusion control methods.

[0049] In summary, the reinforcement learning-based embolization agent infusion control method provided in this application constructs a dynamic vascular topology model to quantify vascular bifurcation risk and curvature parameters in real time, enabling dynamic perception of changes in complex vascular network morphology. Combined with timestamp alignment and feature normalization of multimodal sensor data, it achieves spatiotemporal synchronous fusion of vascular geometric characteristics and blood flow parameters, overcoming the fundamental deficiency of traditional static parameter settings in adapting to intraoperative vascular dynamic variations. The reinforcement learning-based decision-making mechanism maps the state space to a continuous action domain via a neural network, enabling adaptive analysis of pressure adjustment coefficients and branch selection probabilities, overcoming the subjective bias of human experience-based decision-making in bifurcation path selection. Curvature-compensated flow rate calculation, by correcting the attenuation factor in the Poiseuille equation, accurately matches the hydrodynamic characteristics of the target branch, solving the theoretical flow rate deviation caused by the neglect of vascular tortuosity in traditional models. The closed-loop false embolization monitoring mechanism, through the overlap analysis of non-target areas and embolization agent distribution, dynamically generates strategy gradient reward values ​​to optimize the decision-making model, achieving a closed-loop control link of "infusion-monitoring-optimization".

[0050] To better understand and implement this invention, the following detailed description is provided in conjunction with the accompanying drawings. Attached Figure Description

[0051] Figure 1 A flowchart illustrating a reinforcement learning-based embolic agent infusion control method provided in this application embodiment;

[0052] Figure 2 A schematic flowchart illustrating the generation of injection pump control signals provided in an embodiment of this application;

[0053] Figure 3 This is a schematic diagram of a reinforcement learning-based embolic agent infusion control device provided in another embodiment of this application. Detailed Implementation

[0054] To facilitate understanding of the present invention, a more complete description will be given below with reference to the accompanying drawings. Preferred embodiments of the invention are shown in the drawings. However, the invention can be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided to provide a thorough and complete understanding of the disclosure of the invention.

[0055] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0056] In one embodiment, such as Figure 1 As shown, a reinforcement learning-based method for controlling embolization agent infusion is provided. This embodiment illustrates the application of this method to a terminal. It is understood that this method can also be applied to a server, or to a system including both a terminal and a server, and implemented through interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0057] S1: Perform vascular topology modeling on the acquired intraoperative digital subtraction angiography sequence, construct the vascular network diagram structure, calculate the bifurcation risk and curvature parameters, and generate a dynamic vascular topology diagram.

[0058] Specifically, the system acquires real-time DSA image sequences during surgery, filters each frame to remove noise, enhances contrast to improve the distinction between blood vessels and background, and eliminates minor artifacts using morphological operations. The system employs a semantic segmentation network to segment the preprocessed images into vascular regions, outputting a binary vascular mask map. The training dataset for this network includes DSA images from clinical liver cancer TACE surgeries and corresponding manually annotated vascular masks. Based on the segmentation results, the system extracts the vessel centerline using a vessel tracking algorithm, iteratively tracks vessel pixels along the gray-level gradient direction using the microcatheter exit as the seed point, records the three-dimensional coordinates of the centerline, calculates the vessel diameter using distance transformation, and identifies vessel bifurcation points. The system calculates bifurcation risk parameters based on a weighted fusion of two indicators: bifurcation angle and branch diameter ratio. It calculates the angle between the bifurcation vessel centerlines using vector dot product, calculates the diameter ratio of the target branch to the non-target branch, and weights the results according to a predefined logic to obtain the bifurcation risk value.

[0059] Furthermore, the system calculates curvature based on the arc length parameter of the vessel centerline and smooths continuous curvature values ​​to obtain the average curvature. The system constructs a dynamic vascular topology map in the form of a directed weighted graph. The node set contains vessel bifurcation points, endpoints, and microcatheter exit points and stores related attributes. The edge set corresponds to vessel segments and stores attributes such as diameter, length, average curvature, bifurcation risk value, and blood flow direction. The parameters of the corresponding vessel segment are updated each time a new frame of DSA image is acquired, realizing real-time dynamic updating of the topology map.

[0060] S2: Acquire real-time measurement data from pressure sensors and flow meters, perform multimodal fusion by combining dynamic vascular topology maps, align timestamps and normalize vascular features and sensor readings to generate standardized state vectors.

[0061] Specifically, the system acquires real-time data through a pressure sensor integrated with the infusion pump and an ultrasonic flow meter. It filters the raw data to remove pulse noise and smooths the data to reduce fluctuations. Because of the difference in sampling frequencies between DSA images and sensors, the system adds timestamps to each frame of image and each set of sensor data based on a unified clock. It then performs temporal interpolation of vascular topological features using the sensor sampling frequency as a reference, obtaining a continuous sequence of topological features to ensure a one-to-one correspondence between vascular features and sensor data in the time dimension.

[0062] To eliminate dimensional differences in features across different dimensions, the system employs a min-max normalization method for vascular topology features, processing them based on the feature extrema updated in real-time during surgery. For sensor data, a Z-score normalization method is used, processing them based on the statistical characteristics of historical sampling data. The system constructs a standardized state vector that integrates vascular topology feature sub-vectors, real-time sensor data sub-vectors, and historical trend sub-vectors. The vascular topology feature sub-vector contains the normalized features of each vascular segment, the real-time sensor data sub-vector contains normalized pressure and flow data, and the historical trend sub-vector contains the pressure and flow rate changes of previous sampling points. The vector dimension is dynamically adjusted according to the number of vascular branches, forming a unified state representation.

[0063] S3: Perform reinforcement learning decision-making on the standardized state vector, call the forward propagation state vector of the neural network to generate a continuous action space mapping, decompose the action vector into pressure adjustment coefficient and branch selection probability distribution, and output control commands containing pressure adjustment amount and target branch selection instruction.

[0064] Specifically, the system employs a deep deterministic policy gradient algorithm to construct a decision model. This model comprises a policy network and a value network, both using a fully connected neural network structure, suitable for continuous action space scenarios, and can avoid vascular damage caused by abrupt changes in embolization agent infusion parameters. The system defines the structures of the policy network and the value network. The input layer dimension of the policy network is consistent with the normalized state vector, and the output layer dimension is related to the number of vessel segments. The input layer dimension of the value network is the sum of the dimensions of the state vector and the action vector, and the output layer consists of a single neuron for outputting the state-action value. Each layer of the network uses a corresponding activation function, and the weights and biases are initialized according to a set method.

[0065] Furthermore, the system defines the forward propagation output of the policy network as an action vector. The first element of the action vector is the pressure adjustment coefficient, which is mapped to a set interval after inverse normalization. The remaining elements are the original branch selection scores, which are converted into a branch selection probability distribution using a probability transformation function. The system calculates the pressure adjustment amount, determines the target branch based on the branch selection probability distribution, and generates a structured control instruction containing an instruction type identifier, pressure adjustment amount, target branch number, and probability confidence level.

[0066] S4: Perform curvature compensation flow calculation processing on the pressure adjustment amount, target branch selection instruction and dynamic vascular topology map in the control command. Combined with the preset vascular geometry characteristics, apply the curvature attenuation factor to correct the Poiseuille equation in the target branch, calculate the target flow value and convert it into a voltage control waveform, generate the infusion pump control signal and send the infusion pump control signal to the infusion pump actuator.

[0067] Specifically, the system, based on the impact of vascular curvature on blood flow, introduces a curvature attenuation factor to correct the traditional Poiseuille equation. The traditional equation assumes a rigid, straight vessel, which differs from actual clinical vascular conditions. This correction improves the accuracy of flow rate calculation. The system calculates the curvature attenuation factor using empirical calibration coefficients and the average curvature of the target branch. The calibration coefficients are determined through in vitro simulation experiments. Biomimetic vascular models with different curvatures are used to measure and fit the actual flow rate under different pressures. The system calculates the pressure difference, which is the difference between the target infusion pressure and the baseline pressure within the vessel. The target infusion pressure is obtained by summing the current pressure and the pressure adjustment amount. The baseline pressure within the vessel is acquired in real-time through a dedicated monitoring channel of a pressure sensor.

[0068] Preferably, the system uses a modified Poiseuille equation to calculate the target flow rate. The equation inputs include the curvature attenuation factor, pressure difference, pipe diameter and length of the target branch, and the dynamic viscosity of the embolic agent. The dynamic viscosity can be selected and adjusted according to the type of embolic agent. The system performs constraint processing on the calculated target flow rate to ensure that the flow rate is within a reasonable range. Based on the flow-voltage relationship of the injection pump drive mode, the system converts the target flow rate into a drive voltage, generating a PWM waveform. The waveform parameters conform to the set standards. The system encapsulates the PWM waveform into a control signal conforming to the communication protocol and sends it to the injection pump actuator through the communication interface, driving the stepper motor to deliver the embolic agent according to the target flow rate.

[0069] S5: Obtain digital subtraction angiography images after embolization, combine them with dynamic vascular topology maps for false embolization monitoring, segment non-target regions and calculate the overlap with embolic agent distribution, and update the policy gradient reward value of the neural network.

[0070] Specifically, after the infusion pump executes control commands, the system acquires post-embolization DSA images. Preprocessing operations consistent with the vascular topology modeling steps are performed, including filtering, contrast enhancement, and morphological operations to eliminate noise and artifacts. The system predefines non-target region templates based on the patient's preoperative angiography images, manually annotating and storing high-risk non-target branches and their blood supply areas. The system uses an image registration algorithm to map the preoperatively annotated non-target region templates to the post-embolization DSA images, obtaining an intraoperative non-target region mask. Morphological operations are performed on the registered mask to remove edge errors, improving the accuracy of non-target region segmentation. Utilizing the characteristics of the embolic agent in the DSA images, the system employs an adaptive threshold segmentation algorithm to segment the embolic agent distribution area, obtaining an embolic agent mask. The overlap between the embolic agent and the non-target region is quantified using the intersection-union ratio (IU / U), which is the ratio of the intersection area to the union area of ​​the embolic agent and the non-target region.

[0071] Preferably, the system design comprises a reward function with multiple weighted components. The reward value includes a target embolization adequacy reward, a false embolization penalty, and a flow deviation penalty. The target embolization adequacy reward is calculated based on the coverage rate of the embolic agent in the target area; the false embolization penalty is calculated based on the cross-union ratio; and the flow deviation penalty is calculated based on the difference between the actual infusion flow rate and the target flow rate. The system employs a proximal strategy optimization algorithm to update the weight parameters of the neural network. The update process is based on the gradient calculation of the strategy objective function. The advantage function is determined by the difference between the reward value and the state value. The system updates the weight parameters at a set frequency to achieve real-time strategy optimization.

[0072] In summary, the reinforcement learning-based embolization agent infusion control method provided in this application constructs a dynamic vascular topology model to quantify vascular bifurcation risk and curvature parameters in real time, enabling dynamic perception of changes in complex vascular network morphology. Combined with timestamp alignment and feature normalization of multimodal sensor data, it achieves spatiotemporal synchronous fusion of vascular geometric characteristics and blood flow parameters, overcoming the fundamental deficiency of traditional static parameter settings in adapting to intraoperative vascular dynamic variations. The reinforcement learning-based decision-making mechanism maps the state space to a continuous action domain via a neural network, enabling adaptive analysis of pressure adjustment coefficients and branch selection probabilities, overcoming the subjective bias of human experience-based decision-making in bifurcation path selection. Curvature-compensated flow rate calculation, by correcting the attenuation factor in the Poiseuille equation, accurately matches the hydrodynamic characteristics of the target branch, solving the theoretical flow rate deviation caused by the neglect of vascular tortuosity in traditional models. The closed-loop false embolization monitoring mechanism, through the overlap analysis of non-target areas and embolization agent distribution, dynamically generates strategy gradient reward values ​​to optimize the decision-making model, achieving a closed-loop control link of "infusion-monitoring-optimization".

[0073] In one embodiment, S1 of the reinforcement learning-based embolic agent infusion control method provided by the present invention specifically includes the following steps:

[0074] S11: Based on the intraoperative digital subtraction angiography sequence acquired by medical imaging equipment, perform vascular segmentation processing on the digital subtraction angiography sequence, distinguish the vascular tissue and background area in each frame of the image, and generate a binary mask image.

[0075] Specifically, the system receives intraoperative continuous digital subtraction angiography (DSA) sequences transmitted from medical imaging equipment. These sequences contain dynamic image information of vascular structures during the surgery. The system first preprocesses each frame of the sequence, reducing interference signals through filtering and enhancing the distinction between vascular tissue and background regions through grayscale enhancement. After preprocessing, the system uses a vascular segmentation algorithm to process each frame. This algorithm sets segmentation rules based on the grayscale feature differences between vascular tissue and background regions, determining pixel-by-pixel whether each point in the image belongs to vascular tissue or background. Pixels identified as vascular tissue are assigned a unified identifier value; pixels identified as background regions are assigned a different unified identifier value. By performing this processing frame by frame, the system completes vascular segmentation for all images, generating a binary mask image corresponding to each frame. The binary mask image contains only binary distinguishing information between vascular tissue and background regions.

[0076] S12: Perform skeletonization processing on the binary mask image, extract the centerline structure of a single pixel width in the binary mask, and generate a blood vessel centerline map.

[0077] Specifically, the system calls a skeletonization algorithm to process each frame of the binary mask image. During processing, the topological connectivity of the blood vessel structure remains unchanged, and redundant pixels at the edges of the blood vessel region are gradually removed. The algorithm iteratively calculates the boundary pixels of the blood vessel region and determines the retention or removal conditions of pixels, ensuring that the continuous structure of the blood vessel is not destroyed after pixel removal, while making the remaining pixels form a line structure with a single pixel width. During processing, the system monitors the connectivity of the blood vessel structure in real time to avoid breaks or redundant branches. After multiple rounds of iterative processing, the blood vessel region in the binary mask image is simplified to a centerline with a single pixel width. The system integrates the processing results of all frames to generate a blood vessel centerline map corresponding to each frame of digital subtraction angiography image. The centerline map accurately reflects the direction, branch connection relationship and overall morphological characteristics of the blood vessel.

[0078] S13: Perform graph structure transformation on the vascular centerline diagram, mark the bifurcation points and endpoints in the centerline network and connect adjacent nodes to generate the initial vascular topology diagram.

[0079] Specifically, the system performs pixel-level traversal of the vascular centerline map to identify feature points in the centerline network. Points with a number of connected centerline pixels in their neighborhood that meet preset conditions are marked as bifurcation points; centerline endpoint pixels with only a single connected direction are marked as endpoints. The system records the position information of all bifurcation points and endpoints, and establishes a node association table based on the connectivity of the centerlines, where bifurcation points and endpoints constitute a node set in a graph structure. The system defines continuous centerline segments between adjacent nodes as edges, each edge corresponding to a vascular segment, and records the connection relationships between edges and nodes. Based on the node and edge sets, the system constructs graph structure data, with node numbers, edge numbers, and node-edge relationships as core content, forming an initial vascular topology graph. This initial vascular topology graph visually presents the branching connection architecture of the vascular network.

[0080] S14: Calculate the geometric parameters of the initial vascular topology map, measure the diameter, length and curvature of each vascular segment, and generate a vascular topology map with parameters.

[0081] Specifically, the system uses an initial vascular topology map as a basis, combined with the corresponding binary mask image and vascular centerline map, to calculate the geometric parameters of each vascular segment. For the diameter parameter, the system uses a distance transformation method, taking the vascular centerline as a reference, to measure the distance from each point on the centerline to the edge of the vascular segment, and takes the average value as the diameter of the corresponding vascular segment. For the length parameter, the system calculates the cumulative length of the centerlines between adjacent nodes based on the positional information between nodes, and takes this as the length of the corresponding vascular segment. For the curvature parameter, the system uniformly samples the centerline of the vascular segment to obtain a sequence of sampling points, and derives the curvature value of each sampling point by calculating the change in the angle between the vectors formed by adjacent sampling points. The statistical result of the curvature values ​​of all sampling points is taken as the curvature parameter of the vascular segment. The system associates and stores the diameter, length, and curvature parameters of each vascular segment with the corresponding edges in the initial vascular topology map, generating a parameterized vascular topology map, thus achieving the fusion of vascular topology and geometric attributes.

[0082] S15: Perform risk factor calculation on the parameterized vascular topology map, quantify the difference in sub-vascular diameter at bifurcation points and calculate the risk value of angle abnormalities, and generate a dynamic vascular topology map.

[0083] Specifically, the system extracts the geometric parameters of sub-vessel segments at each bifurcation point in the parameterized vascular topology map, quantitatively analyzes the diameter differences between sub-vessel segments, and obtains the quantitative results of diameter differences by calculating the ratio of sub-vessel diameters. Simultaneously, the system measures the angle between the centerlines of each sub-vessel segment at the bifurcation point, compares it with a preset standard angle range, calculates the degree of angle deviation, and obtains an angle anomaly risk value. The system sets risk factor calculation rules, integrates the quantitative results of diameter differences with the angle anomaly risk value, and obtains a comprehensive risk factor for each bifurcation point.

[0084] Furthermore, the system associates and stores the comprehensive risk factors, the quantification results of tube diameter differences, the risk values ​​of angle anomalies, and the corresponding bifurcation points in the parameterized vascular topology map. For continuous digital subtraction angiography sequences, the system updates the geometric parameters and risk factors of the vascular topology map frame by frame, forming a dynamically updated vascular topology map. The dynamic vascular topology map reflects the geometric features and risk distribution changes of the vascular structure in real time during the operation.

[0085] In one embodiment, S2 of the reinforcement learning-based embolic agent infusion control method provided by the present invention specifically includes the following steps:

[0086] S21: Perform feature extraction processing on the dynamic vascular topology map, select the diameter, length, curvature and risk factors of the current target branch, and generate a vascular feature vector.

[0087] Specifically, the system uses a dynamic vascular topology map as its data foundation. First, it locates the current target branch through topological relationships, clarifying the connection position and range of the target branch within the vascular network. The system then extracts key geometric and risk features of the target branch from the dynamic vascular topology map, specifically including the branch's diameter, length, curvature, and risk factors. Further, the system combines the extracted feature parameters into a vascular feature vector in a preset order, with the vector construction following the formula:

[0088]

[0089] in, The average diameter of the target branch is obtained by statistically analyzing the distances from each point on the centerline of the target branch to the edge of the blood vessel. The actual length of the target branch is calculated by the cumulative distance between the centerlines of the nodes at both ends of the branch. The curvature parameter representing the target branch is derived by the change in the vector angle between the sampling points on the branch centerline; This represents the comprehensive risk factor at the bifurcation point corresponding to the target branch, calculated by integrating the quantified results of the diameter difference with the risk value of angle anomalies. The system uses this formula to complete the structured integration of feature parameters, generating a fixed-dimensional vascular feature vector.

[0090] S22: Acquire real-time measurement data from pressure sensors and flow meters, time-align the real-time measurement data, synchronously sample sensor readings within the time window, and generate timestamp-aligned sensor data.

[0091] Specifically, the system collects real-time pressure and flow data during the infusion process using pressure sensors and flow meters, respectively. The two types of sensors output continuous measurement data according to their respective sampling mechanisms, and their sampling frequencies differ. The system first extracts the acquisition timestamp of each sensor data point to determine a unified sampling time window, which covers the acquisition period of the current frame of digital subtraction angiography image. The system calculates the deviation between the timestamps of different sensor data and the sampling time window, and uses a linear interpolation method to achieve time alignment. The interpolation calculation follows the formula below:

[0092]

[0093] in, Indicates the target time within the aligned sampling time window. Sensor data; and Representing the target time respectively Two adjacent raw sensor data entries; and Corresponding to and The system performs the above interpolation calculation on all sensor data for each time period, synchronously sampling pressure and flow readings within the time window to generate timestamp-aligned sensor data.

[0094] S23: Normalize the sensor data, scaling the pressure and flow values ​​to a uniform range to generate a normalized sensor vector.

[0095] Specifically, the system receives timestamp-aligned sensor data, which includes two types of physical quantities: pressure measurements and flow measurements. The numerical ranges of these two types of physical quantities differ. Preferably, the system can use a Min-Max normalization method to scale the sensor data, ensuring that all data fall within a uniform numerical range. The normalization calculation follows the formula:

[0096]

[0097] in, This represents the normalized sensor data; This represents the raw sensor measurement data; This indicates the minimum value of the data from this type of sensor; This indicates the maximum value of this type of sensor data. The system performs the above normalization calculation on pressure and flow data separately to avoid the impact of differences in the numerical range of different physical quantities on subsequent decision-making processes. After the normalization process is completed, the system combines the normalized pressure and flow values ​​in a preset order to generate a normalized sensor vector with fixed dimensions, thereby achieving standardized processing of sensor data.

[0098] S24: Perform feature concatenation processing on the vascular feature vector and the normalized sensor vector, combine the multimodal data into a single vector, and generate a standardized state vector.

[0099] Specifically, the system first performs dimensionality verification on the vascular feature vector and the normalized sensor vector to confirm that the dimensions of the two types of vectors conform to the preset splicing rules, ensuring the consistency of the data structure. Preferably, the system can use a vector splicing method to integrate multimodal data, taking the vascular feature vector as the preceding part and the normalized sensor vector as the following part, and combining them sequentially into a single vector. The splicing process follows the formula below:

[0100]

[0101] in, Represents a standardized state vector; Represents the vascular feature vector; Represents the normalized sensor vector; This represents the normalized stress data; This represents the normalized flow data. The system uses this formula to perform multimodal fusion of vascular features and sensor data, integrating scattered multidimensional data into a structured single vector. The generated standardized state vector includes vascular geometric features, risk features, and real-time infusion parameters.

[0102] In one embodiment, step S3 of the reinforcement learning-based embolic agent infusion control method provided by the present invention specifically includes the following steps:

[0103] S31: The standardized state vector is processed by a hidden layer of a neural network. The higher-order features in the nonlinear combination of the state vector are extracted through a fully connected layer to generate a hidden feature vector.

[0104] Specifically, after receiving the standardized state vector, the system first verifies the vector dimension and data format to confirm that the data meets the requirements of neural network input. The system then calls a pre-defined fully connected layer network structure to process the standardized state vector. The fully connected layer uses weight matrices and bias terms to perform a non-linear transformation of the input vector, extracting higher-order features. Feature extraction follows the formula:

[0105]

[0106] in, Represents the hidden feature vector; This represents the weight matrix of the fully connected layer, used for linear weighting of the input features; Represents a standardized state vector; This represents the bias term of the fully connected layer, used to adjust the feature distribution; The activation function is used to introduce nonlinear transformations, enhancing the network's ability to fit complex features. The system iteratively extracts the nonlinear combinations of features of different dimensions from the standardized state vector through multiple fully connected layers, transforming low-dimensional original features into high-dimensional abstract features and generating a hidden feature vector with fixed dimensions.

[0107] S32: Perform action space mapping processing on the hidden feature vector, output a multi-dimensional vector in the continuous action space within a preset range of continuous values, and generate the original action vector.

[0108] Specifically, the system takes the hidden feature vector as input and performs action space mapping. This mapping process is implemented through the output layer of a neural network, transforming the high-dimensional hidden features into continuous multi-dimensional vectors that conform to a preset range. The action space mapping follows this formula:

[0109]

[0110] in, Represents the original action vector; This represents the weight matrix of the output layer; This represents the bias term of the output layer; This represents the output activation function, used to constrain the mapping result within a preset continuous range. The system uses this formula to complete the transformation from the feature space to the action space, ensuring that each component of the output original action vector is within the preset continuous interval, meeting the requirements of continuous action control. The dimension of the original action vector corresponds to the dimension of the parameters required by the control command, and each component corresponds to the original value of a control parameter.

[0111] S33: Perform instruction parsing processing on the original action vector, separate the pressure adjustment coefficient of the first dimension component and the branch selection probability of the second and third dimension components, linearly map the pressure coefficient to the physical pressure range and convert the branch probability into discrete index values, and generate control instructions containing pressure adjustment amount and target branch selection instructions.

[0112] Specifically, after receiving the original action vector, the system parses the instruction, first separating the components of each dimension of the original action vector. The first dimension is defined as the pressure adjustment coefficient, and the second and third dimensions are defined as the branch selection probabilities. For the pressure adjustment coefficient, the system converts it into a physical pressure adjustment amount through a linear mapping, which follows the formula:

[0113]

[0114] in, Indicates the amount of physical pressure adjustment; This indicates the current base pressure value; This represents the pressure adjustment coefficient in the original action vector. For the branch selection probability, the system determines the target branch based on the probability maximization principle, and the transformation follows the formula below:

[0115]

[0116] in, Represents the discrete index value of the target branch; and These represent the branch selection probabilities of the second and third dimensions in the original action vector, respectively. The system combines the physical pressure adjustment amount and the discrete index value of the target branch according to a preset protocol format, clarifies the structure and field meaning of the control command, and generates a control command containing the pressure adjustment amount and the target branch selection command, ensuring that the injection pump actuator can accurately parse and execute the corresponding control operation.

[0117] In one embodiment, such as Figure 2 As shown, S4 of the reinforcement learning-based embolic agent infusion control method provided by the present invention specifically includes the following steps:

[0118] S41: Perform branch location processing on the target branch selection instruction of the control command, index the geometric parameters of the target branch in the dynamic vascular topology map, and generate the target branch parameter set.

[0119] Specifically, the system receives a target branch selection instruction from the control commands, which includes the discrete index value of the target branch. Using this discrete index value as the retrieval basis, the system executes the branch localization process in the dynamic vascular topology map. By matching the index value with the branch number in the topology map, the system matches the specific location of the target branch in the vascular network, clarifying the starting node, ending node, and associated vascular segment range of the target branch. After localization, the system extracts the core geometric parameters of the target branch from the dynamic vascular topology map, including the branch's diameter, length, and curvature. These parameters are standardized geometric feature data already stored in the dynamic vascular topology map. The system then systematically integrates the extracted diameter, length, and curvature parameters according to a preset data structure to generate a target branch parameter set. This parameter set directly serves subsequent curvature compensation calculations, providing necessary geometric parameter support for flow equation correction and ensuring accurate matching between the compensation calculation and the actual structural features of the target branch.

[0120] S42: Perform curvature compensation calculations on the pressure adjustment amount of the control command and the target branch parameter set, apply the attenuation factor to correct the Poiseuille flow equation, and generate the target flow value.

[0121] Specifically, the system acquires the pressure adjustment amount and target branch parameter set from the control command, performs curvature compensation calculation, and, considering the influence of vascular curvature on blood flow, introduces a curvature attenuation factor to modify the traditional Poiseuille flow equation. The modified target flow value is calculated according to the following formula:

[0122]

[0123] in, The target branch's theoretical flow rate, i.e., the target flow rate value. For the target branch pipe diameter, For pressure adjustment amount, For blood viscosity, For the target branch length, The curvature attenuation coefficient, The target branch curvature is used. The system substitutes the pressure adjustment amount and the corresponding parameters from the target branch parameter set into the formula, and completes the calculation of the numerator and denominator and the final product through step-by-step calculation to generate the target flow rate value. This value takes into account the dual effects of pressure adjustment and vascular curvature to ensure that the flow rate calculation is consistent with the actual vascular environment.

[0124] S43: Perform control conversion processing on the target flow rate value, calculate the flow rate deviation and output proportional-integral-derivative control quantity, and generate the injection pump control voltage signal.

[0125] Specifically, after receiving the target flow rate value, the system synchronously acquires the real-time actual flow rate data fed back by the flow meter, performs control conversion processing, and first calculates the difference between the target flow rate value and the real-time actual flow rate data to obtain the flow deviation. To achieve precise flow control, the system can use a proportional-integral-derivative (PID) control algorithm to handle the flow deviation. The control quantity calculation follows the formula below:

[0126]

[0127] in, Represents the proportional-integral-derivative control quantity; Indicates flow deviation; This represents the proportionality coefficient. Represents the integral coefficient. These represent the differential coefficients; the three types of coefficients are used to adjust the control weights of the proportional, integral, and derivative components. The integral term representing the flow deviation, The differential term represents the flow deviation. The system obtains the proportional-integral-derivative control quantity by calculating the sum of the proportional, integral, and derivative terms. This control quantity is then linearly mapped to the driving voltage range of the syringe pump, generating a syringe pump control voltage signal, thus realizing the conversion from flow control requirements to voltage drive signals.

[0128] S44: Perform waveform modulation processing on the injection pump control voltage signal to generate a pulse width modulation signal, and generate the injection pump control signal.

[0129] Specifically, after receiving the control voltage signal from the infusion pump, the system performs waveform modulation processing. First, it analyzes the amplitude of the control voltage signal and determines the duty cycle of the pulse width modulation (PWM) signal based on the amplitude. The duty cycle and voltage signal amplitude have a linear relationship. The system sets a fixed carrier frequency to ensure stable driving of the infusion pump's stepper motor. Based on the carrier frequency and the determined duty cycle, the system generates the PWM signal. The PWM signal is presented in an alternating high and low level format. The duration ratio of the high and low levels is determined by the duty cycle. This ratio is used to adjust the effective voltage value per unit time to match the driving requirements of the infusion pump. The system performs format verification on the generated PWM signal to ensure that the signal level standard, frequency, and duty cycle meet the receiving requirements of the infusion pump actuator. After successful verification, the system generates the infusion pump control signal, which can be directly transmitted to the infusion pump actuator to drive it to complete the corresponding infusion action.

[0130] In one embodiment, step S5 of the reinforcement learning-based embolic agent infusion control method provided by the present invention specifically includes the following steps:

[0131] S51: Perform high-risk region identification processing on the dynamic vascular topology map, screen vascular branches in the vascular branch set whose risk factors exceed the preset risk threshold, and generate a non-target region coordinate set.

[0132] Specifically, the system uses a dynamic vascular topology map as its analysis basis, traversing all branches in the vascular branch set according to branch number order, and extracting the stored comprehensive risk factors for each branch through a topological association index. The comprehensive risk factors are obtained by integrating the quantified results of diameter differences calculated in previous steps with the risk values ​​of angle anomalies according to preset rules. Further, the system compares the comprehensive risk factors of each branch with preset risk thresholds one by one, and the determination follows the formula:

[0133]

[0134] in, This represents the comprehensive risk factor for the i-th vascular branch, which is obtained by weighting the risks of branch diameter differences and angle abnormalities. This indicates a preset risk threshold, set based on the criteria for identifying high-risk branches in clinical anatomy. The system performs a coordinate extraction process on high-risk vascular branches that meet the criteria. First, it obtains the coordinates of uniformly sampled points along the branch's centerline. Then, it derives the boundary coordinates of the vessel edge through distance transformation, integrating the sampled point coordinates and boundary coordinates to form complete branch region coordinate data. The system categorizes and archives the coordinate data according to the high-risk branch number, clearly defining the spatial boundaries and extension range of each high-risk branch, and generating a structured set of non-target region coordinates.

[0135] S52: Perform embolization agent segmentation processing on the acquired digital subtraction angiography images after embolization, identify the diffusion area of ​​the contrast agent in the three-dimensional vascular space, and generate an embolization area mask.

[0136] Specifically, the system receives digital subtraction angiography images after embolization, first filters the images to reduce random noise, and then enhances the grayscale distinction between the embolic agent and surrounding vascular tissue through grayscale transformation. Preferably, the system can use an adaptive threshold segmentation algorithm to process the preprocessed images pixel by pixel to identify the diffusion area of ​​the embolic agent in the three-dimensional vascular space. The segmentation follows the formula:

[0137]

[0138] in, The pixel identifier value represents the mask of the embolization area; The three-dimensional pixel coordinates of the image are represented by the three-dimensional spatial coordinates obtained through inter-layer interpolation of the image sequence. This represents the grayscale measurement value of the corresponding three-dimensional coordinate point; The grayscale range corresponding to the embolic agent is determined based on the X-ray attenuation characteristics of the embolic agent. The system performs a judgment operation on all three-dimensional pixels in the image according to the above formula, assigning a first identifier value to pixels covered by the embolic agent and a second identifier value to pixels in non-embolic agent areas, generating a three-dimensional embolization area mask. This mask completely records the spatial distribution range and morphological characteristics of the embolic agent in the vascular network.

[0139] S53: Perform overlap analysis on the coordinate set of the non-target area and the mask of the embolization area, calculate the proportion of the embolic agent coverage area in the non-target area, and generate the false embolization index.

[0140] Specifically, the system first calls the calibration parameters of the digital subtraction angiography equipment to perform spatial coordinate alignment between the non-target region coordinate set and the embolization region mask, ensuring that both are analyzed based on the same spatial coordinate system. The system traverses all non-target regions according to the branch numbering order in the non-target region coordinate set, calculates the number of overlapping pixels between each non-target region and the embolization region mask using a pixel counting method, and then calculates the overlapping area through pixel size conversion. Simultaneously, the total area of ​​each non-target region is calculated. The calculation of the false embolism index follows the following formula:

[0141]

[0142] in, Indicates indicators of accidental embolism; The area of ​​overlap between the i-th non-target region and the embolism region is obtained by multiplying the number of overlapping pixels by the area of ​​a single pixel. The total area of ​​the i-th non-target region is calculated geometrically using the boundary coordinates of the non-target regions; n represents the total number of non-target regions, determined by counting the number of branch numbers in the coordinate set of the non-target regions. The system uses this formula to integrate and calculate the overlap ratio of all non-target regions, generating a single quantitative index for embolization errors. This index directly reflects the overall degree to which embolic agents have entered non-target regions.

[0143] S54: Perform target embolization calculation processing on the embolization area mask, quantify the embolic agent coverage in the target area, and generate the target embolization increment.

[0144] Specifically, the system locates the target region corresponding to the target branch based on the target branch identifier in the dynamic vascular topology map, extracts the spatial coordinate boundary of the target region through topological correlation, and then calculates the total area of ​​the target region. The system matches the embolization region mask with the spatial coordinate boundary of the target region, uses a pixel counting method to count the number of pixels covered by the embolic agent within the target region, calculates the area covered by the embolic agent within the target region, and then calculates the current target region embolization coverage rate by the ratio of the covered area to the total area of ​​the target region. The system retrieves the target region embolization coverage rate data from the historical database according to the control cycle sequence, and calculates the target region embolization increment by the difference between the current coverage rate and the historical coverage rate, following the formula:

[0145]

[0146] in, Indicates the incremental embolism in the target area; It represents the current target area embolization coverage rate, calculated as the ratio of the area covered by embolic agent in the current target area to the total area of ​​the target area; This represents the target area embolism coverage rate of the previous control cycle, retrieved from the system's stored historical control data by timestamp index. The system uses this formula to quantify the improvement in target area embolism effect after the current control command is executed, generating incremental target area embolism data for reward calculation.

[0147] S55: Perform reward function calculation on the false embolism index, target area embolism increment, and pressure adjustment in the control command, and weighted sum the safety penalty term, efficiency reward term, and stability penalty term to generate the policy gradient reward value for updating the neural network.

[0148] Specifically, the system constructs a multi-dimensional reward function that comprehensively considers the safety, efficiency, and stability of the embolization process. The calculation follows the formula below: (The formula is not provided in the original text.)

[0149]

[0150] in, This represents the policy gradient reward value; The weighting coefficient for safety penalty items is set based on safety priorities during clinical surgery; The weighting coefficient for the efficiency reward item is set based on the treatment efficiency requirements of target area embolization; The weighting coefficient for the stability penalty term is set according to the pressure stability requirements of the infusion process; This indicates the accidental embolization index; the higher the value, the greater the safety penalty. This represents the increase in target area embolism; the larger the value, the greater the efficiency bonus. This represents the pressure adjustment amount in the control command; the larger its absolute value, the greater the stability penalty. The system performs weighted calculations of each component according to the above formula to obtain a policy gradient reward value that reflects the quality of the current control strategy. The system sends this reward value to the neural network through a data transmission channel, triggering the network's policy gradient parameter update process and realizing iterative optimization of the control strategy.

[0151] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0152] Based on the same inventive concept, this application also provides a reinforcement learning-based embolization agent infusion control device for implementing the reinforcement learning-based embolization agent infusion control method described above. The solution provided by this device is similar to the solution described in the above method; therefore, the specific limitations of one or more reinforcement learning-based embolization agent infusion control device embodiments provided below can be found in the limitations of the reinforcement learning-based embolization agent infusion control method described above, and will not be repeated here.

[0153] Preferably, such as Figure 3 As shown, the present invention provides a reinforcement learning-based embolic agent infusion control system 600, which is configured with the following modules:

[0154] The vascular topology modeling module 610 is used to perform vascular topology modeling on the acquired intraoperative digital subtraction angiography sequence, construct the vascular network diagram structure, calculate the bifurcation risk and curvature parameters, and generate a dynamic vascular topology diagram.

[0155] The multimodal data fusion module 620 is used to acquire real-time measurement data from pressure sensors and flow meters, combine them with dynamic vascular topology maps for multimodal fusion, align timestamps and normalize vascular features and sensor readings to generate standardized state vectors.

[0156] The reinforcement learning decision module 630 is used to perform reinforcement learning decisions on the standardized state vector. It calls the forward propagation state vector of the neural network to generate a continuous action space mapping and decomposes the action vector into a pressure adjustment coefficient and a branch selection probability distribution. It outputs a control command containing a pressure adjustment amount and a target branch selection instruction.

[0157] The curvature-compensated flow calculation and control signal generation module 640 is used to perform curvature-compensated flow calculation processing on the pressure adjustment amount, target branch selection instruction and dynamic vascular topology map in the control command. It combines the preset vascular geometric characteristics, applies the curvature attenuation factor to correct the Poiseuille equation in the target branch, calculates the target flow value and converts it into a voltage control waveform, generates the infusion pump control signal and sends the infusion pump control signal to the infusion pump actuator.

[0158] The embolization detection and strategy update module 650 is used to acquire digital subtraction angiography images after embolization, combine them with dynamic vascular topology maps for embolization detection, segment non-target regions and calculate the overlap with the embolic agent distribution, and update the strategy gradient reward value of the neural network. In one embodiment, this application also provides a computer device, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the above-described reinforcement learning-based embolic agent infusion control method.

[0159] Preferably, the vascular topology modeling module 610 provided in this application is configured with the following units:

[0160] The vessel segmentation unit is used to perform vessel segmentation processing on the intraoperative digital subtraction angiography sequence acquired by medical imaging equipment, distinguish the vascular tissue from the background region in each frame of the image, and generate a binary mask image.

[0161] The vascular skeletonization unit is used to skeletonize the binary mask image, extract the centerline structure of a single pixel width in the binary mask, and generate a vascular centerline map.

[0162] The vascular topology initialization unit is used to perform graph structure transformation on the vascular centerline diagram, mark the bifurcation points and endpoints in the centerline network and connect adjacent nodes to generate the initial vascular topology diagram.

[0163] The vascular geometry parameter calculation unit is used to calculate the geometric parameters of the initial vascular topology map, measure the diameter, length and curvature of each vascular segment, and generate a vascular topology map with parameters.

[0164] The vascular risk factor calculation unit is used to perform risk factor calculation on the parameterized vascular topology map, quantify the differences in the diameter of sub-vessels at bifurcation points and calculate the risk value of angle abnormalities, and generate a dynamic vascular topology map.

[0165] Preferably, the multimodal data fusion module 620 provided in this application is configured with the following units:

[0166] The vascular feature extraction unit is used to perform feature extraction processing on the dynamic vascular topology map, select the diameter, length, curvature and risk factors of the current target branch, and generate vascular feature vectors.

[0167] The sensor data time alignment unit is used to acquire real-time measurement data from pressure sensors and flow meters, align the real-time measurement data to time, synchronize sensor readings within the sampling time window, and generate timestamp-aligned sensor data.

[0168] The sensor data normalization unit is used to normalize sensor data, scale pressure and flow values ​​to a uniform range, and generate a normalized sensor vector.

[0169] The multimodal feature stitching unit is used to perform feature stitching processing on the vascular feature vector and the normalized sensor vector, combining the multimodal data into a single vector to generate a standardized state vector.

[0170] Preferably, the reinforcement learning decision module 630 provided in this application is configured with the following units:

[0171] The neural network high-order feature extraction unit is used to process the standardized state vector through the neural network hidden layer. It extracts high-order features from the nonlinear combination of the state vector through the fully connected layer to generate hidden feature vectors.

[0172] The action space mapping unit is used to perform action space mapping processing on the hidden feature vectors, outputting a multi-dimensional vector in the continuous action space within a preset range of continuous values, and generating the original action vector.

[0173] The control command parsing and generation unit is used to parse the original action vector, separate the pressure adjustment coefficient of the first dimension component and the branch selection probability of the second and third dimension components, linearly map the pressure coefficient to the physical pressure range and convert the branch probability into discrete index values, and generate control commands containing pressure adjustment amount and target branch selection command.

[0174] Preferably, the curvature compensation flow calculation and control signal generation module 640 provided in this application is configured with the following units:

[0175] The target branch localization unit is used to perform branch localization processing on the target branch selection instruction of the control command, index the geometric parameters of the target branch in the dynamic vascular topology map, and generate the target branch parameter set.

[0176] The curvature-compensated flow calculation unit is used to perform curvature compensation calculation on the pressure adjustment amount of the control command and the target branch parameter set, and to apply the attenuation factor to correct the Poiseuille flow equation to generate the target flow value.

[0177] The PID control quantity conversion unit is used to control and convert the target flow value, calculate the flow deviation and output the proportional-integral-derivative control quantity, and generate the injection pump control voltage signal.

[0178] The PWM waveform modulation unit is used to perform waveform modulation processing on the injection pump control voltage signal to generate a pulse width modulation signal and thus generate the injection pump control signal.

[0179] Preferably, the false thrombosis monitoring and strategy update module 650 provided in this application is configured with the following units:

[0180] The non-target region identification unit is used to identify high-risk regions in the dynamic vascular topology map, filter out vascular branches in the vascular branch set whose risk factors exceed a preset risk threshold, and generate a non-target region coordinate set.

[0181] The embolization agent region segmentation unit is used to perform embolization agent segmentation processing on the acquired digital subtraction angiography images after embolization, identify the diffusion area of ​​the contrast agent in the three-dimensional vascular space, and generate an embolization region mask.

[0182] The unit for calculating the false embolism index is used to perform overlapping analysis on the coordinate set of the non-target area and the mask of the embolism area, calculate the proportion of the embolic agent coverage area in the non-target area, and generate the false embolism index.

[0183] The target embolization increment calculation unit is used to perform target embolization calculation processing on the embolization area mask, quantify the embolization agent coverage in the target area, and generate the target embolization increment.

[0184] The strategy gradient reward value generation unit is used to calculate the reward function for the false embolism index, the target area embolism increment, and the pressure adjustment in the control command. It weights and sums the safety penalty term, efficiency reward term, and stability penalty term to generate the strategy gradient reward value for updating the neural network.

[0185] In one embodiment, this application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the above-described reinforcement learning-based embolic agent infusion control method.

[0186] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of those different embodiments or examples.

[0187] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The components described as separate parts may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this disclosure according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0188] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various variations or substitutions within the technical scope disclosed in this application, and these should all be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A reinforcement learning-based method for controlling embolic agent infusion, characterized in that, Includes the following steps: S1: Perform vascular topology modeling on the acquired intraoperative digital subtraction angiography sequence, construct the vascular network diagram structure, calculate the bifurcation risk and curvature parameters, and generate a dynamic vascular topology diagram. S2: Acquire real-time measurement data from pressure sensors and flow meters, combine the dynamic vascular topology map for multimodal fusion, align timestamps and normalize vascular features and sensor readings to generate a standardized state vector; S3: Perform reinforcement learning decision-making on the standardized state vector, call the forward propagation state vector of the neural network to generate a continuous action space mapping, decompose the action vector into pressure adjustment coefficient and branch selection probability distribution, and output a control command containing pressure adjustment amount and target branch selection instruction; S4: Perform curvature compensation flow calculation processing on the pressure adjustment amount, target branch selection instruction and dynamic vascular topology map in the control command, apply curvature attenuation factor to correct the Poiseuille equation in the target branch in combination with the preset vascular geometric characteristics, calculate the target flow value and convert it into a voltage control waveform, generate an injection pump control signal and send the injection pump control signal to the injection pump actuator. S5: Acquire digital subtraction angiography images after embolization, combine them with the dynamic vascular topology map for false embolization monitoring, segment non-target areas and calculate the overlap with the distribution of embolic agent, and update the policy gradient reward value of the neural network.

2. The method according to claim 1, characterized in that, S1 includes: S11: Based on the intraoperative digital subtraction angiography sequence acquired by medical imaging equipment, perform vascular segmentation processing on the digital subtraction angiography sequence, distinguish vascular tissue from background areas in each frame of image, and generate a binary mask image. S12: Perform skeletonization processing on the binary mask image, extract the centerline structure of a single pixel width in the binary mask, and generate a blood vessel centerline map; S13: Perform graph structure transformation on the blood vessel centerline diagram, mark the bifurcation points and endpoints in the centerline network and connect adjacent nodes to generate an initial blood vessel topology diagram; S14: Calculate the geometric parameters of the initial vascular topology map, measure the diameter, length and curvature of each vascular segment, and generate a vascular topology map with parameters; S15: Perform risk factor calculation on the parameterized vascular topology map, quantify the difference in sub-vascular diameter at bifurcation points and calculate the risk value of angle abnormality, and generate a dynamic vascular topology map.

3. The method according to claim 1, characterized in that, S2 includes: S21: Perform feature extraction processing on the dynamic vascular topology map, select the diameter, length, curvature and risk factors of the current target branch, and generate a vascular feature vector; S22: Acquire real-time measurement data from the pressure sensor and flow meter, time-align the real-time measurement data, synchronously sample sensor readings within the time window, and generate timestamp-aligned sensor data; S23: Normalize the sensor data, scale the pressure and flow values ​​to a uniform range, and generate a normalized sensor vector. S24: Perform feature concatenation processing on the blood vessel feature vector and the normalized sensor vector to combine the multimodal data into a single vector and generate a standardized state vector.

4. The method according to claim 1, characterized in that, S3 includes: S31: The standardized state vector is processed by a neural network hidden layer, and high-order features in the nonlinear combination of the state vector are extracted through a fully connected layer to generate a hidden feature vector; S32: Perform action space mapping processing on the hidden feature vector, output a multi-dimensional vector in the continuous action space within a preset range of continuous values, and generate the original action vector; S33: Perform instruction parsing processing on the original action vector, separate the pressure adjustment coefficient of the first dimension component and the branch selection probability of the second and third dimension components, linearly map the pressure coefficient to the physical pressure range and convert the branch probability into discrete index values, and generate a control instruction containing the pressure adjustment amount and the target branch selection instruction.

5. The method according to claim 1, characterized in that, S4 includes: S41: Perform branch positioning processing on the target branch selection instruction of the control instruction, index the geometric parameters of the target branch in the dynamic vascular topology map, and generate a target branch parameter set; S42: Perform curvature compensation calculation on the pressure adjustment amount of the control command and the target branch parameter set, apply the attenuation factor to correct the Poiseuille flow equation, and generate the target flow value; S43: Perform control conversion processing on the target flow rate value, calculate the flow rate deviation and output proportional-integral-derivative control quantity, and generate injection pump control voltage signal; S44: Perform waveform modulation processing on the injection pump control voltage signal to generate a pulse width modulation signal, and generate an injection pump control signal.

6. The method according to claim 5, characterized in that, The formula for calculating the target flow rate is: in, The target branch's theoretical flow rate, i.e., the target flow rate value. For the target branch pipe diameter, For pressure adjustment amount, For blood viscosity, For the target branch length, The curvature attenuation coefficient, The target branch curvature.

7. The method according to any one of claims 1-6, characterized in that, S5 includes: S51: Perform high-risk region identification processing on the dynamic vascular topology map, screen out vascular branches in the vascular branch set whose risk factors exceed a preset risk threshold, and generate a non-target region coordinate set. S52: Perform embolization agent segmentation processing on the acquired digital subtraction angiography images after embolization, identify the diffusion area of ​​the contrast agent in the three-dimensional vascular space, and generate an embolization area mask. S53: Perform overlap analysis on the coordinate set of the non-target area and the mask of the embolization area, calculate the proportion of the embolic agent coverage area in the non-target area, and generate the false embolization index. S54: Perform target embolization calculation processing on the embolization area mask, quantify the embolization agent coverage rate in the target area, and generate the target embolization increment; S55: Perform reward function calculation on the false embolism index, the target area embolism increment, and the pressure adjustment amount in the control command, and weight sum the safety penalty term, efficiency reward term, and stability penalty term to generate the policy gradient reward value for updating the neural network.

8. A reinforcement learning-based embolic agent infusion control system, characterized in that, The system includes: The vascular topology modeling module is used to perform vascular topology modeling on the acquired intraoperative digital subtraction angiography sequence, construct the vascular network diagram structure, calculate the bifurcation risk and curvature parameters, and generate a dynamic vascular topology diagram. The multimodal data fusion module is used to acquire real-time measurement data from pressure sensors and flow meters, combine them with the dynamic vascular topology map to perform multimodal fusion, align timestamps and normalize vascular features and sensor readings to generate a standardized state vector. The reinforcement learning decision module is used to perform reinforcement learning decisions on the standardized state vector, call the forward propagation state vector of the neural network to generate a continuous action space mapping and decompose the action vector into a pressure adjustment coefficient and a branch selection probability distribution, and output a control command containing a pressure adjustment amount and a target branch selection instruction. The curvature-compensated flow calculation and control signal generation module is used to perform curvature-compensated flow calculation processing on the pressure adjustment amount, target branch selection instruction and dynamic vascular topology map in the control command, and apply the curvature attenuation factor to correct the Poiseuille equation in the target branch in combination with the preset vascular geometric characteristics, calculate the target flow value and convert it into a voltage control waveform, generate an injection pump control signal and send the injection pump control signal to the injection pump actuator. The embolization detection and strategy update module is used to acquire digital subtraction angiography images after embolization, combine them with the dynamic vascular topology map to detect embolization, segment non-target areas and calculate the overlap with the distribution of embolic agents, and update the strategy gradient reward value of the neural network.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the method of any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 7.

Images