Radiation diagnosis and treatment safety early warning system and method based on multi-modal data fusion

Abstract

The invention relates to the technical field of radiotherapy and medical data processing, in particular to a radiotherapy safety early warning system and method based on multi-modal data fusion. Comprising the following steps: a multi-modal acquisition step: synchronously acquiring a static plan, a dynamic log and a real-time monitoring stream; an ideal field reconstruction step: reconstructing an ideal reference field by using the log-driven plan; a fault simulation step: injecting an interference operator to generate a theoretical fault state space; a double-track difference step: respectively extracting a real residual feature library and a theoretical residual feature library; a risk positioning step: calculating topological similarity to generate an auxiliary decision report; according to the method, a double-track differential mechanism is used for eliminating time asynchronization errors, and high-sensitivity recognition of tiny mechanical faults is achieved under the physiological noise background.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of radiotherapy and medical data processing, in particular to a radiotherapy safety warning system and method based on multi-modal data fusion. BACKGROUND

[0002] The existing radiotherapy system is usually equipped with a safety warning device for monitoring the treatment implementation process; in the related art, in order to detect whether there is a device failure or an execution error in the radiotherapy process, a simple threshold monitoring of the device parameters can be performed by using the mechanical log data; or, a direct comparison between the real-time monitoring image and the static planning image before treatment is performed to determine whether there is an abnormality; however, in the actual treatment scene containing a large amount of physiological motion noise, the sensitivity of the former to the minor faults is low, and the latter is limited by the time synchronization problem, and the false positive rate of the detection is high; therefore, the radiotherapy safety warning method in the related art cannot balance the high sensitivity identification and the low false positive rate control. SUMMARY

[0003] The present application relates to the technical field of radiotherapy and medical data processing, in particular to a radiotherapy safety warning system and method based on multi-modal data fusion. The radiotherapy safety warning system based on multi-modal data fusion comprises: A multi-modal data acquisition component, which is adapted to acquire static treatment planning data, dynamic mechanical execution log data and real-time monitoring stream data; An ideal physical field reconstruction engine, which is adapted to drive an anatomical model in the static treatment planning data by using the dynamic mechanical execution log data, and reconstruct time-varying ideal reference field data by a physical transmission algorithm; A fault state space simulation component, which is adapted to call a preset fault mode knowledge base, inject a parameterized disturbance operator into the ideal reference field data, and generate theoretical fault state space data; A dual-track difference extraction component, which is adapted to calculate the difference between the real-time monitoring stream data and the ideal reference field data to generate real residual features, and calculate the difference between the theoretical fault state space data and the ideal reference field data to generate a theoretical residual feature library; A risk quantification and positioning component, which is adapted to calculate the topological similarity of each element in the real residual features and the theoretical residual feature library, and generate a risk quantification index and corresponding potential fault type positioning information based on the topological similarity to form an auxiliary decision report.

[0004] Preferably, the ideal physical field reconstruction engine comprises: a virtual radiotherapy driving unit, the virtual radiotherapy driving unit is adapted to parse the millisecond-level mechanical state parameters in the dynamic mechanical execution log data, and map the mechanical state parameters to a spatial coordinate system defined by the static diagnosis and treatment plan data; a dose calculation unit, the dose calculation unit is adapted to calculate the theoretical dose distribution in the target object and the theoretical projection image on the body surface based on the mapped spatial coordinate system by using a phase space-based convolution superposition algorithm or a ray tracing algorithm, to form the ideal reference field data.

[0005] Preferably, the fault state space simulation component comprises: a parameter definition unit, the parameter definition unit is adapted to convert common mechanical failure modes and positioning error modes into mathematical spatial transformation matrices or flux shielding functions; a manifold generation unit, the manifold generation unit is adapted to apply the spatial transformation matrices or flux shielding functions to the ideal reference field data respectively, to simulate and generate a virtual image data set corresponding to different fault types, to form the theoretical fault state space data.

[0006] Preferably, the risk quantification and positioning component comprises: a feature space mapping unit, the feature space mapping unit is adapted to map the real residual features and the theoretical residual feature library to a high-dimensional topological feature space; a similarity calculation and matching unit, the similarity calculation and matching unit is adapted to calculate the cosine similarity or topological homology group features of the real residual features in the high-dimensional topological feature space with the theoretical residual feature library, and match the theoretical fault mode with the highest similarity; a report generation unit, the report generation unit is adapted to generate a visual auxiliary decision report containing risk quantification indicators, suspected fault types and their probabilities, and residual topological mode analysis according to the matching results.

[0007] Preferably, it further comprises: an auxiliary control suggestion component, the auxiliary control suggestion component is adapted to generate device control parameter adjustment suggestions or process intervention suggestions in response to the auxiliary decision report; a visual interaction component, the visual interaction component is adapted to display the auxiliary decision report and the control parameter adjustment suggestions on the operation interface, and highlight the suspected fault area and the residual topological mode.

[0008] Preferably, the multi-modal data acquisition component is configured to: obtain static diagnosis and treatment plan data containing CT images and structure delineation information through a hospital information system interface; Real-time reading of dynamic mechanical execution log data containing multi-leaf collimator position and gantry angle through the linear accelerator control interface; Real-time monitoring flow data is collected through an electronic portal imaging device or an optical surface monitoring system.

[0009] The radiation diagnosis and treatment safety early warning method based on multi-modal data fusion comprises the following steps: Obtain the static diagnosis and treatment plan data of the target object, and real-time collect the dynamic mechanical execution log data and real-time monitoring flow data of the treatment equipment; Drive the static diagnosis and treatment plan data based on the dynamic mechanical execution log data, and reconstruct the ideal reference field data at the current time; Superimpose the preset fault interference operator on the ideal reference field data to simulate and generate theoretical fault state space data containing multiple fault modes; Calculate the difference between the real-time monitoring flow data and the ideal reference field data, and the difference between the theoretical fault state space data and the ideal reference field data, respectively, to obtain the real residual feature and the theoretical residual feature library; Compare the topological morphology of the real residual feature and the theoretical residual feature library to generate an auxiliary decision report containing risk quantification indicators and potential fault positioning.

[0010] Preferably, comparing the topological morphology of the real residual feature and the theoretical residual feature library comprises: Using the synthetic analysis method, search for the target fault fingerprint with the maximum topological similarity to the real residual feature in the theoretical residual feature library; If the maximum topological similarity exceeds the preset judgment threshold, it is marked as a high-probability systematic risk in the auxiliary decision report, and is associated with a specific fault mode; If the maximum topological similarity does not exceed the preset judgment threshold, it is classified as a low-risk difference in the auxiliary decision report, which may be caused by physiological fluctuations or random noise.

[0011] Compared with the prior art, the present application has the following beneficial effects: 1、The present application realizes the digital twin reconstruction of the radiotherapy process by the collaborative work of the ideal physical field reconstruction engine and the virtual radiotherapy driving unit, using the dynamic mechanical execution log data to drive the static diagnosis and treatment plan, which can eliminate the system error caused by the asynchronization of the real-time monitoring flow data and the static plan time through the time domain integral synchronization mechanism and the coordinate transformation operation; By introducing the golden standard phase space file and the source model periodic calibration mechanism, combining the chemometric calibration method and the detector response convolution, an ideal reference field with energy conservation and time consistency is constructed, which provides an accurate zero reference for subsequent difference calculation, thereby effectively solving the high false alarm rate problem caused by the mismatch of physical parameters in the traditional direct comparison method.

[0012] 2、The application introduces a systematic grid sampling strategy based on orthogonal basis and Fibonacci sphere algorithm through the construction of fault state space simulation components, establishes theoretical fault state space data covering arbitrary direction micro-offset and flux shielding, can convert complex fault detection problems into structured feature matching problems;Through the injection of parameterized interference operator and the generation process of full permutation traversal, the completeness of the fault dictionary is ensured, so that the system can find highly consistent bases in the theoretical library even when facing diagonal or small amplitude mechanical faults, thereby overcoming the missed detection defects caused by single random sampling strategy, and significantly improving the recognition sensitivity of small mechanical faults.

[0013] 3、The application establishes a signal separation mechanism based on synthetic analysis method through the deep integration of double-track differential extraction components and risk quantification positioning components, uses persistent homology barcode and Wasserstein distance measurement in high-dimensional topological feature space to realize morphological analysis of residual features;Through adaptive fusion measurement and directional rectification mechanism, mechanical fault signals with specific topological structure and physiological motion noise without structural features can be effectively distinguished, and in complex diagnosis and treatment scenes containing a large amount of respiratory motion background, hidden fault fingerprints can still be captured sensitively, realizing low false alarm rate control and high confidence decision under physiological noise interference.

[0014] 4、The application realizes full-process coverage from physical perception to decision intervention through the hardware interface configuration of the multi-modal data acquisition component and the logic closed loop of the auxiliary control suggestion component, can synchronously input static anatomical structure, dynamic mechanical parameters and real-time image feedback in real time;Through the heat map superposition of the visual interaction component and the risk probability radar chart display, abstract algorithm results are converted into intuitive visual signals, and combined with the automatic interlocking trigger mechanism based on dynamic threshold, the decision-making time of the clinician is greatly shortened, ensuring that correct intervention measures can be taken quickly in the moment when systematic mechanical faults or positioning errors are identified, and the life safety of patients in the process of radiotherapy is ensured. BRIEF DESCRIPTION OF DRAWINGS

[0015] The application will be further explained in conjunction with the drawings and examples: Figure 1 is a structural diagram of the system of the application, Figure 2 is a flowchart of the method of the application. DETAILED DESCRIPTION

[0016] In order to make the purpose, technical scheme and advantages of the application clearer and more apparent, the application will be further described in detail in conjunction with specific examples.

[0017] Example 1: Please refer to Figure 1, a radiotherapy safety early warning system based on multi-modal data fusion, comprising: a multi-modal data acquisition component, the multi-modal data acquisition component is adapted to acquire static treatment plan data, dynamic mechanical execution log data and real-time monitoring stream data; an ideal physical field reconstruction engine, the ideal physical field reconstruction engine is adapted to drive an anatomical model in the static treatment plan data with the dynamic mechanical execution log data, and reconstruct an ideal reference field data changing over time through a physical transmission algorithm; a fault state space simulation component, the fault state space simulation component is adapted to call a preset fault mode knowledge base, inject a parameterized disturbance operator into the ideal reference field data, to generate theoretical fault state space data; a dual-track differential extraction component, the dual-track differential extraction component is adapted to calculate the difference between the real-time monitoring stream data and the ideal reference field data to generate real residual features, and calculate the difference between the theoretical fault state space data and the ideal reference field data to generate a theoretical residual feature library; a risk quantification and positioning component, the risk quantification and positioning component is adapted to calculate the topological similarity of each element in the real residual feature and the theoretical residual feature library, and generate risk quantification indicators and corresponding potential fault type positioning information based on the topological similarity, to form an auxiliary decision report.

[0018] The embodiment details the core operation mechanism of the above-mentioned system, which is based on a synthetic analysis method architecture and aims to solve the problem of physiological noise and mechanical fault signal superposition in radiotherapy; the system starts the multi-modal data acquisition component, which acts as a perception front end and synchronously imports three types of heterogeneous data: static treatment plan data from DICOM-RT files prepared before treatment , dynamic mechanical execution log data from real-time control feedback of the linear accelerator , and real-time monitoring stream data from the image device during beam delivery ; The ideal physical field reconstruction engine does not directly trust the static plan, but performs a digital twin construction process, drives the anatomical model in with real mechanical parameters in , and calculates the ideal reference field data that should theoretically be present at the current time through a physical transmission algorithm ; the fault state space simulation component intervenes, converts expert experience in the preset fault mode knowledge base into mathematical operators, injects a parameterized disturbance operator , , thereby generating theoretical fault state space data covering multiple potential fault modes ; On this basis, the dual-track differential extraction component performs signal separation operations, on the one hand, calculates and The difference is used to obtain realistic residual characteristics that include a mixture of noise and fault signals. On the other hand, calculation and To obtain a pure theoretical residual feature library by considering the differences. The risk quantification and positioning component performs topology-level pattern matching and calculates... and The topological similarity of each element is used to generate risk quantification indicators and locate potential fault types. This embodiment introduces a dynamically reconstructed ideal reference field. As a differential zero, it effectively eliminates system errors caused by time asynchrony; at the same time, by using the dual-track differential mechanism, the complex fault detection problem is transformed into a morphological matching problem of residual features. Even in complex scenarios where real-world monitoring data contains a large amount of physiological noise such as respiratory movements, the system can still keenly capture the topological structure hidden in the noise background that highly matches the theoretical fault fingerprint. Thus, in high-risk scenarios of radiology diagnosis and treatment, it achieves high-sensitivity identification and low false alarm rate control of minor mechanical faults.

[0019] The ideal physical field reconstruction engine includes: a virtual radiotherapy driving unit, which is adapted to parse millisecond-level mechanical state parameters in dynamic mechanical execution log data and map the mechanical state parameters to a spatial coordinate system defined by static treatment plan data; and a dose calculation unit, which is adapted to use a phase space-based convolution superposition algorithm or ray tracing algorithm to calculate the theoretical dose distribution within the target object and the theoretical projection image on the body surface based on the mapped spatial coordinate system, so as to form the ideal reference field data.

[0020] This embodiment further refines the internal logic of the ideal physics field reconstruction engine, particularly enhancing the time synchronization and energy conservation of the physics field, and clarifying the implementation details of the core algorithm; the virtual radiotherapy driving unit not only parses single-frame logs but also executes a time-domain integration synchronization mechanism: identifying the current real-time monitoring stream data. For example, EPD imagery, its physical integration time window For example, if the image acquisition frame rate is 10Hz, the window width is 100ms, and the serialized machine log data packets are extracted within this window. For each discrete time in the sequence The coordinate transformation operation is performed, and its calculation formula is as follows:

[0021] in, This is the original position vector in the local coordinate system of the component (i.e., the defined position of the blade endpoint of the multi-leaf collimator or the feature point of the ballast in the head coordinate system). is the target position vector in global isocenter coordinate system after transformation; is the rotation matrix based on gantry angle lookup table at time ; is the gravity sag correction vector, which is derived from factory physical correction table, which is stored as discrete key-value pairs , covering gantry angles from 0 to 360 degrees every 10 degrees; the unit of vector value is cm; when the angle is not covered, linear interpolation is used to obtain, define as the maximum scale angle not greater than , as the minimum scale angle not less than , calculate the weight , and use the interpolation formula for calculation; The dose calculation unit calculates the three-dimensional dose distribution in the target object based on the beam flux reconstructed by the dynamic log using pencil beam convolution superposition algorithm or folding cone convolution algorithm ; in order to compare with real-time monitoring flow, the algorithm does not calculate a single moment field, but performs riemann sum integration on the instantaneous field in the window to generate cumulative flux distribution , the calculation formula is:

[0022] , wherein is the total number of frames of dynamic log data packets collected in the time window ; is the recording time of the th frame data, in particular, define ; is the instantaneous flux rate distribution at time , the dimension is particle number ; the calculation of this instantaneous value depends on the multi-energy spectrum superposition algorithm, where the key physical parameters are obtained as follows: energy spectrum weight is derived from the golden standard phase space file, the acquisition of this weight is processed by the following statistical binning: divide the energy range into discrete intervals, such as one interval per 0.1 MeV, traverse all particle records in the phase space file, extract the weight field in each particle record, accumulate the particle weight value falling into the th energy interval to obtain the total sum , and perform normalization calculation to obtain; The file adopts IAEA phsp standard format; the file acquisition approach is: using BEAMnrc Monte Carlo simulation code, based on the treatment head blueprint provided by the equipment manufacturer, containing target, primary collimator, uniform filter geometric parameters, setting incident electron beam energy as 6MeV, simulating A primary history, setting a phase space score plane under the uniform filter at Z=100cm to record particle information passing to generate; the specific data structure follows the IAEA-PHSP specification file header is fixed length 32 bytes, containing total particle number And maximum energy Field; particle record list adopts binary compact storage, and each particle record contains: energy , floating point type, position coordinate , floating point type, unit: cm, direction cosine , floating point type, weight , floating point type and history mark LATCH, integer type, record particle generation area; In order to ensure the real-time response capability of the system in clinical application, the gold standard phase space file is configured as a pre-generated static database file in commercial implementation, rather than being generated in real time during treatment; specifically, the file is generated by offline high-performance computing during system installation or physical debugging phase, and is solidified and stored in the local high-speed storage medium of the system; during real-time operation, the dose operation unit directly calls the particle data in the preset file as a constant input source through file indexing, thereby avoiding the huge computing overhead required by repeated running of Monte Carlo simulation, and meeting the stringent requirements of the safety warning system on low delay; In addition, in order to overcome the actual output spectrum drift problem caused by physical wear of components during long-term operation of the equipment, the embodiment also configures a source model periodic calibration mechanism: the system supports periodic import of measured water tank dose data, i.e. PDD and Profile curve, and uses iterative optimization algorithms such as simulated annealing to fine-tune the particle weight And angular distribution parameters in the gold standard phase space file; In the simulated annealing algorithm setting, the recommended initial temperature Cooling coefficient Markov chain length at each temperature Total iteration number is not less than Times, the objective function is defined as the minimization of the root mean square error of the measured curve and the calculated curve; the modified version of the phase space file suitable for the current equipment aging state is generated, ensuring that the reconstruction basis of the ideal reference field is dynamically synchronized with the evolution of the physical characteristics of the equipment, and eliminating systematic errors; Wherein, Represents the voxel index in the three-dimensional space grid, Energy interval index representing discrete energy spectrum; for each voxel , defined material parameters and : using stoichiometric calibration method, specifically through piecewise linear function to establish the mapping relationship between CT value, i.e. HU, and element weight percentage and density: when , i.e. lung tissue interval, the density mapping is , and the element ratio is set to ; When , i.e. soft tissue and bone interval, Schneider fitting formula is used, and further element component mapping step is performed: defining soft tissue reference element weight vector and dense bone reference element weight vector ; wherein, is defined according to ICRU-44 standard as , is defined according to ICRP standard as ; when , directly use as the element ratio of the current voxel; when , calculate the interpolation coefficient, and the calculation formula is:

[0023] Wherein, the denominator 2000 is a normalization constant based on the medical CT dynamic range, corresponding to the typical CT value span from soft tissue to cortical bone; limited to 0 to 1, i.e. abnormal HU values outside the range of [-100, 2100] are truncated to prevent non-physical negative mass weight, and the element weight of the current voxel is calculated as

[0024] Based on the calculated and , and according to NIST XCOM database, the accurate linear attenuation coefficient of the current voxel at the energy level is synthesized; In order to ensure the consistency of physical field and imaging device characteristics, the algorithm performs detector response convolution, and the calculation formula is:

[0025] Wherein, is the pixel space coordinate in the ideal reference field image, is the integral variable, is a double Gaussian point spread function used to simulate the optical scattering effect in amorphous silicon detector; in the formula, represents the relative amplitude weight of different scattering sources, corresponding primary ray, corresponding extranuclear scattering, representing the standard deviation of Gaussian distribution space of corresponding scattering source, in mm; parameters derived from the physical debugging report of the device, under the configuration of typical amorphous silicon flat panel detector, the weight of the primary ray is usually in the range of , the weight of the extranuclear scattering , the primary peak width is about to , the scattering peak width is about to ; for example, take ; the generated by the above integral and convolution operation has strictly consistent time integration properties and energy response characteristics with the real-time monitoring image in the physical sense, so as to ensure that the subsequent difference calculation can truly reflect the unexpected physical deviation, rather than the artifacts caused by time desynchronization.

[0026] The fault state space simulation component includes: a parameter definition unit, which is adapted to convert common mechanical failure modes and positioning error modes into mathematical space transformation matrices or flux blocking functions; a manifold generation unit, which is adapted to apply the space transformation matrix or the flux blocking function to the ideal reference field data, respectively, to simulate and generate a set of virtual image data corresponding to different fault types, to form theoretical fault state space data.

[0027] The embodiment further refines the mathematical construction process of the fault state space simulation component, focusing on constructing a complete orthogonal fault basis to cover deviations in any direction; the parameter definition unit discards the random sampling strategy and instead adopts a systematic grid sampling strategy to define fault parameters: for MLC card sticking failure, define the flux blocking function , and the specific mathematical form is a pixel-level mask generator:

[0028] wherein, is the leaf transmittance, such as 0.015, is an indicator function, which takes a value of 1 when the point is located in the area , and is defined as the polygon closure area composed of the profile coordinates of the leaf pair that is stuck on the projection plane; the sticking coefficient is in Discrete in the interval with 0.1 as a step; for spatial positioning error, in order to cover the 6-DoF rigid body transformation space, the following base is constructed: translation base: traverse the three main axis directions of the Cartesian coordinate system ; rotation base: generate uniformly distributed rotation axis vectors on the unit sphere by using the Fibonacci sphere algorithm ; The specific calculation steps of the algorithm are as follows: let the golden ratio , for index , calculate and radius , azimuth angle , and the generated rotation axis vector ; the selection basis of the point number is , for example, take , wherein is a preset angle resolution to ensure the spatial coverage rate of rotation sampling; the transformation matrix construction, the calculation formula is:

[0029] , wherein is the normalized fault intensity ; is a preset maximum rotation fault angle, usually the value range is to , and the embodiment takes to cover the common small rotation error of the gantry or treatment bed in clinic; The rotation matrix is constructed by using the Rodrigues rotation formula; let the rotation angle , the rotation axis unit vector , and the corresponding skew-symmetric matrix be:

[0030] Then the rotation matrix is calculated as follows:

[0031] , wherein is the unit matrix; the manifold generation unit executes the full permutation traversal generation process, and the calculation formula is:

[0032] That is, for each fault type, each discrete direction, and each intensity level, a corresponding virtual image data is generated, forming a structured theoretical fault state space data tensor; for spatial transformation operation, a trilinear interpolation algorithm is used to resample the 3D ideal dose field; for flux shielding operation, a mask is directly applied on the 2D projection plane performing a Hadamard product operation; The embodiment ensures that no matter what direction of the microshift occurs in reality, such as 45 degrees, the residual feature generated thereby can find a highly similar base or linear combination neighborhood thereof in the theoretical library, thereby solving the missed detection problem caused by single random sampling.

[0033] The risk quantification and positioning component includes: a feature space mapping unit adapted to map the real residual feature and the theoretical residual feature library to a high-dimensional topological feature space; a similarity calculation and matching unit adapted to calculate the cosine similarity or topological homology group feature of the real residual feature in the high-dimensional topological feature space with the theoretical residual feature library, and match the theoretical fault mode with the highest similarity; a report generation unit adapted to generate a visual auxiliary decision report including risk quantification indicators, suspected fault types and their probabilities, and residual topological mode analysis according to the matching result.

[0034] The embodiment further refines the operation logic of the risk quantification and positioning component, and clearly defines the mathematical definition of the high-dimensional topological feature space and the specific formula of the fusion metric; the feature space mapping unit constructs a Cartesian product metric space:

[0035] wherein, is the Euclidean feature subspace based on PCA dimension reduction, , is the Wasserstein metric space based on persistent homology barcode; The specific mapping process is as follows: vector mapping: flatten and project the residual matrix onto the orthogonal basis obtained by SVD decomposition of the covariance matrix S of the theoretical residual library , to obtain the eigenvector ; topological mapping: the residual image is regarded as a scalar field, and a square complex is constructed by adopting the lower level set filter, the Betti number changes of 0 and 1 are calculated, and a persistent homology barcode set is generated; The similarity calculation and matching unit performs adaptive fusion metric to calculate the comprehensive matching index , and the calculation formula is:

[0036] The formula contains two correction terms: a directional rectification term for measuring the real residual feature vector and a theoretical residual feature vector In the feature space, to avoid division by zero errors and enhance numerical stability, a small regularization constant is introduced For example The calculation formula is as follows:

[0037] Wherein, represents the Euclidean norm, is a linear rectification function, which is used to force the filtering of negative correlation results and ensure that a response is only generated when the fault direction is consistent; Topological kernel function term: wherein, is the Wasserstein distance between two barcode sets is a topological bandwidth parameter; it should be noted that has the same dimension as , i.e., distance unit, so as to ensure that the input parameter of the exponential function is a dimensionless quantity; The minimum weight perfect matching is obtained by the Hungarian algorithm, and the specific construction rules of the cost matrix are as follows: let set have points, set have points, and the cost matrix of is constructed, wherein the elements of the upper left corner block, i.e., the Chebyshev distance; the elements of the lower left and upper right corner blocks are defined as the Chebyshev distance from the feature point to the diagonal line, considering the persistence of the topological feature, i.e., the difference between the birth time and the death time , the calculation formula is modified to a linear form, and the calculation formula is:

[0038] Wherein, is the birth time (Birth) of the th topological feature point, is the death time of the feature point; to ensure that the overall cost matrix follows the consistency of norm, avoiding divergence of optimization solution due to mixed dimensions; Wherein, ​To allocate matrix elements, when the first... The feature point and the first The value is 1 when a feature point is matched, and 0 otherwise. The value is obtained by kernel density estimation (KDE) of the distance distribution between samples in the theoretical library; in specific implementation, the theoretical library is calculated. The set of Wasserstein distances between all sample pairs in the set. A Gaussian kernel function was selected as the estimation kernel, and the optimal bandwidth was determined using the Silverman rule. ,set up The formula for Silverman's rule is:

[0039] in, For set standard deviation Interquartile range, The total number of sample pairs; this parameter is used for adaptive normalization of the distance scale; in the context of topological feature analysis of radiotherapy dose maps, it is calculated using the aforementioned Silverman rule. Typical value range is usually in Between; smaller The value can capture subtle differences in dose texture, while larger values... The value focuses on the overall distribution of topological void structures; The report generation unit traverses all elements in the library, searching for elements that make... The theoretical failure mode is maximized, and based on The numerical generation risk probability; This embodiment achieves precise fingerprint-level locking of fault modes by fusing orientation-sensitive algebraic features and deformation-robust topological features in the feature space and introducing rectification and numerical stabilization mechanisms.

[0040] The auxiliary control suggestion component is suitable for generating equipment control parameter adjustment suggestions or process intervention suggestions in response to the auxiliary decision report; the visualization interaction component is suitable for displaying the auxiliary decision report and control parameter adjustment suggestions on the operation interface, and highlighting suspected fault areas and residual topology. This embodiment describes in detail the auxiliary control and interaction functions of the system; the auxiliary control suggestion component performs logical judgment on the risk analysis results, and in response to the identification of systemic mechanical faults, such as MLC jamming, generates process intervention suggestions and triggers interlocking circuits to interrupt the beam; In response to identifying correctable positioning errors, such as minor body position shifts, a device control parameter adjustment suggestion is generated, instructing the treatment bed to make a reverse compensatory movement; at the same time, the visual interactive component uses heat map technology to superimpose the suspected fault area on the real-time monitoring image on the doctor's operation interface, and displays the risk probability in the form of a radar chart; The embodiment converts abstract algorithm results into intuitive visual signals and executable operation instructions, thereby constructing a safe closed loop of human-machine interaction. In the time-critical radiotherapy process, the scheme greatly shortens the doctor's decision-making time, ensures that the correct intervention measures can be taken quickly at the moment of failure, and guarantees the safety of patient treatment.

[0041] The multi-modal data acquisition component is configured to: obtain static diagnosis and treatment plan data containing CT images and structural delineation information through a hospital information system interface; read dynamic mechanical execution log data containing multi-leaf collimator positions and gantry angles in real time through a linear accelerator control interface; and acquire real-time monitoring stream data through an electronic portal imaging device or an optical body surface monitoring system.

[0042] The embodiment specifically defines the hardware connection configuration of the multi-modal data acquisition component; the component is configured with a DICOM-RT protocol interface and connected to a hospital information system or a tumor radiotherapy network system through a local area network to obtain static diagnosis and treatment plan data; The component is configured with a TCP / IP listening interface and directly connected to the control cabinet of the linear accelerator to acquire dynamic mechanical execution log data containing gantry angles, collimator angles and MLC leaf positions in real time at a high sampling rate; at the same time, the component is connected to an electronic portal imaging device through an image acquisition card to obtain a fluoroscopy image stream, or connected to an optical body surface monitoring system through a data interface to obtain a three-dimensional point cloud data stream; The embodiment ensures full-dimensional data coverage from static anatomical structures, dynamic mechanical parameters to real-time image feedback through explicit hardware interface configuration; in a complex medical device networking environment, this multi-channel parallel acquisition mechanism guarantees the real-time and integrity of the data, providing a solid physical data foundation for subsequent multi-modal fusion analysis; Please refer to Figure 2 , the radiotherapy diagnosis and treatment safety early warning method based on multi-modal data fusion, applied to the radiotherapy diagnosis and treatment safety early warning system based on multi-modal data fusion as in the foregoing embodiments, comprising the following steps: obtaining static diagnosis and treatment plan data of a target object, and acquiring dynamic mechanical execution log data and real-time monitoring stream data of a treatment device in real time; driving the static diagnosis and treatment plan data based on the dynamic mechanical execution log data to reconstruct ideal reference field data at the current time; superimposing a preset fault disturbance operator on the ideal reference field data to simulate and generate theoretical fault state space data containing multiple fault modes; The differences between real-time monitoring stream data and ideal reference field data, as well as the differences between theoretical fault state space data and ideal reference field data, are calculated separately to obtain the actual residual characteristics and theoretical residual characteristic library. The topological forms of the actual residual characteristics and theoretical residual characteristic library are compared to generate an auxiliary decision-making report containing risk quantification indicators and potential fault location.

[0043] This embodiment details the execution flow of a radiology and treatment safety early warning method based on multimodal data fusion; the system performs a data ingestion step, loading the static plan of the target patient. And start a listener thread to synchronously cache the machine log. and monitoring stream ; Perform the baseline reconstruction step, utilizing Real-time parameters in By performing rigid body transformation and projection calculations, the ideal reference field at the current moment can be reconstructed. ; Enter the fault simulation step, traverse the preset set of fault operators, and superimpose them respectively to This generates a state space containing multiple hypothetical faults; Based on this, the difference extraction step is performed to calculate respectively. and The difference is worth the real residual. and theoretical fault state data and The difference is used to obtain the theoretical residual. ; Perform the matching decision step and compare With each item Based on the topological structure, output a risk report; The method flow of this embodiment constitutes a complete closed loop of forward modeling and reverse verification. Through active simulation and differential library construction steps, the system has a standard fault template for comparison when facing unknown real-time data, thereby significantly improving the confidence of the judgment and realizing dynamic monitoring and accurate identification of potential risks in the process of radiotherapy.

[0044] The comparison of the topological features of actual residuals with those of the theoretical residual feature library includes: using synthetic analysis to search for target fault fingerprints in the theoretical residual feature library that have the maximum topological similarity to actual residuals; if the maximum topological similarity exceeds a preset judgment threshold, it is marked as a high-probability systemic risk in the auxiliary decision-making report and associated with a specific fault mode; if the maximum topological similarity does not exceed the preset judgment threshold, it is classified as a low-risk difference in the auxiliary decision-making report, which may originate from physiological fluctuations or random noise.

[0045] This embodiment further refines the risk classification logic in the comparison step; this step employs a synthetic analysis strategy to search the feature library for features that match the actual residuals. Target fault fingerprint with maximum comprehensive matching index System performs threshold decision logic: in response to exceeding a preset decision threshold , the system determines that the pattern of the actual residual is highly isomorphic to a certain theoretical fault, and labels it as a high-probability systematic risk and associates it with a specific fault mode; in response to not exceeding , the system infers that the difference is due to unstructured random factors, and classifies it as a low-risk difference; wherein, The value of is obtained by offline statistical learning: collect cases, such as , residual data during normal treatment in history, calculate the maximum comprehensive matching index distribution with the fault library, use Gaussian distribution or Gamma distribution for fitting, and set the upper limit of the distribution with a confidence of 99% as ; as a specific implementation example, after collecting 1500 cases of normal treatment log data, the calculated distribution obeys Gamma distribution , and the 99% quantile point calculated according to the cumulative distribution function is ; therefore, the decision threshold is set in this embodiment; when the real-time calculated matching index , the system triggers a high-risk alarm; This embodiment effectively solves the alarm fatigue problem by introducing a dynamic threshold based on big data statistics and binary classification logic; in the clinical monitoring scene, this scheme can intelligently ignore false alarms that have numerical differences but no fault structure characteristics, such as normal respiratory fluctuations in patients, and as soon as it captures weak signals that meet the fault topology structure, such as early mechanical wear, it immediately issues a high-priority alarm, achieving a balance between specificity and sensitivity.

[0046] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present application and not to limit it, although the present application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application can be modified or replaced with equivalent without departing from the spirit and scope of the technical solutions of the present application.

Claims

1. A radiology and treatment safety early warning system based on multimodal data fusion, characterized in that, include: A multimodal data acquisition component, which is suitable for acquiring static treatment plan data, dynamic mechanical execution log data, and real-time monitoring stream data; An ideal physics field reconstruction engine is adapted to use the dynamic mechanical execution log data to drive the anatomical model in the static treatment plan data, and reconstruct time-varying ideal reference field data through a physical transmission algorithm. A fault-state space simulation component is adapted to call a preset fault mode knowledge base and inject parameterized disturbance operators into the ideal reference field data to generate theoretical fault-state space data. A dual-track differential extraction component is adapted to calculate the difference between the real-time monitoring stream data and the ideal reference field data to generate realistic residual features, and to calculate the difference between the theoretical fault state space data and the ideal reference field data to generate a theoretical residual feature library. A risk quantification and location component is adapted to calculate the topological similarity between the actual residual features and each element in the theoretical residual feature library, and generate risk quantification indicators and corresponding potential fault type location information based on the topological similarity to form an auxiliary decision-making report.

2. The radiodiagnosis and treatment safety early warning system based on multimodal data fusion according to claim 1, characterized in that, The ideal physics field reconstruction engine includes: A virtual radiotherapy driving unit, which is adapted to parse millisecond-level mechanical state parameters in the dynamic mechanical execution log data and map the mechanical state parameters to the spatial coordinate system defined by the static treatment plan data; The dose calculation unit is adapted to use a phase space-based convolution superposition algorithm or ray tracing algorithm to calculate the theoretical dose distribution within the target object and the theoretical projection image of the body surface based on the mapped spatial coordinate system, so as to form the ideal reference field data.

3. The radiodiagnosis and treatment safety early warning system based on multimodal data fusion according to claim 1, characterized in that, The fault-state space simulation component includes: A parameter definition unit, which is adapted to convert common mechanical failure modes and positioning error modes into mathematical spatial transformation matrices or flux blocking functions. A manifold generation unit is adapted to apply the spatial transformation matrix or the flux occlusion function to the ideal reference field data to simulate and generate virtual image data sets corresponding to different fault types, thereby constituting the theoretical fault state spatial data.

4. The radiodiagnosis and treatment safety early warning system based on multimodal data fusion according to claim 1, characterized in that, The risk quantification and positioning component includes: A feature space mapping unit, wherein the feature space mapping unit is adapted to map the real residual features and the theoretical residual feature library to a high-dimensional topological feature space; The similarity calculation and matching unit is adapted to calculate the cosine similarity or topological homology group feature between the actual residual features and the theoretical residual feature library in the high-dimensional topological feature space, and match the theoretical fault morphology with the highest similarity. The report generation unit is adapted to generate a visual auxiliary decision-making report based on the matching results, which includes risk quantification indicators, suspected fault types and their probabilities, and residual topology analysis.

5. The radiodiagnosis and treatment safety early warning system based on multimodal data fusion according to claim 1, characterized in that, Also includes: An auxiliary control suggestion component, which is adapted to generate equipment control parameter adjustment suggestions or process intervention suggestions in response to the auxiliary decision report; A visualization and interaction component is provided, which is adapted to display the auxiliary decision-making report and the control parameter adjustment suggestions on the operation interface, and to highlight the suspected fault areas and residual topology.

6. The radiodiagnosis and treatment safety early warning system based on multimodal data fusion according to claim 1, characterized in that, The multimodal data acquisition component is configured as follows: The static treatment plan data, which includes CT images and structural delineation information, is obtained through the hospital information system interface; The dynamic mechanical execution log data, including the position of the multi-leaf collimator and the frame angle, is read in real time through the linear accelerator control interface. The real-time monitoring stream data is acquired through an electronic field imaging device or an optical surface monitoring system.

7. A radiotherapy safety early warning method based on multimodal data fusion, employing the radiotherapy safety early warning system based on multimodal data fusion as described in any one of claims 1-6, characterized in that, Includes the following steps: Acquire static treatment plan data for the target object, and collect dynamic mechanical execution log data and real-time monitoring stream data of the treatment equipment in real time; Based on the dynamic mechanical execution log data, the static treatment plan data is driven to reconstruct the ideal reference field data at the current moment; The preset fault disturbance operator is superimposed on the ideal reference field data to simulate and generate theoretical fault state space data containing multiple fault modes. The differences between the real-time monitoring stream data and the ideal reference field data, and the differences between the theoretical fault state space data and the ideal reference field data are calculated respectively to obtain the actual residual features and the theoretical residual feature library; By comparing the actual residual characteristics with the topological form of the theoretical residual characteristic library, an auxiliary decision-making report containing risk quantification indicators and potential fault location is generated.

8. The radiodiagnosis and treatment safety early warning method based on multimodal data fusion according to claim 7, characterized in that, The topological form of comparing the actual residual features with the theoretical residual feature library includes: Using synthetic analysis, a target fault fingerprint with the maximum topological similarity to the actual residual features is searched in the theoretical residual feature library; If the maximum topological similarity exceeds a preset judgment threshold, it will be marked as a high-probability systemic risk in the auxiliary decision-making report and associated with a specific failure mode. If the maximum topological similarity does not exceed the preset judgment threshold, it will be classified as a low-risk difference in the auxiliary decision report, which may originate from physiological fluctuations or random noise.

Images