Multimodal-based intelligent analysis and supervision method and system for medical device data
By using multimodal data acquisition and storage and state matrix evaluation, the problems of data traceability difficulties and transmission conflicts in medical device data supervision have been solved. This has enabled the accurate integration of multimodal data and the accurate determination of collaborative relationships, thereby improving the intelligence level of the supervision system.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- JIANGSU KANGYITONG TECH CO LTD
- Filing Date
- 2026-02-03
- Publication Date
- 2026-06-05
AI Technical Summary
Existing medical device data monitoring technologies suffer from several drawbacks. The lack of a unified coding standard for multimodal data acquisition nodes leads to difficulties in data traceability and correlation, low data fusion efficiency, inability to quantify transmission frequency, lack of status assessment mechanisms, and a tendency for data transmission conflicts. Furthermore, the significant differences in the modalities of multi-source data result in insufficient fusion accuracy.
A multimodal data acquisition and storage module is used for unified encoding and storage. An event processing module is used to calculate the transmission frequency, construct a data transmission status matrix and evaluate concurrent cooperation relationships. Boolean logic operations are used to determine the cooperation relationships, thereby achieving standardized representation and accurate determination of data transmission status.
It achieves accurate integration of multimodal data and quantitative assessment of transmission status, avoids data transmission conflicts, and improves the intelligence level and data fusion accuracy of the regulatory system.
Smart Images
Figure CN122158047A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of medical device interactive data transmission monitoring technology, specifically to a multimodal intelligent analysis and monitoring method and system for medical device data. Background Technology
[0002] With the rapid development of medical informatization, the trend of intelligent and multimodal medical equipment is becoming increasingly significant. Various medical devices, such as imaging equipment, sensing equipment, and physiological parameter monitoring equipment, are widely used in clinical diagnosis, treatment, and monitoring. The security and efficiency of the multi-source data interaction and transmission they generate directly affect the quality of medical services and patient safety. Medical equipment data supervision, as a core link in ensuring stable equipment operation and reliable data transmission, has received high attention from the industry.
[0003] However, existing medical device data monitoring technologies face several pressing technical challenges: First, the lack of a unified coding standard for multimodal data acquisition nodes leads to confusion regarding the attribution of acquisition and fusion ends for different types and manufacturers of medical devices. This results in difficulties in tracing and correlating multi-source monitoring data (images, sensors, physiological parameters, operational logs, etc.), and inefficient data integration. Second, existing technologies primarily focus on the fusion processing of the data itself, neglecting to analyze the correlation between data collaborative transmission duration and actual fusion volume. This makes it impossible to quantify the frequency of data transmission, resulting in a large amount of redundant transmission data consuming bandwidth resources. Furthermore, it makes it difficult to identify valuable data transfers for medical decision-making. Third, there is a lack of effective data transmission status assessment mechanisms and a lack of standardized status representation models, making it impossible to accurately distinguish between redundant and gain states during transmission, resulting in a lack of data support for regulatory decisions. Fourth, existing technologies struggle to assess the concurrent and collaborative relationships between different medical device operation events. When multiple devices are running simultaneously, it is impossible to determine whether there is a need for collaborative transmission, which can easily lead to data transmission conflicts or a lack of collaboration, resulting in regulatory delays and a high rate of misjudgment. Fifth, the modal differences between multi-source data are significant, and existing fusion technologies have not established targeted sets of analytical behaviors, resulting in insufficient accuracy in data fusion and an inability to provide an effective data foundation for regulation.
[0004] These technical challenges result in low intelligence and limited efficiency of existing medical equipment data monitoring systems, making it difficult to meet the needs of complex medical scenarios involving multiple devices working collaboratively. For example, scenarios such as real-time data interaction monitoring of multimodal monitoring devices in intensive care units and collaborative transmission assurance of medical devices in operating rooms urgently require a monitoring technology that can accurately integrate multimodal data, quantitatively assess transmission status, and intelligently determine concurrent collaborative relationships to address the shortcomings of existing technologies. Summary of the Invention
[0005] The purpose of this invention is to provide a method and system for intelligent analysis and supervision of medical device data based on multimodality, so as to solve the problems mentioned in the background art.
[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution:
[0007] The medical device data intelligent analysis and monitoring system based on multimodal data includes: a multimodal data acquisition and storage module, an operation event processing and frequency calculation module, a data transmission status matrix construction module, and a multimodal acquisition node concurrent collaborative evaluation module.
[0008] The multimodal data acquisition and storage module is used to set up multimodal data acquisition nodes and perform unified encoding, store multi-source monitoring data and multimodal interaction logs, and record the duration of data collaborative transmission.
[0009] The event processing and frequency calculation module is used to uniformly compile medical equipment operation events, generate analysis behavior sets, initialize data fusion volume, and calculate data transmission frequency.
[0010] The data transmission state matrix construction module is used to obtain the transmission frequency under the same interaction behavior and calculate the average transmission frequency, construct the data transmission state matrix and mark the corresponding transmission state.
[0011] The multimodal acquisition node concurrent collaboration evaluation module evaluates the concurrency of multimodal acquisition nodes among different medical device operation events based on the data transmission status matrix, determines the concurrent collaboration relationship based on the concurrency of multimodal acquisition nodes, and outputs the relevant medical device operation events.
[0012] As a preferred embodiment of the present invention, the multimodal data acquisition and storage module is specifically used for: setting up multimodal data acquisition nodes in the medical device operation process; uniformly encoding all acquisition nodes, with each acquisition node corresponding to the data acquisition end or data fusion end of the medical device; storing multi-source monitoring data of each acquisition node, the multi-source monitoring data covering acquisition end data and fusion end data related to images, sensors, physiological parameters, and device operation logs, wherein the fusion end can correspond to a combination of one or more acquisition ends; and simultaneously storing multimodal interaction logs of each acquisition node, the logs recording medical device operation events and data collaborative transmission duration, the collaborative transmission duration being defined as the total time from the acquisition end sending out multimodal data to the fusion end completing data fusion.
[0013] As a preferred embodiment of the present invention, the operation event processing and frequency calculation module is specifically used for: uniformly compiling medical equipment operation events, establishing a monitoring center with each multimodal acquisition node as a fusion end, statistically analyzing all acquisition ends corresponding to each operation event based on the data collaborative transmission duration, generating an analysis behavior set corresponding to the fusion end; initializing the initial data fusion amount when the acquisition end and the fusion end cooperate within a unit time range, and calculating the data transmission frequency between the acquisition end and the fusion end by combining the actual data fusion amount recorded in the analysis behavior set.
[0014] As a preferred embodiment of the present invention, the data transmission state matrix construction module is specifically used for: collecting the data transmission frequency of the same interactive behavior in all medical device operation events, and calculating the average transmission frequency corresponding to the interactive behavior; using the encoding range of the multimodal acquisition node as the row index and column index to construct the data transmission state matrix corresponding to each medical device operation event; comparing the frequency of a single data transmission with the average value, and if it is lower than the average value, marking the corresponding position in the matrix as a value representing the redundancy state, and if it is higher than or equal to the average value, marking it as a value representing the gain state.
[0015] As a preferred embodiment of the present invention, the multimodal acquisition node concurrent collaborative evaluation module is specifically used for: extracting the data transmission state matrix corresponding to any two medical device operation events; calculating the number of values representing the gain state after performing a Boolean logical AND operation on the two matrices, and the number of values representing the gain state after performing a Boolean logical OR operation; obtaining the multimodal acquisition node concurrency degree between the two operation events by the ratio of the two; setting a preset concurrency similarity threshold; when the calculated concurrency degree is greater than or equal to the threshold, determining that the two medical device operation events have a multimodal acquisition node concurrent collaborative relationship, and outputting all medical device operation events with this relationship.
[0016] A multimodal intelligent analysis and supervision method for medical device data, comprising the following steps:
[0017] Step S1: Set up multimodal data acquisition nodes and perform unified encoding, store multi-source monitoring data and multimodal interaction logs, and record the duration of data collaborative transmission;
[0018] Step S2: Compile medical equipment operation events in a unified manner, generate an analysis behavior set, initialize the data fusion volume, and calculate the data transmission frequency;
[0019] Step S3: Obtain the transmission frequency under the same interaction behavior and calculate the average transmission frequency, construct the data transmission state matrix and mark the corresponding transmission state;
[0020] Step S4: Evaluate the concurrency of multimodal acquisition nodes among different medical device operation events based on the data transmission state matrix, determine the concurrent collaboration relationship based on the concurrency of multimodal acquisition nodes, and output the relevant medical device operation events.
[0021] As a preferred embodiment of the present invention, the specific implementation process of step S1 includes:
[0022] In the operation process of medical equipment, multimodal data acquisition nodes are set up, and these nodes are uniformly coded. The i-th multimodal acquisition node is denoted as... The system stores multi-source monitoring data from each multimodal acquisition node. This multi-source monitoring data includes medical device acquisition and fusion data in image mode, sensor mode, physiological parameter mode, and device operation log mode. Each multimodal acquisition node corresponds to one medical device data acquisition or fusion terminal. If the fusion terminal is a multimodal acquisition node... Then the acquisition end is the non-multimodal acquisition node. At least one multimodal acquisition node or a combination of multiple multimodal acquisition nodes other than those mentioned above;
[0023] The system stores multimodal interaction logs for each multimodal acquisition node. These logs record medical device operation events, including the collaborative transmission duration of multimodal data. This collaborative transmission duration is the time from when multimodal data is sent from the acquisition end to when the fusion end completes the multimodal data fusion. The system then stores the multimodal interaction logs for each multimodal acquisition node. As the fusion end, and the j-th multimodal acquisition node When acting as a data acquisition terminal, the duration of the generated collaborative data transmission is denoted as... .
[0024] As a preferred embodiment of the present invention, the specific implementation process of step S2 includes:
[0025] The operation events of medical equipment are uniformly compiled, and the k-th operation event of medical equipment is denoted as... Multimodal acquisition nodes It serves as a monitoring center for the converged terminals and, based on the duration of collaborative data transmission, statistically analyzes medical device operational events. All acquisition terminals below generate multimodal acquisition nodes. Let the set of behaviors analyzed at the fusion end be denoted as ,in, This represents the total number of multimodal acquisition nodes. Data modality tags are used to indicate the duration of collaborative data transmission. Inside, multimodal acquisition nodes As a fusion terminal and multimodal acquisition node The actual amount of data fusion when used as a collection end;
[0026] Initial configuration within a unit time range, multimodal acquisition nodes As a fusion terminal and multimodal acquisition node Initial data fusion volume when used as a data acquisition end ;
[0027] Based on the initial data fusion volume Integration volume with actual data Evaluation of multimodal acquisition nodes As a fusion terminal and multimodal acquisition node Data transmission frequency when acting as a data acquisition terminal .
[0028] As a preferred embodiment of the present invention, the specific implementation process of step S3 includes:
[0029] Under the same interactive behavior, i.e., multimodal acquisition nodes As a fusion terminal and multimodal acquisition node As the data acquisition endpoint, the same interactive behavior is recorded during medical device operation events. The frequency of data transmission in the data is denoted as ;
[0030] Calculate the average frequency of data transmission. Where U represents the total number of medical equipment operation events;
[0031] The range of encoding sequence numbers for multimodal acquisition nodes Build medical device operation events for row and column indexes. The data transmission state matrix, if the data transmission frequency In the data transmission state matrix, the element in the i-th row and j-th column is marked as 0, which is used to characterize the multimodal acquisition node. As a fusion terminal and multimodal acquisition node The interaction behavior of the acquisition end is redundant; if the data transmission frequency is high... In the data transmission state matrix, the matrix element in the i-th row and j-th column is marked as 1, which is used to characterize the multimodal acquisition node. As a fusion terminal and multimodal acquisition node The interactive behavior of the acquisition end is in a gain state;
[0032] Then the medical equipment operation event is obtained. Data transmission state matrix .
[0033] As a preferred embodiment of the present invention, the specific implementation process of step S4 includes:
[0034] Based on the data transmission state matrix, the concurrency of multimodal acquisition nodes among medical device operation events is evaluated. In the formula, This represents the g-th medical device operation event. Indicates medical device operation events The data transmission state matrix, Represents the data transmission state matrix With data transmission state matrix The number of 1s contained in the result of an inter-Boolean logical AND operation. Represents the data transmission state matrix With data transmission state matrix The number of 1s contained after a Boolean logical OR operation;
[0035] A preset concurrent similarity threshold is set to determine the concurrency of multimodal acquisition nodes. If the similarity is greater than or equal to the concurrent similarity threshold, then a medical device operation event is determined. Medical equipment operation events There are concurrent collaborative relationships between multimodal acquisition nodes, and the operation events of each medical device with concurrent collaborative relationships between multimodal acquisition nodes are output.
[0036] Compared with the prior art, the beneficial effects achieved by the present invention are:
[0037] This invention clarifies the flexible combination relationship between the acquisition end and the fusion end, solves the problems of ambiguous ownership and difficulty in association of multi-source data in the prior art, and realizes the accuracy of data traceability. At the same time, it combines the data collaborative transmission time with the actual fusion amount and the initial fusion amount to calculate the transmission frequency, establishes a quantitative evaluation standard for transmission behavior, and can accurately distinguish between redundant transmission and gain transmission.
[0038] A data transmission status matrix is constructed, and the transmission status is intuitively marked by 0 / 1 values, which realizes the standardized representation of the transmission status. At the same time, the concurrency evaluation method based on matrix Boolean operation realizes the accurate determination of the concurrent collaboration relationship between different medical device operation events. Existing technologies cannot achieve the quantitative analysis of this type of collaboration relationship. Moreover, this invention effectively avoids data transmission conflicts and lack of collaboration, and improves the level of intelligent supervision. Attached Figure Description
[0039] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof.
[0040] Figure 1This is a schematic diagram illustrating the steps of the intelligent analysis and monitoring method for medical device data based on multimodality, as described in this invention. Detailed Implementation
[0041] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0042] In this first embodiment: a multimodal medical device data intelligent analysis and monitoring system is provided, which includes: a multimodal data acquisition and storage module, an operation event processing and frequency calculation module, a data transmission status matrix construction module, and a multimodal acquisition node concurrent collaborative evaluation module;
[0043] The multimodal data acquisition and storage module is used to set up multimodal data acquisition nodes and perform unified encoding, store multi-source monitoring data and multimodal interaction logs, and record the duration of data collaborative transmission.
[0044] Specifically, multimodal data acquisition nodes are set up in the medical equipment operation process, and all acquisition nodes are uniformly coded. Each acquisition node corresponds to the data acquisition end or data fusion end of the medical equipment. Multi-source monitoring data of each acquisition node is stored. The multi-source monitoring data covers data from the acquisition end and data from the fusion end related to images, sensors, physiological parameters, and equipment operation logs. The fusion end can correspond to a combination of one or more acquisition ends. At the same time, multimodal interaction logs of each acquisition node are stored. The logs record medical equipment operation events and data collaborative transmission duration. The collaborative transmission duration is defined as the total time from the acquisition end sending multimodal data to the fusion end completing data fusion.
[0045] The event processing and frequency calculation module is used to uniformly compile medical equipment operation events, generate analysis behavior sets, initialize data fusion volume, and calculate data transmission frequency.
[0046] Specifically, medical equipment operation events are uniformly compiled, and a monitoring center is established with each multimodal acquisition node as the fusion end. Based on the data collaborative transmission duration, all acquisition ends corresponding to each operation event are counted, and an analysis behavior set corresponding to the fusion end is generated. The initial data fusion volume when the acquisition end and the fusion end cooperate within a unit time range is initialized, and the data transmission frequency between the acquisition end and the fusion end is calculated by combining the actual data fusion volume recorded in the analysis behavior set.
[0047] The data transmission state matrix construction module is used to obtain the transmission frequency under the same interaction behavior and calculate the average transmission frequency, construct the data transmission state matrix and mark the corresponding transmission state.
[0048] Specifically, for the same interactive behavior, the data transmission frequency in all medical device operation events is collected, and the average transmission frequency corresponding to the interactive behavior is calculated; the encoding range of the multimodal acquisition node is used as the row index and column index to construct the data transmission state matrix corresponding to each medical device operation event; the frequency of a single data transmission is compared with the average value, and if it is lower than the average value, the corresponding position in the matrix is marked as a value representing the redundancy state, and if it is higher than or equal to the average value, it is marked as a value representing the gain state.
[0049] The multimodal acquisition node concurrent collaboration evaluation module evaluates the concurrency of multimodal acquisition nodes among different medical device operation events based on the data transmission status matrix, determines the concurrent collaboration relationship based on the concurrency of multimodal acquisition nodes, and outputs the relevant medical device operation events.
[0050] Specifically, the data transmission state matrix corresponding to any two medical device operation events is extracted. The number of values representing the gain state after the two matrices are ANDed by Boolean logic and the number of values representing the gain state after the two matrices are ORed by Boolean logic are calculated. The multimodal acquisition node concurrency between the two operation events is obtained by the ratio of the two. A concurrency similarity threshold is preset. When the calculated concurrency is greater than or equal to the threshold, it is determined that there is a multimodal acquisition node concurrent collaboration relationship between the two medical device operation events, and all medical device operation events with this relationship are output.
[0051] Please see Figure 1 In this second embodiment, a multimodal intelligent analysis and monitoring method for medical device data is provided to be applicable to the first embodiment. This embodiment uses the intensive care unit (ICU) of a tertiary hospital as the application scenario. This scenario includes 8 multimodal medical devices (3 data fusion terminal devices: central monitor, data server, and clinical decision terminal; 5 data acquisition terminal devices: high-definition imaging device, electrocardiogram sensor, blood pressure monitor, respiratory sensor, and device operation status monitor). Real-time monitoring of the multimodal data interaction and transmission between the devices is required to ensure reliable transmission of monitoring data for critically ill patients and collaborative operation of the devices.
[0052] The total number of multimodal acquisition nodes is m=8, uniformly coded as M1 (central monitor, fusion terminal), M2 (data server, fusion terminal), M3 (clinical decision terminal, fusion terminal), M4 (high-definition imaging device, acquisition terminal), M5 (ECG sensor, acquisition terminal), M6 (blood pressure monitor, acquisition terminal), M7 (respiratory sensor, acquisition terminal), and M8 (equipment operation status monitor, acquisition terminal).
[0053] The total number of medical device operation events is U=50 (covering scenarios such as patient monitoring, data backup, and clinical decision support), and is uniformly coded as E1-E50;
[0054] Initialize the data fusion volume h_ij = 50MB / hour (the baseline fusion volume from the acquisition end to the fusion end within a unit of time).
[0055] The statistical range of collaborative transmission duration T_ij is the duration of a single data transmission from each acquisition terminal to the fusion terminal (1-5 minutes).
[0056] Concurrent similarity threshold = 0.7 (verified through multiple experiments);
[0057] The method includes the following steps:
[0058] Step S1: Set up multimodal data acquisition nodes and perform unified encoding, store multi-source monitoring data and multimodal interaction logs, and record the duration of data collaborative transmission;
[0059] For example, in the operation process of medical equipment, multimodal data acquisition nodes are set up, and the multimodal acquisition nodes are uniformly coded, with the i-th multimodal acquisition node denoted as... It stores multi-source monitoring data from each multimodal acquisition node. This multi-source monitoring data includes data from the medical device acquisition and fusion ends, encompassing imaging modality, sensor modality, physiological parameter modality, and equipment operation log modality. Each multimodal acquisition node corresponds to one medical device data acquisition end or data fusion end. If the fusion end is a multimodal acquisition node... Then the acquisition end is the non-multimodal acquisition node. At least one multimodal acquisition node or a combination of multiple multimodal acquisition nodes other than those mentioned above;
[0060] The system stores multimodal interaction logs for each multimodal acquisition node. These logs record medical device operation events, which include the collaborative transmission duration of multimodal data. This collaborative transmission duration is the time from when multimodal data is sent from the acquisition end to when the fusion end completes the multimodal data fusion. The system then stores the multimodal interaction logs for each multimodal acquisition node. As the fusion end, and the j-th multimodal acquisition node When acting as a data acquisition terminal, the duration of the generated collaborative data transmission is denoted as... ;
[0061] For example, M1-M8 are uniformly encoded to store multi-source monitoring data (image data of M4, physiological parameter data of M5-M7, and operation log data of M8) and interaction logs, and T_ij is recorded (e.g., T_41=3 minutes for data transmission from M4 to M1, and T_52=2 minutes for data transmission from M5 to M2).
[0062] Step S2: Compile medical equipment operation events in a unified manner, generate an analysis behavior set, initialize the data fusion volume, and calculate the data transmission frequency;
[0063] For example, medical device operation events are uniformly compiled, and the kth medical device operation event is denoted as... Multimodal acquisition nodes It serves as a monitoring center for the converged terminals and, based on the duration of collaborative data transmission, statistically analyzes medical device operational events. All acquisition terminals below generate multimodal acquisition nodes. Let the set of behaviors analyzed at the fusion end be denoted as ,in, This represents the total number of multimodal acquisition nodes. Data modality tags are used to indicate the duration of collaborative data transmission. Inside, multimodal acquisition nodes As a fusion terminal and multimodal acquisition node The actual amount of data fusion when used as a collection end;
[0064] Initial configuration within a unit time range, multimodal acquisition nodes As a fusion terminal and multimodal acquisition node Initial data fusion volume when used as a data acquisition end ;
[0065] Based on the initial data fusion volume Integration volume with actual data Evaluation of multimodal acquisition nodes As a fusion terminal and multimodal acquisition node Data transmission frequency when acting as a data acquisition terminal ;
[0066] For example, E1-E50 are compiled, and an analysis behavior set E_k(M1)={H(M4, T_41), H(M5, T_51), ..., H(M8, T_81)} is generated with M1 as the fusion endpoint. The transmission frequency L(Mi, Mj) is then calculated. For example, in E10, H(M4, T_41)=120MB, L(M1, M4)=(120-50×3) / 50=(-30) / 50=-0.6; in E25, H(M4, T_41)=200MB, L(M1, M4)=(200-50×3) / 50=50 / 50=1. The frequency can effectively reflect the transmission intensity; negative values correspond to low-load transmission, and positive values correspond to high-load transmission.
[0067] Step S3: Obtain the transmission frequency under the same interaction behavior and calculate the average transmission frequency, construct the data transmission state matrix and mark the corresponding transmission state;
[0068] For example, under the same interaction behavior, i.e., multimodal acquisition node As a fusion terminal and multimodal acquisition node As the data acquisition endpoint, the same interactive behavior is recorded during medical device operation events. The frequency of data transmission in the data is denoted as ;
[0069] Calculate the average frequency of data transmission. Where U represents the total number of medical equipment operation events;
[0070] The range of encoding sequence numbers for multimodal acquisition nodes Build medical device operation events for row and column indexes. The data transmission state matrix, if the data transmission frequency In the data transmission state matrix, the element in the i-th row and j-th column is marked as 0, which is used to characterize the multimodal acquisition node. As a fusion terminal and multimodal acquisition node The interaction behavior of the acquisition end is redundant; if the data transmission frequency is high... In the data transmission state matrix, the matrix element in the i-th row and j-th column is marked as 1, which is used to characterize the multimodal acquisition node. As a fusion terminal and multimodal acquisition node The interactive behavior of the acquisition end is in a gain state;
[0071] Then the medical equipment operation event is obtained. Data transmission state matrix .
[0072] Step S4: Evaluate the concurrency of multimodal acquisition nodes among different medical device operation events based on the data transmission state matrix, determine the concurrency and collaboration relationship based on the concurrency of multimodal acquisition nodes, and output the relevant medical device operation events;
[0073] For example, based on the data transmission state matrix, the concurrency of multimodal acquisition nodes among medical device operation events is evaluated. In the formula, This represents the g-th medical device operation event. Indicates medical device operation events The data transmission state matrix, Represents the data transmission state matrix With data transmission state matrix The number of 1s contained in the result of an inter-Boolean logical AND operation. Represents the data transmission state matrix With data transmission state matrix The number of 1s contained after a Boolean logical OR operation;
[0074] A preset concurrent similarity threshold is set to determine the concurrency of multimodal acquisition nodes. If the similarity is greater than or equal to the concurrent similarity threshold, then a medical device operation event is determined. Medical equipment operation events There are concurrent collaborative relationships between multimodal acquisition nodes, and the operation events of each medical device with concurrent collaborative relationships between multimodal acquisition nodes are output.
[0075] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.
[0076] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.
Claims
1. A multimodal intelligent analysis and monitoring method for medical device data, characterized in that, The method includes the following steps: Step S1: Set up multimodal data acquisition nodes and perform unified encoding, store multi-source monitoring data and multimodal interaction logs, and record the duration of data collaborative transmission; Step S2: Compile medical equipment operation events in a unified manner, generate an analysis behavior set, initialize the data fusion volume, and calculate the data transmission frequency; Step S3: Obtain the transmission frequency under the same interaction behavior and calculate the average transmission frequency, construct the data transmission state matrix and mark the corresponding transmission state; Step S4: Evaluate the concurrency of multimodal acquisition nodes among different medical device operation events based on the data transmission state matrix, determine the concurrent collaboration relationship based on the concurrency of multimodal acquisition nodes, and output the relevant medical device operation events.
2. The intelligent analysis and monitoring method for medical device data based on multimodality according to claim 1, characterized in that, The specific implementation process of step S1 includes: In the operation process of medical equipment, multimodal data acquisition nodes are set up, and these nodes are uniformly coded. The i-th multimodal acquisition node is denoted as... The system stores multi-source monitoring data from each multimodal acquisition node. This multi-source monitoring data includes medical device acquisition and fusion data in image mode, sensor mode, physiological parameter mode, and device operation log mode. Each multimodal acquisition node corresponds to one medical device data acquisition or fusion terminal. If the fusion terminal is a multimodal acquisition node... Then the acquisition end is the non-multimodal acquisition node. At least one multimodal acquisition node or a combination of multiple multimodal acquisition nodes other than those mentioned above; The system stores multimodal interaction logs for each multimodal acquisition node. These logs record medical device operation events, including the collaborative transmission duration of multimodal data. This collaborative transmission duration is the time from when multimodal data is sent from the acquisition end to when the fusion end completes the multimodal data fusion. The system then stores the multimodal interaction logs for each multimodal acquisition node. As the fusion end, and the j-th multimodal acquisition node When acting as a data acquisition terminal, the duration of the generated collaborative data transmission is denoted as... .
3. The intelligent analysis and monitoring method for medical device data based on multimodality according to claim 2, characterized in that, The specific implementation process of step S2 includes: The operation events of medical equipment are uniformly compiled, and the k-th operation event of medical equipment is denoted as... Multimodal acquisition nodes It serves as a monitoring center for the converged terminals and, based on the duration of collaborative data transmission, statistically analyzes medical device operational events. All acquisition terminals below generate multimodal acquisition nodes. Let the set of behaviors analyzed at the fusion end be denoted as ,in, This represents the total number of multimodal acquisition nodes. Data modality tags are used to indicate the duration of collaborative data transmission. Inside, multimodal acquisition nodes As a fusion terminal and multimodal acquisition node The actual amount of data fusion when used as a collection end; Initial configuration within a unit time range, multimodal acquisition nodes As a fusion terminal and multimodal acquisition node Initial data fusion volume when used as a data acquisition end ; Based on the initial data fusion volume Integration volume with actual data Evaluation of multimodal acquisition nodes As a fusion terminal and multimodal acquisition node Data transmission frequency when acting as a data acquisition terminal .
4. The intelligent analysis and monitoring method for medical device data based on multimodality according to claim 3, characterized in that, The specific implementation process of step S3 includes: Under the same interactive behavior, i.e., multimodal acquisition nodes As a fusion terminal and multimodal acquisition node As the data acquisition endpoint, the same interactive behavior is recorded during medical device operation events. The frequency of data transmission in the data is denoted as ; Calculate the average frequency of data transmission. Where U represents the total number of medical equipment operation events; The range of encoding sequence numbers for multimodal acquisition nodes Build medical device operation events for row and column indexes. The data transmission state matrix, if the data transmission frequency In the data transmission state matrix, the element in the i-th row and j-th column is marked as 0, which is used to characterize the multimodal acquisition node. As a fusion terminal and multimodal acquisition node The interaction behavior of the acquisition end is redundant; if the data transmission frequency is high... In the data transmission state matrix, the matrix element in the i-th row and j-th column is marked as 1, which is used to characterize the multimodal acquisition node. As a fusion terminal and multimodal acquisition node The interactive behavior of the acquisition end is in a gain state; Then the medical equipment operation event is obtained. Data transmission state matrix .
5. The intelligent analysis and monitoring method for medical device data based on multimodality according to claim 4, characterized in that, The specific implementation process of step S4 includes: Based on the data transmission state matrix, the concurrency of multimodal acquisition nodes among medical device operation events is evaluated. In the formula, This represents the g-th medical device operation event. Indicates medical device operation events The data transmission state matrix, Represents the data transmission state matrix With data transmission state matrix The number of 1s contained in the result of an inter-Boolean logical AND operation. Represents the data transmission state matrix With data transmission state matrix The number of 1s contained after a Boolean logical OR operation; A preset concurrent similarity threshold is set to determine the concurrency of multimodal acquisition nodes. If the similarity is greater than or equal to the concurrent similarity threshold, then a medical device operation event is determined. Medical equipment operation events There are concurrent collaborative relationships between multimodal acquisition nodes, and the operation events of each medical device with concurrent collaborative relationships between multimodal acquisition nodes are output.
6. A multimodal intelligent analysis and monitoring system for medical device data, executing the multimodal intelligent analysis and monitoring method for medical device data as described in any one of claims 1-5, characterized in that, The system includes: a multimodal data acquisition and storage module, a runtime event processing and frequency calculation module, a data transmission state matrix construction module, and a multimodal acquisition node concurrent collaborative evaluation module; The multimodal data acquisition and storage module is used to set up multimodal data acquisition nodes and perform unified encoding, store multi-source monitoring data and multimodal interaction logs, and record the duration of data collaborative transmission. The event processing and frequency calculation module is used to uniformly compile medical equipment operation events, generate analysis behavior sets, initialize data fusion volume, and calculate data transmission frequency. The data transmission state matrix construction module is used to obtain the transmission frequency under the same interaction behavior and calculate the average transmission frequency, construct the data transmission state matrix and mark the corresponding transmission state. The multimodal acquisition node concurrent collaboration evaluation module evaluates the concurrency of multimodal acquisition nodes among different medical device operation events based on the data transmission status matrix, determines the concurrent collaboration relationship based on the concurrency of multimodal acquisition nodes, and outputs the relevant medical device operation events.
7. The intelligent analysis and monitoring system for medical device data based on multimodality according to claim 6, characterized in that, The multimodal data acquisition and storage module is specifically used for: setting up multimodal data acquisition nodes in the medical device operation process; uniformly encoding all acquisition nodes, with each acquisition node corresponding to the data acquisition end or data fusion end of the medical device; storing multi-source monitoring data of each acquisition node, which includes acquisition end data and fusion end data related to images, sensors, physiological parameters, and device operation logs, wherein the fusion end can correspond to a combination of one or more acquisition ends; and simultaneously storing multimodal interaction logs of each acquisition node, which record medical device operation events and data collaborative transmission duration, wherein the collaborative transmission duration is defined as the total time from the acquisition end sending out multimodal data to the fusion end completing data fusion.
8. The intelligent analysis and monitoring system for medical device data based on multimodality according to claim 6, characterized in that, The operation event processing and frequency calculation module is specifically used for: uniformly compiling medical equipment operation events, establishing a monitoring center with each multimodal acquisition node as the fusion end, statistically analyzing all acquisition ends corresponding to each operation event based on the data collaborative transmission duration, and generating the analysis behavior set corresponding to the fusion end; The initial data fusion volume when the acquisition end and fusion end cooperate within the initial unit time range is calculated. Combined with the actual data fusion volume recorded in the analysis behavior set, the data transmission frequency between the acquisition end and fusion end is calculated.
9. The intelligent analysis and monitoring system for medical device data based on multimodality according to claim 6, characterized in that, The data transmission state matrix construction module is specifically used for: collecting the data transmission frequency of the same interactive behavior across all medical device operation events, and calculating the average transmission frequency corresponding to the interactive behavior; constructing a data transmission state matrix corresponding to each medical device operation event using the encoding range of the multimodal acquisition node as the row and column index; comparing the frequency of a single data transmission with the average value, and marking the corresponding position in the matrix as a value representing a redundant state if it is lower than the average value, and marking it as a value representing a gain state if it is higher than or equal to the average value.
10. The intelligent analysis and monitoring system for medical device data based on multimodality according to claim 6, characterized in that, The multimodal acquisition node concurrent collaborative evaluation module is specifically used to: extract the data transmission state matrix corresponding to any two medical device operation events, calculate the number of values representing the gain state after the two matrices are ANDed by Boolean logic, and the number of values representing the gain state after the two matrices are ORed by Boolean logic, and obtain the multimodal acquisition node concurrency between the two operation events by the ratio of the two. A preset concurrency similarity threshold is set. When the calculated concurrency is greater than or equal to the threshold, it is determined that there is a multimodal acquisition node concurrent collaboration relationship between two medical device operation events, and all medical device operation events with this relationship are output.