Multi-source heterogeneous data processing method in the medical device field based on multimodal large model

Abstract

This invention provides a method for processing multi-source heterogeneous data in the medical device field based on a multimodal large model, relating to the field of intelligent monitoring of medical devices. The method includes: acquiring device image, audio, and parameter data and inputting them into a multimodal large model; fusing them to generate multidimensional feature representations; constructing a component operation mode library for state matching; analyzing state evolution chains to identify abnormal propagation paths and locate faults; calculating task migration costs to generate scheduling schemes; and achieving lossless system transformation. This invention effectively improves the accuracy of medical device fault diagnosis, reduces the risk of system interruption, and ensures the safety of clinical applications.

Description

Technical Field

[0001] This invention relates to intelligent monitoring technology for medical devices, and more particularly to a method for processing multi-source heterogeneous data in the field of medical devices based on a multimodal large model. Background Technology

[0002] With the continuous development of medical technology, medical devices are playing an increasingly important role in medical diagnosis and treatment. Modern medical devices typically integrate multiple functional modules and components, forming complex system architectures. Their stability and reliability are directly related to patient safety and treatment outcomes. During operation, medical devices generate a large amount of multi-source heterogeneous data, including image data, sound data, and various parameter data. This data contains rich information about the device's operating status, and effectively processing and analyzing this data is crucial for ensuring the stable operation of medical devices. In recent years, the development of artificial intelligence technology, especially multimodal large-scale models, has provided new technical means and methodologies for processing multi-source heterogeneous data from medical devices.

[0003] Existing medical device data processing technologies often analyze single-modal data, making it difficult to effectively integrate multimodal data such as images, audio, and parameters. This results in insufficient information utilization and an inability to fully reflect the true operating status of the equipment. Most existing fault diagnosis systems rely on pre-set rule bases or simple statistical models, lacking in-depth analysis of fault propagation paths and inter-component relationships. This makes it difficult to accurately locate the fault source and its impact range, leading to low maintenance efficiency. When a component fails in a medical device, current technologies typically employ a complete system shutdown, lacking intelligent task migration and functional reconfiguration mechanisms. This prevents lossless system transition and continuous service, potentially causing serious consequences in emergency medical situations. Summary of the Invention

[0004] This invention provides a method for processing multi-source heterogeneous data in the field of medical devices based on a multimodal large model, which can solve the problems in the prior art.

[0005] A first aspect of this invention provides a method for processing multi-source heterogeneous data in the medical device field based on a multimodal large model, comprising:

[0006] Image data, audio data, and parameter data of medical devices are collected and input into a multimodal large model. The multimodal large model uses a cross-modal attention mechanism to extract modal features, constructs semantic mapping relationships between modalities through contrastive learning, and fuses them to generate multidimensional feature representations.

[0007] A component operation mode library is constructed based on multidimensional feature representation. The real-time operation data of the component is matched with the component operation mode library, and the status description information is output.

[0008] Extract component operation metrics and component status scores from status description information, construct a status evolution chain, identify anomaly propagation paths by analyzing the status score change trends in the status evolution chain, and locate fault source components and fault-related components.

[0009] For fault source components and fault-related components, extract the component functional completeness and task capacity, calculate the task migration cost between components, and generate a scheduling scheme.

[0010] The working mode of medical devices is reconstructed according to the scheduling scheme, and the tasks of the fault source components are allocated to the stable components according to their functional dependencies, so as to achieve lossless conversion of the system.

[0011] Image, audio, and parameter data of medical devices are collected and input into a multimodal large model. This model employs a cross-modal attention mechanism to extract modal features, constructs intermodal semantic mapping relationships through contrastive learning, and fuses these features to generate a multidimensional feature representation, including:

[0012] Image data, audio data, and parameter data of medical devices are collected, and the image data, audio data, and parameter data are preprocessed to obtain preprocessed multimodal data.

[0013] The preprocessed multimodal data is input into a multimodal large model. The multimodal large model performs convolution operations on each modality to generate a corresponding feature map, calculates the attention score of the feature map in the channel dimension and the attention score in the spatial dimension, and generates a weight matrix.

[0014] The weighted matrix and the feature map are weighted to obtain the enhanced modal features;

[0015] Based on the enhanced modal features, the similarity difference between modal features is calculated, the sample with the largest feature difference is selected to construct a contrast learning sample pair, the feature distribution difference of the contrast learning sample pair is calculated, and normalization is performed to obtain the adaptive temperature coefficient.

[0016] In a multimodal large model, the adaptive temperature coefficient is used to perform semantic alignment and fusion operations on the enhanced modal features to generate multidimensional feature representations.

[0017] The multimodal large model includes:

[0018] A training sample library is constructed by collecting multimodal data of medical devices under different operating conditions. The training sample library includes normal operation data and fault data.

[0019] Feature decoupling is performed on the multimodal data in the training sample library to extract component-level features and system-level features. A cross-modal attention module is trained based on the component-level features, and a contrastive learning module is trained based on the system-level features.

[0020] The attention module is trained using component state labels as supervision information, and the contrastive learning module is trained using semantic consistency between modal features as a constraint.

[0021] A multi-task training objective is constructed, which includes fault identification loss and feature alignment loss. An alternating optimization strategy is adopted to iteratively train the parameters of the attention module and the contrastive learning module until the model converges, resulting in a multimodal large model for medical device fault diagnosis.

[0022] A component runtime mode library is constructed based on multidimensional feature representation. Real-time component runtime data is matched with the component runtime mode library, and the output state description information includes:

[0023] The multidimensional feature representation is segmented into time windows to obtain feature vectors. Fluctuation features are extracted from the feature vectors, and the fluctuation features are sorted according to their amplitudes to obtain sorted fluctuation features.

[0024] A dynamic density threshold is calculated based on the ranking fluctuation characteristics, and the feature vector is density-clustered using the dynamic density threshold to obtain the operating mode, which includes the fluctuation characteristic distribution.

[0025] The degree of dispersion of the operating mode is calculated based on the fluctuation characteristic distribution. The dynamic density threshold is adjusted based on the degree of dispersion. The optimized operating mode is obtained by clustering the feature vector using the adjusted dynamic density threshold.

[0026] Calculate the confidence level of the optimized running mode, and build a component running mode library from the optimized running modes with confidence levels higher than the preset confidence threshold;

[0027] The same time window is used to extract fluctuation features from the real-time running data of the component, and the fluctuation features are matched with the running modes in the component running mode library. Based on the matching results, status description information including running indicators and status scores is output.

[0028] Component operation metrics and component status scores are extracted from the status description information to construct a status evolution chain. Anomaly propagation paths are identified by analyzing the status score change trends within the status evolution chain, and fault source components and associated components are located, including:

[0029] Extract component operation metrics and component state scores, calculate the temporal correlation of component operation metrics to generate a functional dependency matrix, and determine the component state propagation path based on the functional dependency matrix and component physical connection relationships;

[0030] The component operation indicators and component status scores are organized into a component status sequence according to the time series. The difference values ​​of the component status sequence at adjacent time points are calculated to generate a state change matrix. A state evolution chain is constructed based on the state change matrix and the component state transmission path.

[0031] Mark the state anomalies in the state evolution chain whose state score changes exceed a preset threshold, construct an anomaly propagation sequence in chronological order, and identify the anomaly propagation path based on the anomaly propagation sequence and the component state transmission path;

[0032] The component that is the earliest to show an abnormal state point in the abnormal propagation path and has the largest change in state score is identified as the fault source component. Starting from the fault source component, components with abnormal state scores along the abnormal propagation path are identified as fault associated components.

[0033] For the fault source component and fault-related components, extract the component's functional completeness and task capacity, calculate the task migration cost between components, and generate a scheduling scheme including:

[0034] For fault source components and fault-related components, the execution parameters and performance indicators of the components are collected, and the functional integrity of the components is extracted based on the status values ​​of the execution parameters and the change values ​​of the performance indicators.

[0035] The computational load and response time of the collected components are used to extract the component task load based on the changes in the utilization rate of the computational load and the response time.

[0036] The task migration cost between components is calculated based on the difference in functional integrity and the ratio of task capacity between adjacent components, and the task migration cost is constructed into a cost matrix that reflects the difficulty of component task migration.

[0037] The task migration path with the minimum total cost is determined in the cost matrix. The functional integrity and task capacity of the components on the task migration path are constrained and verified. A scheduling scheme is generated based on the verification results.

[0038] The medical device operating mode is reconstructed according to the scheduling scheme, and the tasks of the faulty component are allocated to the stable component according to the functional dependency, achieving a lossless system conversion, including:

[0039] Extract the functional unit sequence of the fault source component according to the scheduling scheme, count the data flow frequency and resource sharing duration between each function in the functional unit sequence, and generate a functional dependency matrix that records the dependency relationship between functional units.

[0040] Obtain the resource utilization rate and task execution records of each component in the system, filter components that meet the preset resource threshold as target components based on the functional dependency matrix, and determine the task allocation order based on the remaining resource capacity of the target components.

[0041] The tasks of the fault source component are sorted according to the dependency relationship values ​​in the functional dependency matrix, and the sorted tasks are assigned to the target component in turn.

[0042] Generate a snapshot of the task status and perform migration according to the task allocation results to complete the synchronous switching of data status and functional status, and achieve lossless system conversion.

[0043] A second aspect of the present invention provides an electronic device, comprising:

[0044] processor;

[0045] Memory used to store processor-executable instructions;

[0046] The processor is configured to invoke instructions stored in the memory to execute the aforementioned method.

[0047] A third aspect of the present invention provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the aforementioned method.

[0048] In this embodiment, by employing a multimodal large model to process the multi-source heterogeneous data of medical devices, this invention achieves the fusion analysis of image, audio, and parameter data, improving the comprehensive perception capability of the medical device's operating status and enhancing the system's adaptability to complex working environments. The state evolution chain constructed by this invention can accurately identify anomaly propagation paths, accurately locate fault source components and fault-related components, improve the fault diagnosis efficiency of the medical device system, reduce the false positive rate, and provide maintenance personnel with more accurate fault handling basis. By evaluating the functional integrity and task capacity of components, calculating the task migration cost between components, and generating an optimal scheduling scheme, this invention achieves lossless transformation and functional reconstruction of the medical device system under fault conditions, ensuring the continuous and stable operation of the medical device when facing component failures, and improving the reliability and safety of the medical device. Attached Figure Description

[0049] Figure 1 This is a flowchart illustrating the multi-source heterogeneous data processing method for the medical device field based on a multimodal large model, according to an embodiment of the present invention.

[0050] Figure 2 The flowchart for generating the scheduling scheme in an embodiment of the present invention is shown. Detailed Implementation

[0051] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0052] The technical solution of the present invention will be described in detail below with reference to specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0053] Figure 1 This is a flowchart illustrating a multi-source heterogeneous data processing method for the medical device field based on a multimodal large model, as described in an embodiment of the present invention. Figure 1 As shown, the method includes:

[0054] Image data, audio data, and parameter data of medical devices are collected and input into a multimodal large model. The multimodal large model uses a cross-modal attention mechanism to extract modal features, constructs semantic mapping relationships between modalities through contrastive learning, and fuses them to generate multidimensional feature representations.

[0055] A component operation mode library is constructed based on multidimensional feature representation. The real-time operation data of the component is matched with the component operation mode library, and the status description information is output.

[0056] Extract component operation metrics and component status scores from status description information, construct a status evolution chain, identify anomaly propagation paths by analyzing the status score change trends in the status evolution chain, and locate fault source components and fault-related components.

[0057] For fault source components and fault-related components, extract the component functional completeness and task capacity, calculate the task migration cost between components, and generate a scheduling scheme.

[0058] The working mode of medical devices is reconstructed according to the scheduling scheme, and the tasks of the fault source components are allocated to the stable components according to their functional dependencies, so as to achieve lossless conversion of the system.

[0059] In one optional implementation, image data, audio data, and parameter data of a medical device are collected and input into a multimodal large model. The multimodal large model employs a cross-modal attention mechanism to extract modal features, constructs inter-modal semantic mapping relationships through contrastive learning, and fuses these features to generate a multidimensional feature representation, including:

[0060] Image data, audio data, and parameter data of medical devices are collected, and the image data, audio data, and parameter data are preprocessed to obtain preprocessed multimodal data.

[0061] The preprocessed multimodal data is input into a multimodal large model. The multimodal large model performs convolution operations on each modality to generate a corresponding feature map, calculates the attention score of the feature map in the channel dimension and the attention score in the spatial dimension, and generates a weight matrix.

[0062] The weighted matrix and the feature map are weighted to obtain the enhanced modal features;

[0063] Based on the enhanced modal features, the similarity difference between modal features is calculated, the sample with the largest feature difference is selected to construct a contrast learning sample pair, the feature distribution difference of the contrast learning sample pair is calculated, and normalization is performed to obtain the adaptive temperature coefficient.

[0064] In a multimodal large model, the adaptive temperature coefficient is used to perform semantic alignment and fusion operations on the enhanced modal features to generate multidimensional feature representations.

[0065] In this embodiment, image data, audio data, and parameter data of the medical device are first acquired. Image data is acquired by using a high-definition camera to capture visible light images of the medical device during operation, with a resolution of 1920×1080 pixels and a frame rate of 30 frames per second; audio data is acquired by using an omnidirectional microphone array to acquire sound signals generated during the operation of the medical device, with a sampling rate of 48kHz and a quantization bit depth of 16 bits; parameter data is acquired by using sensors to acquire operating parameters of the medical device such as temperature, vibration, and current, with a sampling frequency of 100Hz.

[0066] After acquisition, the multimodal data underwent preprocessing. Image data was normalized, with pixel values ​​scaled to the [-1, 1] range, and randomly cropped to 224×224 pixels to meet model input requirements. Audio data was denoised by using short-time Fourier transform to extract spectral features and convert the audio signal into a Mel spectrogram of size 128×128. Parameter data was standardized to ensure that all parameters were distributed within a uniform range, and missing value imputation and outlier handling were performed to ensure data quality.

[0067] The preprocessed multimodal data is input into a large multimodal model for further processing. This model first applies convolutional operations to each modality to generate corresponding feature maps. For image data, a 5-layer convolutional network is applied with a kernel size of 3×3 and a stride of 1, resulting in an output feature map size of 14×14×512. For audio Mel-ray spectrograms, a 4-layer convolutional network is applied with a kernel size of 3×3 and a stride of 1, resulting in an output feature map size of 8×8×256. For parametric data, a 3-layer fully connected network is applied to convert it into a 128-dimensional feature vector.

[0068] After feature extraction, attention scores for the feature maps in the channel and spatial dimensions are calculated. Channel-dimensional attention compresses the feature maps into channel descriptors through global average pooling, and then generates channel weights through two fully connected layers, with weight values ​​ranging from [0, 1]. Spatial-dimensional attention generates spatial feature maps through cross-channel pooling, and then obtains spatial weight matrices through convolution operations, with weight values ​​ranging from [0, 1] at each position.

[0069] The calculated weight matrix is ​​then weighted and multiplied with the original feature map. For the image feature map, the channel weights are multiplied with the feature map, and then multiplied with the spatial weight matrix to obtain the enhanced image features. Audio feature maps and parametric features are processed in the same way to obtain the corresponding enhanced features. For example, if the original image feature value is 0.75, and the corresponding channel weight is 0.9 and the spatial weight is 0.8, then the enhanced feature value is 0.75 × 0.9 × 0.8 = 0.54.

[0070] Based on the enhanced modal features, the model calculates the similarity differences between different modal features. A similarity matrix is ​​constructed by calculating the cosine similarity between feature vectors. For example, the similarity between image features and audio features is 0.65, the similarity between image features and parametric features is 0.72, and the similarity between audio features and parametric features is 0.58. According to the similarity matrix, the sample pair with the largest feature difference, i.e., the modal pair with the lowest similarity (in the example above, the audio feature and parametric feature pair), is selected as the sample pair for contrastive learning. The feature distribution difference is calculated for the selected sample pair. Specifically, the square root of the sum of the squares of the differences in each dimension of the two feature vectors is calculated to obtain the Euclidean distance value. In the example above, the Euclidean distance between the audio feature and the parametric feature is 1.75. This distance value is normalized and transformed to the interval [0.1, 10] to obtain the adaptive temperature coefficient.

[0071] Using the calculated adaptive temperature coefficient, semantic alignment and fusion are performed on the enhanced modal features in a multimodal large model. During semantic alignment, each modal feature is projected onto a shared semantic space, with the projection matrix size determined based on the dimension of each modal feature. For example, image features (14×14×512) are projected into a 512-dimensional vector, audio features (8×8×256) are projected into a 512-dimensional vector, and parametric features (128-dimensional) are projected into a 512-dimensional vector.

[0072] In the semantic space, an adaptive temperature coefficient τ is used to adjust the distribution of different modal features, and multimodal features are fused through a weighted summation method. The weights are dynamically allocated based on the reliability and information content of each modal feature; for example, image features have a weight of 0.4, audio features have a weight of 0.3, and parametric features have a weight of 0.3. The final generated multidimensional feature representation has a dimension of 512, containing comprehensive information about the medical device.

[0073] Through the above processing, the multimodal large model can effectively integrate the image, audio and parameter information of medical devices to generate high-quality multidimensional feature representations, providing strong support for subsequent equipment status monitoring, fault diagnosis and predictive maintenance, and greatly improving the efficiency of medical device management and maintenance.

[0074] In one alternative implementation, the multimodal large model includes:

[0075] A training sample library is constructed by collecting multimodal data of medical devices under different operating conditions. The training sample library includes normal operation data and fault data.

[0076] Feature decoupling is performed on the multimodal data in the training sample library to extract component-level features and system-level features. A cross-modal attention module is trained based on the component-level features, and a contrastive learning module is trained based on the system-level features.

[0077] The attention module is trained using component state labels as supervision information, and the contrastive learning module is trained using semantic consistency between modal features as a constraint.

[0078] A multi-task training objective is constructed, which includes fault identification loss and feature alignment loss. An alternating optimization strategy is adopted to iteratively train the parameters of the attention module and the contrastive learning module until the model converges, resulting in a multimodal large model for medical device fault diagnosis.

[0079] This invention provides an implementation method for a multimodal large-scale model for fault diagnosis of medical devices. First, multimodal data acquisition is performed, constructing a training sample library by collecting data under different operating conditions of the medical device. Taking medical imaging equipment as an example, multimodal data such as sound signals, vibration signals, temperature data, current data, and system logs can be collected during equipment operation. Specifically, a microphone array is used to collect sound signals at a sampling rate of 44.1 kHz; a triaxial accelerometer is used to collect vibration signals at a sampling frequency of 10 kHz; an infrared temperature sensor is used to collect the temperature of key components at a sampling frequency of 1 Hz; a Hall current sensor is used to collect the motor drive current at a sampling frequency of 5 kHz; and system operation logs are recorded simultaneously. For each modality of data, normal operation data and ten common fault data are labeled, including bearing wear, motor overheating, power supply failure, and loose transmission components. The data acquisition process lasts for 30 days, accumulating approximately 1000 hours of operating data, of which normal data accounts for 70% and fault data accounts for 30%.

[0080] The acquired multimodal data underwent preprocessing, including denoising, standardization, and time alignment. Wavelet transform was applied to the audio signal to remove environmental noise, a Butterworth low-pass filter was applied to the vibration signal to remove high-frequency interference, and mean normalization was performed on the temperature and current data. Then, the modal data were aligned according to a unified timestamp to form multimodal data samples. Each sample contains a 10-second multimodal data segment and its corresponding device status label.

[0081] Feature decoupling is performed to extract component-level and system-level features. For sound signals, short-time Fourier transform is used to extract spectral features with a window size of 512 points and a step size of 256 points. For vibration signals, time-domain statistical features (mean, standard deviation, peak factor, kurtosis) and frequency-domain features (dominant frequency, frequency band energy distribution) are calculated. For temperature data, features such as mean, rate of change, and fluctuation range are extracted. For current data, features such as RMS value, harmonic content, and fluctuation characteristics are extracted. For system logs, keyword vectors are extracted using text embedding technology. Component-level features focus on the local characteristics of each component, including the vibration characteristics of bearings, the temperature characteristics of motors, and the current characteristics of power supplies. System-level features focus on the overall operating status, including macroscopic information such as equipment operating mode, load level, and environmental conditions.

[0082] A cross-modal attention module is trained based on component-level features. This module employs a multi-head attention mechanism to achieve interaction between features from different modalities. For each modal feature, a query, key, and value matrix is ​​generated through linear mapping. The number of attention heads is set to 8, and the feature dimension of each head is 64. During the attention calculation process, attention weights are applied to different modal features, enabling the model to focus on the modalities and features with the most diagnostic value. For example, bearing failures are mainly reflected in vibration and sound modes, while motor failures are more reflected in temperature and current modes. The input dimension of the attention module is the sum of the dimensions of each modal feature, and the output dimension is 512. Feature fusion is performed through a multilayer perceptron, and the final output is a fused feature vector.

[0083] The contrastive learning module is trained based on system-level features. This module employs a dual-tower structure, with each tower consisting of a multilayer perceptron. The input is system-level features, and the output is a 128-dimensional feature vector. Positive sample pairs are constructed for features of different modalities under the same device state, and negative sample pairs are constructed for features of different device states. The temperature feature mapping tower contains three fully connected layers with 256, 128, and 128 nodes respectively; the vibration feature mapping tower also contains three fully connected layers with the same number of nodes. Through contrastive learning, features of different modalities under the same state are made semantically closer together, while features of different states are made farther apart, enhancing the model's ability to understand device states.

[0084] The attention module is trained using component state labels as supervised information. For each training sample, the fused features are used to predict the component state through a classifier. The classifier consists of a three-layer fully connected network with 256, 128, and 11 nodes respectively (corresponding to 10 fault types and 1 normal state). The cross-entropy loss function is used to measure the difference between the predicted results and the true labels, guiding the attention module to learn the importance weights of features of different modalities.

[0085] The contrastive learning module is trained with semantic consistency between modal features as a constraint. Cosine similarity is calculated for different modal features of the same sample after mapping, and a contrastive loss function is constructed. For positive sample pairs, their similarity is increased; for negative sample pairs, their similarity is decreased. The contrastive learning temperature parameter is set to 0.07, and the sample batch size is 64. By maximizing the mutual information of different modal features, the model's ability to understand and represent multimodal data is enhanced.

[0086] A multi-task training objective is constructed, comprising fault identification loss and feature alignment loss. The weight of the fault identification loss is set to 0.7, and the weight of the feature alignment loss is set to 0.3. An alternating optimization strategy is adopted: in each training batch, the parameters of the contrastive learning module are fixed first to update the attention module, and then the parameters of the attention module are fixed to update the contrastive learning module. The Adam optimizer is used, with an initial learning rate of 0.001, which decays to 0.9 times every 50 training epochs. The training process lasts for 200 epochs with a batch size of 32, until the loss function value on the validation set no longer decreases for 10 consecutive epochs, indicating model convergence, and finally, a multimodal large model is obtained.

[0087] In this embodiment, the following technical effects were achieved by constructing a multimodal large-scale model of medical devices: a comprehensive training sample library was established using multimodal data under different operating conditions; component-level and system-level features were separated through feature decoupling, enhancing the model's ability to identify faults with greater precision; the cross-modal attention module effectively captured the interrelationships between different modal data, improving the model's understanding of complex fault modes; the contrastive learning module ensured semantic consistency between features of different modalities, enhancing the model's robustness; and the design of multi-task training objectives balanced the needs of fault identification and feature alignment, while the alternating optimization strategy avoided bias in the model during training.

[0088] In one optional implementation, a component runtime mode library is constructed based on multidimensional feature representation. Real-time component runtime data is matched against the component runtime mode library, and the output state description information includes:

[0089] The multidimensional feature representation is segmented into time windows to obtain feature vectors. Fluctuation features are extracted from the feature vectors, and the fluctuation features are sorted according to their amplitudes to obtain sorted fluctuation features.

[0090] A dynamic density threshold is calculated based on the ranking fluctuation characteristics, and the feature vector is density-clustered using the dynamic density threshold to obtain the operating mode, which includes the fluctuation characteristic distribution.

[0091] The degree of dispersion of the operating mode is calculated based on the fluctuation characteristic distribution. The dynamic density threshold is adjusted based on the degree of dispersion. The optimized operating mode is obtained by clustering the feature vector using the adjusted dynamic density threshold.

[0092] Calculate the confidence level of the optimized running mode, and build a component running mode library from the optimized running modes with confidence levels higher than the preset confidence threshold;

[0093] The same time window is used to extract fluctuation features from the real-time running data of the component, and the fluctuation features are matched with the running modes in the component running mode library. Based on the matching results, status description information including running indicators and status scores is output.

[0094] In the process of obtaining feature vectors by segmenting multidimensional feature representations into time windows, a sliding window method can be used for data segmentation. A window length of 24 hours and a window sliding step of 1 hour are set, and multidimensional sensor data such as temperature, current, and vibration of the CT equipment cooling system are collected. Statistical characteristics such as mean, standard deviation, maximum, and minimum values ​​are calculated for the data within each window to form a feature vector. When extracting fluctuation features from the feature vector, the rate of change can be obtained by calculating the difference between feature values ​​at adjacent time points, extracting information such as fluctuation amplitude, fluctuation frequency, and fluctuation duration. The fluctuation features are sorted according to their amplitude from largest to smallest to obtain sorted fluctuation features, facilitating subsequent analysis of fluctuation patterns of higher importance.

[0095] In the process of calculating the dynamic density threshold based on the ranking fluctuation characteristics, an adaptive density threshold determination method is introduced. The local density of sample points in the feature space is calculated, and statistical analysis of the density values ​​yields a density distribution histogram. The initial density threshold is determined by analyzing the peak and trough characteristics of the density distribution histogram. This threshold is not fixed but dynamically adjusted according to the data distribution characteristics. When using the dynamic density threshold to perform density clustering on the feature vectors, a density-based spatial clustering algorithm is employed to divide density-connected regions into clusters. The local density of each point in the feature space is calculated, and points with densities higher than the dynamic threshold are identified as cluster centers. The remaining points are then assigned to their respective clusters based on density reachability principles. After clustering, each cluster represents an operating mode, containing a specific fluctuation characteristic distribution.

[0096] When calculating the dispersion of the operating mode based on the fluctuation characteristic distribution, a dispersion metric is used as the standard for measuring the dispersion of points in the feature space. The ratio of the average distance between points within a cluster to the average distance between points in different clusters is calculated as the standard for measuring the dispersion. If the dispersion is too high, it indicates poor clustering performance, and the dynamic density threshold needs to be adjusted. The adjustment method is as follows: when the dispersion is higher than the preset threshold, the dynamic density threshold is lowered to merge similar clusters; when the dispersion is lower than the preset threshold, the dynamic density threshold is increased to subdivide the clusters. The feature vectors are then re-clustered using the adjusted dynamic density threshold to obtain the optimized operating mode.

[0097] When calculating the confidence score of an optimized operating mode, factors such as the number of samples within a cluster, cluster stability, and cluster density are comprehensively evaluated. A larger number of samples within a cluster indicates a higher frequency of occurrence of the operating mode; cluster stability is measured by the consistency of multiple clustering results; and cluster density is calculated by the average distance between samples within a cluster. These factors are weighted and summed to obtain the confidence score. Optimized operating modes with confidence scores higher than a preset confidence threshold are included in the component operating mode library. For the cooling system of medical imaging equipment, a confidence threshold of 0.85 can be set to ensure high reliability of the operating modes in the mode library.

[0098] During the extraction of fluctuation features from real-time operational data of components using the same time window, the same feature extraction method as in the modeling phase is maintained to ensure feature consistency and comparability. For real-time operating medical device components, data is collected in 24-hour windows to extract fluctuation features, which are then matched against existing operating modes in the operating mode library. The matching process employs a feature similarity calculation method to calculate the similarity score between the real-time fluctuation features and each operating mode in the mode library. The similarity calculation considers the degree of matching in fluctuation amplitude, frequency, and duration, assigning different weights to different features. Based on the operating mode with the highest similarity, the corresponding state description information is output.

[0099] The status description information includes two parts: operational metrics and status score. Operational metrics describe the deviation of the component's current operating status from its normal operating status, covering performance indicators, stability indicators, and lifespan indicators. The status score quantifies the component's health condition, divided into four levels: normal, slightly abnormal, moderately abnormal, and severely abnormal, corresponding to score ranges of 90-100, 70-90, 50-70, and 0-50, respectively. For real-time monitoring of medical equipment cooling systems, when the bearing vibration frequency characteristics match the abnormal pattern with a degree of 0.92, the status description information "Moderate abnormal bearing wear, status score 65, maintenance and inspection recommended within two weeks" is output.

[0100] In this embodiment, by constructing a component operation mode library based on multi-dimensional features, accurate identification and prediction of medical device operation status are achieved. The introduction of dynamic density thresholds solves the problem that traditional fixed threshold methods are difficult to adapt to the characteristics of variable data, improving clustering quality. The threshold adaptive adjustment mechanism based on the degree of dispersion enhances the model's ability to distinguish different operation states. Multi-factor confidence assessment ensures the high reliability of operation modes in the mode library and reduces the false positive rate. Real-time matching and status description output provide a scientific basis for equipment maintenance decisions, effectively reducing the risk of medical device failure and extending equipment lifespan.

[0101] In one optional implementation, component operation metrics and component state scores are extracted from state description information to construct a state evolution chain. Anomaly propagation paths are identified by analyzing the state score change trends within the state evolution chain, and fault source components and associated components are located, including:

[0102] Extract component operation metrics and component state scores, calculate the temporal correlation of component operation metrics to generate a functional dependency matrix, and determine the component state propagation path based on the functional dependency matrix and component physical connection relationships;

[0103] The component operation indicators and component status scores are organized into a component status sequence according to the time series. The difference values ​​of the component status sequence at adjacent time points are calculated to generate a state change matrix. A state evolution chain is constructed based on the state change matrix and the component state transmission path.

[0104] Mark the state anomalies in the state evolution chain whose state score changes exceed a preset threshold, construct an anomaly propagation sequence in chronological order, and identify the anomaly propagation path based on the anomaly propagation sequence and the component state transmission path;

[0105] The component that is the earliest to show an abnormal state point in the abnormal propagation path and has the largest change in state score is identified as the fault source component. Starting from the fault source component, components with abnormal state scores along the abnormal propagation path are identified as fault associated components.

[0106] In the monitoring of medical device components, it is necessary to extract component operating indicators and component status scores from the status description information. Quantitative indicator values ​​are separated from the structured status description information using text parsing technology. For medical imaging equipment, operating indicators such as X-ray tube temperature, current stability, and radiation dose can be extracted, along with corresponding status scores. The status scores use a quantification standard of 0-100, where 90-100 represents normal status, 70-90 represents slight abnormality, 50-70 represents moderate abnormality, and 0-50 represents severe abnormality. After extraction, the temporal correlation of component operating indicators is calculated to generate a functional dependency matrix. The temporal correlation calculation uses a sliding time window method with a window length of 72 hours, calculating the degree of interrelation between the operating indicator sequences of each component within the window. The correlation value ranges from 0 to 1, with a larger value indicating a stronger functional dependency between components. Taking medical imaging equipment as an example, the correlation between the X-ray tube temperature indicator and the cooling system flow rate indicator is 0.87, indicating a strong functional dependency between the two. The calculated inter-component correlation values ​​are filled into the corresponding positions in the functional dependency matrix to form a complete functional dependency graph.

[0107] When determining component state propagation paths based on the functional dependency matrix and component physical connections, the connection characteristics at both the functional and physical levels must be considered. Physical connections are obtained from the medical device structural design drawings, representing the direct physical connection states between components. The functional dependency matrix and physical connections are weighted and fused to determine possible state propagation paths. During the fusion process, a weight of 0.6 is assigned to functional dependencies and a weight of 0.4 to physical connections, and the overall propagation strength is calculated. When the overall propagation strength exceeds a threshold of 0.5, it is considered a valid state propagation path. In medical imaging equipment, the overall propagation strength between the X-ray tube and the high-voltage generator is 0.83, thus it is determined to be a state propagation path.

[0108] When organizing component operating indicators and component status scores into a component status sequence according to time series, a multidimensional time series structure is adopted. For each medical device component, the operating indicators and status scores are recorded hourly for 30 consecutive days to form a component status sequence. When calculating the differences between adjacent time points in the component status sequence to generate a status change matrix, the difference between the operating indicators and status scores at two adjacent time points in the status sequence is calculated. The difference calculation adopts the relative rate of change method to avoid direct comparison of indicators with different dimensions. For X-ray tubes, if the temperature indicator changes from 75℃ to 85℃ at a certain time, and the status score decreases from 90 to 75, the corresponding status change values ​​are 13.3% and -16.7%, respectively. The status change values ​​of each component at each time point are filled into the status change matrix, with the matrix rows representing different components and the columns representing different time points.

[0109] When constructing a state evolution chain based on the state change matrix and component state propagation paths, a temporal correlation analysis is performed on state changes and propagation paths. Along the defined state propagation path, state change events are connected in chronological order to form a complete state evolution chain. The state evolution chain is represented as a directed graph, where nodes represent component states, edges represent state propagation relationships, and propagation delays are labeled. For example, after an abnormal temperature in the X-ray tube of a medical imaging device, the cooling system experiences a flow rate decrease after 2 hours, followed by voltage fluctuations in the power supply system after 1.5 hours, forming a state evolution chain.

[0110] When an anomaly point in the state evolution chain is marked with a state score change amplitude exceeding a preset threshold, the preset threshold is set to 15%. When the relative rate of change of a component's state score exceeds this threshold, it is marked as an anomaly point. For a medical equipment control system, if the state score drops sharply from 95 to 75, the relative rate of change is -21.1%, exceeding the preset threshold, it is marked as an anomaly point. An anomaly propagation sequence is constructed chronologically, and the marked anomaly points are sorted by their occurrence time to form a time series of anomaly propagation. Combining the component state transmission path, the transmission relationship between anomaly points is analyzed to identify the anomaly propagation path. The component with the earliest state anomaly point and the largest change amplitude in the anomaly propagation path is identified as the fault source component. Taking medical CT equipment as an example, if the analysis finds that the high-voltage generator first exhibits an anomaly point with a state score dropping from 90 to 70 at 10:00 on August 15th, with a change amplitude of -22.2%, and is the earliest anomaly point in the anomaly propagation path, then the high-voltage generator is identified as the fault source component. Starting from the fault source component, components with abnormal state scores along the anomaly propagation path are marked as fault-related components. For medical CT equipment, along the abnormal propagation path starting from the high-voltage generator, the X-ray tube, cooling system, and control circuit are sequentially labeled as fault-related components, forming a complete set of fault-related components.

[0111] In this embodiment, by constructing a component state evolution chain, the precise location of the medical device fault source and the identification of fault-related components are achieved. A state transmission model that better reflects the actual working mechanism of the system is established, taking into account the functional dependencies and physical connection characteristics of the components. The anomaly propagation path identification mechanism based on time-series analysis effectively captures the fault propagation process within the system. This provides a comprehensive basis for maintenance decisions, avoids recurrence of faults after single-point repairs, improves the overall reliability and lifespan of medical equipment, reduces maintenance costs, and ensures the continuity and safety of medical services, thus having significant value in improving the quality of medical services.

[0112] like Figure 2 The diagram illustrates the scheduling scheme generation process of this embodiment.

[0113] In one optional implementation, for the fault source component and the fault-related components, the component functional integrity and task capacity are extracted, the task migration cost between components is calculated, and a scheduling scheme is generated, including:

[0114] For fault source components and fault-related components, the execution parameters and performance indicators of the components are collected, and the functional integrity of the components is extracted based on the status values ​​of the execution parameters and the change values ​​of the performance indicators.

[0115] The computational load and response time of the collected components are used to extract the component task load based on the changes in the utilization rate of the computational load and the response time.

[0116] The task migration cost between components is calculated based on the difference in functional integrity and the ratio of task capacity between adjacent components, and the task migration cost is constructed into a cost matrix that reflects the difficulty of component task migration.

[0117] The task migration path with the minimum total cost is determined in the cost matrix. The functional integrity and task capacity of the components on the task migration path are constrained and verified. A scheduling scheme is generated based on the verification results.

[0118] For fault-causing and fault-related components, a real-time monitoring framework for the operational status of medical device components needs to be constructed during the collection of execution parameters and performance indicators. For medical image processing workstations, monitored execution parameters include processing volume, data transmission rate, and cache utilization at each stage of the image processing pipeline; performance indicators include image sharpness index, noise ratio, and processing latency. The data collection cycle is set to once per minute, continuously recording for 24 hours to form time-series data of execution parameters and performance indicators. When extracting component functional completeness based on the status values ​​of execution parameters and the changes in performance indicators, a weighted evaluation method is used. Functional completeness characterizes the component's ability to perform its expected functions, ranging from 0 to 1, with higher values ​​indicating more complete functionality. The calculation method is as follows: compare the status values ​​of execution parameters with standard status values ​​to obtain parameter deviation; calculate the recent change rate of performance indicators; and combine parameter deviation and performance change rate to obtain functional completeness through a weighted average. In the weight allocation, the weight of execution parameters is 0.6, and the weight of performance indicators is 0.4. Taking the image reconstruction module of a medical image processing workstation as an example, when its data processing volume drops to 70% of the normal value and the image clarity index drops by 15%, the calculated functional integrity is 0.76, indicating that the module's function is partially affected but can still operate basically.

[0119] During the process of collecting data on the computational load and response time of the components, resource monitoring probes need to be deployed. Computational load metrics include CPU utilization, memory usage, and I / O throughput; response time metrics include task startup latency, processing time, and result return time. Data is collected every 5 seconds, and the results are aggregated to form load and response time curves. When extracting the component's task capacity based on the changes in computational load utilization and response time, a multi-dimensional evaluation model is constructed. Task capacity characterizes the component's ability to handle computational tasks, ranging from 0 to 1, with higher values ​​indicating stronger capacity. The calculation method is as follows: statistically analyze the average and peak utilization of the computational load; analyze the average and fluctuation range of the response time; and combine load utilization characteristics and response time characteristics to obtain the task capacity through nonlinear mapping. In the mapping weights, computational load accounts for 0.55, and response time accounts for 0.45. For the medical image 3D reconstruction module, when its average CPU utilization is 65%, peak utilization reaches 85%, average response time is 200 milliseconds, and fluctuation range is 50 milliseconds, the calculated task capacity is 0.72, indicating that the module has a medium-to-high level of task processing capability.

[0120] When calculating the task migration cost between components based on the functional integrity difference and task load ratio between adjacent components, a migration cost evaluation framework is designed. The task migration cost characterizes the cost required to migrate a task from one component to another, including performance loss, resource consumption, and risk factors. The calculation method is as follows: Obtain the absolute value of the functional integrity difference between components; a larger difference indicates a greater functional difference and higher migration difficulty. Calculate the task load ratio between components, such as the source component's load divided by the target component's load; a larger ratio indicates a heavier burden on the target component. Combine the functional difference and load ratio to obtain the task migration cost through a linear combination. In the combination coefficients, the functional difference coefficient is 0.65, and the load ratio coefficient is 0.35. If the functional integrity of the image segmentation module is 0.85 and that of the image recognition module is 0.75, with an absolute difference of 0.1; and the task load capacity of the image segmentation module is 0.8 and that of the image recognition module is 0.6, with a load capacity ratio of 1.33, then the task migration cost between the two modules is calculated as 0.1 × 0.65 + 1.33 × 0.35 × 0.1 = 0.112, indicating a relatively low migration difficulty. When constructing the task migration cost as a cost matrix reflecting the difficulty of component task migration, an n × n matrix is ​​created, where n is the number of components, and the matrix element (i, j) represents the cost of migrating a task from component i to component j. The diagonal elements of the matrix are set to infinity, indicating that migration of the component itself is meaningless.

[0121] Dijkstra's algorithm is applied to determine the task migration path with the minimum total cost in the cost matrix. The fault source component is set as the starting point, and candidate target components as the ending points. The shortest path is searched in the weighted directed graph formed by the cost matrix. The path search process considers the connection topology between components and the migration transit characteristics to avoid circular dependencies and migration deadlocks. For a data processing system of medical imaging equipment, if the image acquisition module is the fault source component, and the data storage module, image processing module, and result display module are possible migration targets, the minimum cost path calculated through the cost matrix is: image acquisition module → image processing module → data storage module, with a total cost of 0.187. When performing functional integrity and task capacity constraint verification on the components on the task migration path, a minimum threshold of 0.6 for functional integrity and a minimum threshold of 0.5 for task capacity are set. The verification process checks whether each component on the path meets the threshold requirements and evaluates whether the load distribution after migration is balanced. When generating a scheduling plan based on the verification results, a detailed task allocation strategy and migration sequence arrangement are formulated. The scheduling plan includes: priority ranking of migration tasks, resource reservation plan, migration time window setting, and rollback mechanism design. Taking a medical image analysis system as an example, the scheduling scheme stipulates that: data caching tasks of the image acquisition module will be migrated to the data storage module with high priority; image preprocessing tasks will be migrated to the image processing module with medium priority; the migration time window will be set from 2:00 AM to 4:00 AM; 30% of the computing resources of the image processing module will be reserved for migration tasks; and a rollback mechanism will be set when the performance drops by more than 20%.

[0122] In this embodiment, a scientific task migration decision-making mechanism was established by accurately assessing the functional integrity and task capacity of medical device components. The scheme fully considers the functional differences and load-bearing capacity among medical device components and constructs a cost evaluation model reflecting the actual migration difficulty. A task migration strategy based on the minimum cost path effectively balances the relationship between performance loss and resource utilization. A constraint verification mechanism ensures the feasibility and security of the scheduling scheme, preventing cascading failures caused by suboptimal decisions. The overall scheduling scheme has strong adaptability and reliability, enabling the rapid formulation of reasonable task redistribution strategies when critical components of medical devices fail, maximizing the maintenance of core equipment functions, reducing the impact of failures on medical services, improving the fault tolerance and service continuity of medical devices, providing more reliable equipment support for medical institutions, and ensuring uninterrupted patient treatment processes.

[0123] In one optional implementation, the medical device operating mode is reconstructed according to the scheduling scheme, and the tasks of the faulty component are allocated to the stable component according to their functional dependencies, achieving a lossless system conversion, including:

[0124] Extract the functional unit sequence of the fault source component according to the scheduling scheme, count the data flow frequency and resource sharing duration between each function in the functional unit sequence, and generate a functional dependency matrix that records the dependency relationship between functional units.

[0125] Obtain the resource utilization rate and task execution records of each component in the system, filter components that meet the preset resource threshold as target components based on the functional dependency matrix, and determine the task allocation order based on the remaining resource capacity of the target components.

[0126] The tasks of the fault source component are sorted according to the dependency relationship values ​​in the functional dependency matrix, and the sorted tasks are assigned to the target component in turn.

[0127] Generate a snapshot of the task status and perform migration according to the task allocation results to complete the synchronous switching of data status and functional status, and achieve lossless system conversion.

[0128] In this embodiment, when extracting the functional unit sequence of the fault source component according to the scheduling scheme, functional decomposition technology is required to perform fine-grained analysis of the fault source component. Taking the image reconstruction module of a medical image processing system as an example, the functional unit sequence of this module includes functional units such as data preprocessing, fast Fourier transform, back projection calculation, image filtering, and data post-processing. During the extraction process, a complete list of functional unit sequences is formed by parsing the component's functional configuration file, call relationship diagram, and resource allocation table. When statistically analyzing the data flow frequency and resource sharing duration between functions in the functional unit sequence, high-precision task monitoring probes are deployed to record the interaction between functional units during normal system operation.

[0129] Based on these interactive data, a functional dependency matrix is ​​generated, where each element represents the strength of the dependency relationship between two functional units. The dependency strength is calculated by weighting the data flow frequency and resource sharing duration, with a weight ratio of 6:4. For the functional dependency matrix of the image reconstruction module, the dependency strength between data preprocessing and Fast Fourier Transform is 0.78, and the dependency strength between Fast Fourier Transform and backprojection calculation is 0.65.

[0130] When acquiring resource utilization and task execution records for each component in the system, a distributed resource monitoring system is deployed to collect real-time metrics such as CPU utilization, memory utilization, I / O load, and network bandwidth usage. For the medical image processing system, the monitoring shows that the average CPU utilization of the image display module is 35%, with a peak of 60%; the average memory utilization is 28%, with a peak of 45%; the average CPU utilization of the storage module is 42%, with a peak of 68%; and the average memory utilization is 58%, with a peak of 72%. Simultaneously, task execution records for each component are collected, including task type, execution frequency, average execution time, and success rate. When selecting components that meet preset resource thresholds as target components based on the functional dependency matrix, the preset resource thresholds are set as follows: average CPU utilization not exceeding 60%, peak not exceeding 85%; average memory utilization not exceeding 70%, peak not exceeding 90%. Through threshold screening, the image display module and image storage module are determined to meet the resource conditions and can be used as target components. When determining the task allocation order based on the remaining resource capacity of the target components, the resource surplus of each target component is calculated, and components with larger resource surpluses are allocated tasks first. For the medical image processing system, the image display module has 40% of its CPU capacity and 55% of its memory capacity remaining; the storage module has 32% of its CPU capacity and 32% of its memory capacity remaining. After comprehensive evaluation, the task allocation order is determined to be image display module first, followed by storage module.

[0131] When sorting the tasks of the fault source components according to the dependency values ​​in the functional dependency matrix, a descending order is used, with tasks having higher dependency values ​​being assigned priority. For the image reconstruction module tasks, the dependency ranking is as follows: data preprocessing (0.78), backprojection calculation (0.65), image filtering (0.53), data postprocessing (0.41), and fast Fourier transform (0.38). When allocating the sorted tasks to the target components, the matching degree between the resource type requirements of the tasks and the resource structure of the target components is considered. The data preprocessing task mainly consumes CPU resources and is allocated to the image display module; the backprojection calculation task has high memory requirements and is allocated to the image display module with sufficient memory resources; the image filtering task consumes a lot of GPU resources and is allocated to the storage module with GPU acceleration capabilities; the data postprocessing and fast Fourier transform tasks are both allocated to the storage module. During the allocation process, the remaining resource capacity of each target component is dynamically updated to ensure that task allocation does not lead to resource overload.

[0132] When generating task status snapshots and performing migration according to task allocation results, a complete status snapshot is first created for each task before migration, recording key information such as task execution progress, memory status, file handles, and network connections. The status snapshot includes data batch ID 84572 being processed, completion progress at 63%, 12 open data files, and 278 active memory objects. Incremental snapshot technology is used to generate snapshots, recording only changed data to reduce snapshot size. When performing migration according to task allocation results, a gradual task migration strategy is implemented. First, the task runtime environment on the target component is created, loading necessary library files and configuration information; then, the task status snapshot is transferred to restore the task execution state; subsequently, a data channel is established, and data flow is redirected to ensure uninterrupted data transmission between tasks; finally, the migrated task is activated, and its runtime status is verified. When synchronizing the switching between data state and functional state, an atomic switching mechanism is used to ensure system consistency during the switching process. For medical image processing systems, when data preprocessing tasks are migrated from the image reconstruction module to the image display module, it is essential to ensure that all relevant data streams switch synchronously. Input data is routed directly from the original data source to the image display module, and processing results are seamlessly transmitted to the next processing stage. Through synchronous switching, a seamless transition of system functions is achieved, and users are unaware of the component switching process, ensuring the system continuously provides stable service.

[0133] In this embodiment, a scientifically sound task migration decision mechanism is constructed by precisely analyzing the dependencies between functional units of the medical device, effectively solving the problem of functional continuity during system failures. The solution fully utilizes the functional dependency matrix to guide task allocation, ensuring that highly related functional units are processed first, thus guaranteeing the integrity of system functions. A target component screening mechanism based on resource utilization and task execution records avoids resource overload risks and improves system stability. The application of task state snapshots and synchronous switching technology enables lossless conversion of the medical device system, eliminating the functional interruption problem in traditional methods. The overall solution significantly improves the fault recovery capability and service continuity of medical devices, reduces the impact of faults on medical diagnosis and treatment, lowers medical risks, and enhances the reliability of medical equipment.

[0134] A second aspect of the present invention provides an electronic device, comprising:

[0135] processor;

[0136] Memory used to store processor-executable instructions;

[0137] The processor is configured to invoke instructions stored in the memory to execute the aforementioned method.

[0138] A third aspect of the present invention provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the aforementioned method.

[0139] This invention can be a method, apparatus, system, and / or computer program product. The computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for performing various aspects of the invention.

[0140] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A multi-source heterogeneous data processing method in the field of medical instruments based on a multi-modal large model, characterized in that, include: Image data, audio data, and parameter data of medical devices are collected and input into a multimodal large model. The multimodal large model uses a cross-modal attention mechanism to extract modal features, constructs semantic mapping relationships between modalities through contrastive learning, and fuses them to generate multidimensional feature representations. A component operation mode library is constructed based on multidimensional feature representation. The real-time operation data of the component is matched with the component operation mode library, and the status description information is output. Component operation metrics and component status scores are extracted from status description information to construct a state evolution chain. Anomaly propagation paths are identified by analyzing the state score change trends within the state evolution chain, and fault source components and associated components are located, including: Extract component operation metrics and component state scores, calculate the temporal correlation of component operation metrics to generate a functional dependency matrix, and determine the component state propagation path based on the functional dependency matrix and component physical connection relationships; The component operation indicators and component status scores are organized into a component status sequence according to the time series. The difference values ​​of the component status sequence at adjacent time points are calculated to generate a state change matrix. A state evolution chain is constructed based on the state change matrix and the component state transmission path. Mark the abnormal state points in the state evolution chain whose state score changes exceed a preset threshold, and construct an abnormal propagation sequence in chronological order. Identify the abnormal propagation path based on the abnormal propagation sequence and the component state transmission path. The component that is the earliest to appear in the anomaly propagation path and has the largest change in state score is identified as the fault source component. Starting from the fault source component, components with abnormal state scores along the anomaly propagation path are identified as fault associated components. For fault source components and fault-related components, extract the component functional completeness and task capacity, calculate the task migration cost between components, and generate a scheduling scheme. The working mode of medical devices is reconstructed according to the scheduling scheme, and the tasks of the fault source components are allocated to the stable components according to their functional dependencies, so as to achieve lossless conversion of the system.

2. The method of claim 1, wherein, Image, audio, and parameter data of medical devices are collected and input into a multimodal large model. This model employs a cross-modal attention mechanism to extract modal features, constructs intermodal semantic mapping relationships through contrastive learning, and fuses these features to generate a multidimensional feature representation, including: Image data, audio data, and parameter data of medical devices are collected, and the image data, audio data, and parameter data are preprocessed to obtain preprocessed multimodal data. The preprocessed multimodal data is input into a multimodal large model. The multimodal large model performs convolution operations on each modality to generate corresponding feature maps, calculates the attention scores of the feature maps in the channel dimension and the spatial dimension, and generates a weight matrix. The weighted matrix and the feature map are weighted to obtain the enhanced modal features; Based on the enhanced modal features, the similarity difference between modal features is calculated, the sample with the largest feature difference is selected to construct a contrast learning sample pair, the feature distribution difference of the contrast learning sample pair is calculated, and normalization is performed to obtain the adaptive temperature coefficient. In a multimodal large model, the adaptive temperature coefficient is used to perform semantic alignment and fusion operations on the enhanced modal features to generate multidimensional feature representations.

3. The method of claim 1, wherein, The multimodal large model includes: A training sample library is constructed by collecting multimodal data of medical devices under different operating conditions. The training sample library includes normal operation data and fault data. Feature decoupling is performed on the multimodal data in the training sample library to extract component-level features and system-level features. A cross-modal attention module is trained based on the component-level features, and a contrastive learning module is trained based on the system-level features. The attention module is trained using component state labels as supervision information, and the contrastive learning module is trained using semantic consistency between modal features as a constraint. A multi-task training objective is constructed, which includes fault identification loss and feature alignment loss. An alternating optimization strategy is adopted to iteratively train the parameters of the attention module and the contrastive learning module until the model converges, resulting in a multimodal large model for medical device fault diagnosis.

4. The method of claim 1, wherein, A component runtime mode library is constructed based on multidimensional feature representation. Real-time component runtime data is matched with the component runtime mode library, and the output state description information includes: The multidimensional feature representation is segmented into time windows to obtain feature vectors. Fluctuation features are extracted from the feature vectors, and the fluctuation features are sorted according to their amplitudes to obtain sorted fluctuation features. A dynamic density threshold is calculated based on the ranking fluctuation characteristics, and the feature vector is density-clustered using the dynamic density threshold to obtain the operating mode, which includes the fluctuation characteristic distribution. The degree of dispersion of the operating mode is calculated based on the fluctuation characteristic distribution. The dynamic density threshold is adjusted based on the degree of dispersion. The optimized operating mode is obtained by clustering the feature vector using the adjusted dynamic density threshold. Calculate the confidence level of the optimized running mode, and build a component running mode library from the optimized running modes with confidence levels higher than the preset confidence threshold; The same time window is used to extract fluctuation features from the real-time running data of the component, and the fluctuation features are matched with the running modes in the component running mode library. Based on the matching results, status description information including running indicators and status scores is output.

5. The method of claim 1, wherein, For the fault source component and fault-related components, extract the component's functional completeness and task capacity, calculate the task migration cost between components, and generate a scheduling scheme including: For fault source components and fault-related components, the execution parameters and performance indicators of the components are collected, and the functional integrity of the components is extracted based on the status values ​​of the execution parameters and the change values ​​of the performance indicators. The computational load and response time of the collected components are used to extract the component task load based on the changes in the utilization rate of the computational load and the response time. The task migration cost between components is calculated based on the difference in functional integrity and the ratio of task capacity between adjacent components, and the task migration cost is constructed into a cost matrix that reflects the difficulty of component task migration. The task migration path with the minimum total cost is determined in the cost matrix. The functional integrity and task capacity of the components on the task migration path are constrained and verified. A scheduling scheme is generated based on the verification results.

6. The method of claim 1, wherein, The medical device operating mode is reconstructed according to the scheduling scheme, and the tasks of the faulty component are allocated to the stable component according to the functional dependency, achieving a lossless system conversion, including: Extract the functional unit sequence of the fault source component according to the scheduling scheme, count the data flow frequency and resource sharing duration between each function in the functional unit sequence, and generate a functional dependency matrix that records the dependency relationship between functional units. Obtain the resource utilization rate and task execution records of each component in the system, filter components that meet the preset resource threshold as target components based on the functional dependency matrix, and determine the task allocation order based on the remaining resource capacity of the target components. The tasks of the fault source component are sorted according to the dependency relationship values ​​in the functional dependency matrix, and the sorted tasks are assigned to the target component in turn. Generate a snapshot of the task status and perform migration according to the task allocation results to complete the synchronous switching of data status and functional status, and achieve lossless system conversion.

7. An electronic device, comprising: include: processor; Memory used to store processor-executable instructions; The processor is configured to invoke instructions stored in the memory to execute the method according to any one of claims 1 to 6.

8. A computer-readable storage medium having stored thereon computer program instructions, wherein, When the computer program instructions are executed by the processor, they implement the method described in any one of claims 1 to 6.

Images