Space-time multi-dimensional alignment medical big data storage and calculation scheduling method, system, device and medium

Abstract

This invention discloses a spatiotemporally multidimensional aligned medical big data storage and computing scheduling method, system, device, and medium. The method processes raw medical data and maps it to a spatiotemporal diagnostic coordinate system, constructing a three-level joint index. Medical data processing tasks are divided into GPU and CPU execution tasks, and medical event records are sharded and co-located with corresponding computing tasks on storage nodes according to their data residence locations. Medical event records undergo tensor reorganization, establishing a streaming tensor processing pipeline, and performing partial aggregation calculations on the storage side. Load characteristic data is collected at each storage node, and a load prediction model is constructed based on spatiotemporal correlation features and time series features. Horizontal and / or vertical scaling of computing resources is performed according to preset expansion and contraction thresholds. This solution can reduce data migration, improve resource utilization efficiency, and enhance the real-time processing capability of medical big data.

Description

Technical Field

[0001] This invention relates to the field of medical big data processing and resource scheduling technology, specifically to a spatiotemporally multidimensional aligned medical big data storage, computing, and scheduling method, system, device, and medium, which is particularly suitable for scenarios involving efficient storage, rapid retrieval, real-time computing, and dynamic scheduling of computing resources for multi-source heterogeneous medical big data. Background Technology

[0002] With the rapid development of medical informatization and digitalization, medical institutions at all levels are continuously generating massive amounts of multi-source, heterogeneous medical data, covering clinical diagnosis and treatment, medical imaging, physiological sign monitoring, health management, and other types. This type of data also has obvious temporal, spatial, and diagnostic and treatment attribute dimensions, and is an important foundation for intelligent diagnosis and treatment, medical research, and public health management.

[0003] Existing medical big data processing solutions generally suffer from problems such as storage-computation separation and insufficient utilization of spatiotemporal features. First, existing solutions mostly employ a storage-computation separation architecture. Operations such as index building, data preprocessing, feature extraction, and result aggregation typically require transferring massive amounts of data from storage nodes to computing nodes, resulting in significant cross-node data migration overhead. Second, existing index structures are mostly built based on a single dimension or a few dimensions, lacking a unified spatiotemporal diagnostic coordinate system, making it difficult to achieve accurate multi-dimensional alignment and joint retrieval across time, space, and diagnostic attribute dimensions.

[0004] At the heterogeneous computing level, existing GPU / CPU collaborative processing solutions typically do not fully integrate data residing location for task scheduling. The lack of an effective co-location mechanism between data sharding and computing tasks leads to a separation between computing power scheduling and data distribution, resulting in frequent cross-node data transfers. At the streaming processing level, existing processing engines are mostly biased towards batch processing or general streaming processing modes, and have not yet been deeply integrated with storage and computing nodes. The local computing capabilities of the storage side are not fully utilized, making it difficult to meet the low-latency processing requirements of real-time medical services. At the resource scheduling level, existing solutions mostly adopt passive triggering scheduling methods based on fixed thresholds, lacking joint modeling of the spatiotemporal correlations between nodes and time-series load trends. This results in insufficient matching between computing power scaling and data distribution, business peaks, and node load fluctuations, leading to low resource utilization efficiency.

[0005] Therefore, existing technologies have not yet formed a complete solution for medical big data scenarios, which includes "unified spatiotemporal diagnosis and treatment coordinate system + three-level joint index on the near data side + heterogeneous collaborative computing with data task co-location + streaming multi-granularity medical tensor reconstruction and partial aggregation on the storage side + dual-threshold elastic scheduling based on spatiotemporal correlation prediction". It is difficult to adapt to the characteristics of medical big data, such as multidimensional heterogeneity, continuous generation, real-time processing and dynamic load fluctuation. Summary of the Invention

[0006] This invention proposes a spatiotemporally multidimensional aligned medical big data storage and computing scheduling method, as well as corresponding systems, equipment, and computer-readable storage media. This solution revolves around a unified spatiotemporal diagnosis and treatment coordinate system, a three-level joint index on the near data side, GPU / CPU heterogeneous collaborative computing with data tasks co-located, streaming multi-granularity medical tensor reconstruction and partial aggregation on the storage side, and dual-threshold elastic scheduling based on spatiotemporal correlation prediction, to achieve near data processing, dynamic resource allocation, and low-latency real-time computing of medical big data.

[0007] Specifically, the present invention provides the following technical solutions: On the one hand, this invention provides a spatiotemporally multidimensional aligned medical big data storage and computing scheduling method, including: Step 1: Construct a unified spatiotemporal diagnostic and treatment coordinate system and a three-level joint indexing mechanism on the near data side: Standardize and clean the original medical data, and generate medical event records according to time information, spatial location information, and diagnostic and treatment attribute information; map the medical event records to the unified spatiotemporal diagnostic and treatment coordinate system to generate medical event records with spatiotemporal diagnostic and treatment coordinate labels; on the near data side, construct a global main index, local sub-indexes, and feature inverted index based on the medical event records with spatiotemporal diagnostic and treatment coordinate labels, and establish the association between index items and the original medical data storage address to form a multi-dimensional joint retrieval mechanism oriented towards the time dimension, spatial dimension, and diagnostic and treatment attribute dimension; Step 2: Construct a GPU / CPU heterogeneous collaborative computing mechanism for data task co-location: Based on the parallelism, computational complexity, and real-time requirements of the medical data processing tasks, classify the tasks into GPU execution tasks and CPU execution tasks; for the medical event records with spatiotemporal diagnostic coordinate labels formed in Step 1, perform data sharding according to preset time windows, spatial ranges, or diagnostic attribute dimensions to form data shards to be processed corresponding to the medical data processing tasks; co-deploy the data shards to be processed and the corresponding computing tasks on the storage and computing nodes according to the data residence location, and schedule the GPU execution tasks to GPU nodes and the CPU execution tasks to CPU nodes according to the task classification results; link the GPU / CPU scheduling with the data residence strategy to adjust the allocation of computing resources and the task execution order between GPU nodes and CPU nodes; Step 3: Construct a streaming multi-granularity medical tensor reconstruction and storage-side partial aggregation mechanism: Perform tensor-based reconstruction on medical event records with spatiotemporal diagnostic coordinate labels after spatiotemporal multidimensional alignment, constructing a one-dimensional temporal tensor, a two-dimensional or three-dimensional image tensor, and a multi-source fusion high-dimensional tensor. Generate a set of fusion candidate tensors based on patient identification, time dimension, spatial dimension, and diagnostic attribute dimension in the unified spatiotemporal diagnostic coordinates. Establish a streaming tensor processing pipeline on the storage node side, and perform spatiotemporal tensor alignment, feature extraction, multi-source fusion, and incremental processing in the streaming tensor processing pipeline. Perform partial aggregation calculations on the storage side and output window-level statistical feature results, region-level feature results, or attribute-level aggregation results to reduce cross-node data migration and subsequent centralized computing pressure. Step 4: Construct a dual-threshold elastic scheduling mechanism based on a spatiotemporal correlation prediction model: Collect load characteristic data consisting of resource utilization, task status, and data throughput at each storage node; construct a load prediction model based on spatiotemporal correlation characteristics and time series characteristics, and generate a scheduling strategy based on the load prediction results and preset expansion and contraction thresholds; perform horizontal expansion and / or vertical expansion and contraction of computing resources according to the scheduling strategy, and complete task migration and resource mapping updates before resource contraction to adjust the resource configuration relationship between storage nodes.

[0008] Preferably, the original medical data includes clinical diagnosis and treatment data, medical imaging data, physiological signs data, and health management data; the time information includes the collection time, treatment stage, and follow-up period; the spatial information includes the lesion location, image coordinates, and medical institution location; and the diagnosis and treatment attribute information includes the diagnosis results, treatment plan, examination items, and physiological indicators.

[0009] Preferably, a unified task description format is used for data interaction, and task collaboration and data transmission between the GPU and CPU are achieved through storage sharing or network transmission.

[0010] Secondly, the present invention provides a spatiotemporally multidimensional aligned medical big data storage and scheduling system, including: a data acquisition module for acquiring raw medical data and providing standardized input; The event processing and indexing module is used to clean, process, and align the raw medical data in a spatiotemporal manner, generating medical event records with spatiotemporal diagnosis and treatment coordinate labels, and constructing a three-level composite index consisting of a global primary index, local sub-indexes, and feature inverted indexes on the near data side; The heterogeneous computing scheduling module is used to classify medical data processing tasks, perform data sharding on medical event records with spatiotemporal diagnosis and treatment coordinate labels, and co-deploy the data shards and corresponding computing tasks on the storage and computing nodes according to the data residence location, so as to realize the linkage between GPU / CPU collaborative scheduling and data residence strategy. The streaming tensor processing module is used to perform tensor recombination on medical event records, generate a set of fusion candidate tensors, establish a streaming tensor processing pipeline, and perform some aggregation calculations on the storage side. The elastic scheduling module is used to collect load characteristic data, perform load prediction, generate scheduling strategies based on preset expansion and contraction thresholds, and perform horizontal and / or vertical expansion and contraction of computing resources. The basic storage module is used for distributed storage of raw medical data, index data, and calculation results, and provides underlying data read / write and address mapping services.

[0011] Preferably, the event processing and indexing module, the heterogeneous computing scheduling module, the streaming tensor processing module, and the elastic scheduling module are all deployed on the storage node side, and data reading and writing and address mapping are coordinated through the basic storage module.

[0012] Thirdly, the present invention provides a spatiotemporally multidimensional aligned medical big data storage and computing scheduling device, including a processor and a memory, wherein the memory stores computer instructions, and the processor executes the computer instructions to implement the spatiotemporally multidimensional aligned medical big data storage and computing scheduling method as described above.

[0013] Fourthly, the present invention provides a computer-readable storage medium storing program code, which, when executed by a processor, implements the spatiotemporal multidimensional aligned medical big data storage and computing scheduling method as described above.

[0014] The in-store computing features of this invention are manifested in the following ways: index construction and partial computation are performed near the data side; data sharding and computing tasks are co-located and deployed; the storage side supports partial aggregation computation; retrieval processing and data residency strategies are coordinated and linked; and computing resource scheduling is coordinated with the data distribution status.

[0015] Compared with the prior art, the present invention has at least the following advantages: 1. By constructing a unified spatiotemporal diagnosis and treatment coordinate system and transforming raw medical data events into medical event records with spatiotemporal diagnosis and treatment coordinate labels, medical data from different sources, modalities, and time granularities can be uniformly organized and multidimensionally aligned within the same coordinate framework. This fundamentally solves the problems in existing technologies where multi-source heterogeneous medical data is difficult to model uniformly and difficult to collaboratively retrieve across time, space, and diagnosis and treatment attribute dimensions.

[0016] 2. By constructing a three-level composite index structure consisting of a global primary index, local sub-indexes, and feature inverted indexes on the near data side, and establishing the association between index items and the original medical data storage address, the retrieval process can be completed preferentially near the data residence location, reducing the cross-node data migration overhead caused by retrieval and preprocessing, and improving the positioning capability and index update efficiency of multi-dimensional composite retrieval.

[0017] 3. By constructing a GPU / CPU heterogeneous collaborative computing mechanism for data task co-location, medical event records with spatiotemporal diagnostic coordinate labels are divided into data shards to be processed according to preset rules. The tasks are heterogeneously classified according to the parallelism, computational complexity, and real-time level of the tasks, so that the data shards and corresponding computing tasks can be co-located and deployed on the storage and computing nodes according to the data residence location. This fundamentally improves the problem of separation between data location and computing power scheduling in the existing technology, and reduces the data duplication and resource idleness caused by remote task scheduling.

[0018] 4. By constructing a streaming multi-granularity medical tensor reconstruction and storage-side partial aggregation mechanism, medical event records are reconstructed into time-series one-dimensional tensors, image two-dimensional or three-dimensional tensors, and multi-source fusion high-dimensional tensors. Partial aggregation calculations and incremental result outputs are performed on the storage side, enabling some statistical calculations, local feature extraction, and intermediate result generation to be completed close to the data. This reduces the processing pressure on centralized computing nodes from the bottom layer and improves the real-time performance and continuity of streaming medical data processing.

[0019] 5. By constructing a dual-threshold elastic scheduling mechanism based on a spatiotemporal correlation prediction model, the load characteristics of each computing node, such as resource utilization, task status, and data throughput, are continuously collected. The scheduling strategy is generated by combining expansion and contraction thresholds, so that the expansion and contraction of computing resources can match the node load change trend and data distribution status. This improves the foresight and stability of resource scheduling from the bottom layer and reduces the resource oscillation and scheduling lag caused by simply relying on passive scheduling with fixed thresholds.

[0020] This solution, through an integrated unified processing chain, enables medical big data to form a collaborative closed loop across storage, retrieval, computation, and scheduling, thereby fundamentally improving the system's overall data locality utilization, heterogeneous computing power utilization, and dynamic resource allocation capabilities. Attached Figure Description

[0021] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is an overall flowchart of the method provided in the embodiments of the present invention; Figure 2 Flowchart for constructing a unified spatiotemporal diagnostic coordinate system and a near-data-side three-level joint indexing mechanism; Figure 3 Flowchart of a GPU / CPU heterogeneous collaborative computing mechanism for co-located data tasks; Figure 4 Flowchart of the aggregation mechanism of flow cytometry multi-granularity medical tensor recombination and storage side; Figure 5 Flowchart for constructing a dual-threshold elastic scheduling mechanism based on a spatiotemporal correlation prediction model. Detailed Implementation

[0023] The present invention will be further described below with reference to specific embodiments and accompanying drawings. It should be noted that the described embodiments are only a part of the embodiments of the present invention, and not all of them. Other embodiments obtained by those skilled in the art based on the embodiments described in the present invention without creative effort are all within the protection scope of the present invention.

[0024] This invention combines spatiotemporal multidimensional alignment with in-memory computing integration to achieve end-to-end processing including data eventification, near-data-side index construction, co-location of data and computation, streaming tensor processing, and predictive elastic scheduling. Each step is preferably executed near the in-memory computing node, as detailed below.

[0025] S1: Constructing a unified spatiotemporal diagnostic coordinate system and a three-level joint indexing mechanism for near-data side. like Figure 2 As shown, this step is used to realize the event-based processing of raw medical data, spatiotemporal multidimensional alignment, and near-data-side index construction.

[0026] S1.1: Raw Medical Data Acquisition and Standardized Cleaning: Multi-source medical data is collected through a standardized data access interface. This multi-source medical data includes at least clinical diagnosis and treatment data, medical imaging data, physiological sign data, and health management data. After collection, standardized cleaning processing is performed on the storage node side, specifically including: 1. Standardization of clinical diagnosis and treatment data: Field mapping is performed on structured medical records, laboratory test records, diagnostic results, and treatment plans, and they are uniformly converted into a preset data model. Diagnostic terms are preferably mapped using the ICD-10 coding system, examination and test items are preferably mapped using the LOINC coding system, and drug names are preferably mapped using the ATC coding system or the hospital's drug dictionary. Synonyms, abbreviations, and terms with different writing styles are standardized, for example, mapping "CT", "Computed Tomography", and "Computed Tomography Scan" to the same standard terminology identifier.

[0027] 2. Standardization of medical imaging data: Image data should ideally be converted to the DICOM standard format; for JPEG, PNG, or proprietary in-house formats, they can be converted to standard DICOM objects using a format conversion module; image pixel values ​​should be numerically normalized, preferably using the minimum-maximum normalization method.

[0028] Where I is the original pixel value. ′ represents the normalized pixel value, I min and I max These are the minimum and maximum values ​​within the current image or a preset window, respectively. For CT images, grayscale truncation and linear mapping can be further performed based on the preset window width and window level to enhance the comparability of different lesion areas.

[0029] 3. Standardization of time-series physiological data: Time-series data such as ECG, blood pressure, blood oxygen, pulse, and body temperature are resampled according to a unified sampling time axis; for missing sampling points, linear interpolation, forward filling, or sliding window mean-based methods are preferred to fill in the missing values; timestamps of data from different devices are uniformly converted to the same time zone and the same time base, and time alignment is completed according to a preset time window.

[0030] 4. Text data structuring: Sentence segmentation, word segmentation, entity recognition, and field extraction are performed on medical record texts, image reports, and follow-up records. The preferred method is to identify diagnostic entities, site entities, examination item entities, and treatment entities by combining medical dictionaries and rule templates. The extraction results are mapped into structured fields and written into subsequent medical event records.

[0031] 5. Abnormal data handling: Duplicate records are deduplicated based on patient identifier, visit identifier, record time, and data summary; data records missing key fields are marked as incomplete records, and are selected to be removed, supplemented, or retained for review according to preset rules; numeric fields that exceed the reasonable range are marked as abnormal and recorded in the quality control log.

[0032] S1.2 Medical Eventification and Spatiotemporal Multidimensional Alignment: Medical event records are constructed based on standardized medical data. These records use a single examination, single treatment, single image acquisition, single monitoring sampling, or single follow-up visit as the smallest event granularity. Each medical event record includes at least patient identifier, visit identifier, event time, event spatial location, and treatment attribute fields. In some implementations, the i-th medical event record can be mapped to unified spatiotemporal treatment coordinates, represented as:

[0033] Among them, C i p represents the unified spatiotemporal coordinates of the i-th medical event record; i Indicates patient identification; v i Indicates a medical visit identifier; t i Indicates time dimension information; s i Represents spatial dimensional information; a i This indicates information about the diagnostic and treatment attributes.

[0034] After mapping is completed, spatiotemporal diagnosis and treatment coordinate labels are attached to each medical event record and written to the event index cache for subsequent use in the construction of the near-data side index.

[0035] S1.3: Construction of a three-level composite index: On the near data side, a three-level index structure is constructed based on medical event records with spatiotemporal diagnostic coordinate labels, specifically including: S1.3.1. Global Primary Index Construction: The primary index key is constructed based on the patient identifier and event time. The preferred representation of the primary index key is as follows:

[0036] in, Indicates the global primary index key. Indicates patient identification, t i Represents time information, Hash( ) represents the hash mapping function. The global master index is used to locate the patient's event distribution at different times.

[0037] S1.3.2 Local Sub-index Construction: Local data blocks are divided according to data type and spatial range, and local sub-indexes are created within each local data block. Local sub-indexes are used to narrow the search scope, such as limiting it to a specific department, equipment, lesion area, or image block.

[0038] S1.3.3. Construction of Feature Inverted Index: An inverted index table is constructed based on diagnostic results, examination items, treatment plans, physiological indicator categories, and keywords; each inverted item records at least the keyword identifier, event record identifier, and data storage address.

[0039] S1.3.4. Index Update Settings: Incremental index writing is performed on newly added medical event records, old index invalidation is marked and new index is rebuilt on modified event records, and index clearing or tombstone marking is performed on deleted event records.

[0040] During retrieval, it is preferable to first locate the patient and time range through the global master index, then limit the spatial range through the local sub-index, and finally filter the diagnosis and treatment attributes through the feature inverted index to form a multi-dimensional joint retrieval path.

[0041] S2: Constructing a GPU / CPU heterogeneous collaborative computing mechanism for data task co-location like Figure 3 As shown, this step is used to achieve task classification, co-location of data and computing, and linkage between computing power scheduling and data residency strategies.

[0042] S2.1: Computational Task Classification: Based on the parallelism, computational complexity, and real-time requirements of the medical data processing tasks, the tasks are classified into GPU-executed tasks and CPU-executed tasks, and corresponding business priorities can be further set. In some implementations, medical data processing tasks can be classified using a task scoring function, which is expressed as:

[0043] Among them, Score j P represents the scheduling score of the j-th task. j C represents the task parallelism index. j R represents the computational complexity of a task. j Indicates the real-time performance level of the task. , , These are the weighting coefficients.

[0044] The indicators can be determined as follows: 1. Parallelism index P j The parallelism metric is used to characterize the extent to which a task can be broken down into multiple concurrent execution units.

[0045] In some implementations, the parallelism index can be determined based on the number of data fragments corresponding to the task, the number of data blocks that can be processed in parallel, or the number of computing units that can be executed simultaneously. Preferably, it can be calculated using the following formula:

[0046] Where, N j This represents the number of parallelizable execution units corresponding to the j-th task, where the number of parallelizable execution units includes one or more of the following: the number of data slices, the number of tensor blocks, the number of image blocks, or the number of batch processing subtasks; N max This represents the preset maximum number of parallel units. After normalization, P j ∈[0,1].

[0047] 2. Computational complexity index C j The computational complexity index is used to characterize the degree of computational resource consumption required for a task.

[0048] In some implementations, the computational complexity index can be determined based on one or more of the following: the amount of data processed, feature dimensions, tensor size, image resolution, number of model inference layers, or estimated number of computations. Preferably, a weighted method can be used for calculation.

[0049] Among them, D j F represents the amount of data corresponding to the j-th task. j Represents the feature dimension or tensor size of the j-th task, O j D represents the estimated computational cost of the j-th task. max F max O max These represent the preset maximum values ​​of the corresponding indicators, where λ1, λ2, and λ3 are the weight coefficients of the complexity components, and λ1 + λ2 + λ3 = 1. After normalization, C j ∈[0,1].

[0050] 3. Real-time performance index R j The real-time performance index is used to characterize the sensitivity of a task to processing latency.

[0051] In some implementations, the mapping can be determined based on the task's business type, preset response time limit, or business priority. Preferably, the real-time performance level can be divided into three levels: high real-time, medium real-time, and low real-time, and assigned values ​​respectively:

[0052] Among them, high real-time tasks include emergency image processing and real-time monitoring and early warning; medium real-time tasks include routine auxiliary analysis tasks; and low real-time tasks include offline statistical analysis and batch index updates.

[0053] In some implementations... When Score j When ≥ θ, the task is divided into GPU execution tasks; When Score j When θ < θ, the task is divided into CPU execution tasks. Where θ is the preset classification threshold.

[0054] The division of the above-mentioned high-parallelism image registration, tensor operation, model inference tasks, as well as index maintenance, rule matching, logic control, and lightweight preprocessing tasks, preferably follows the calculation results of the task scoring function and the threshold comparison rules.

[0055] Among them, high-parallelism image registration, tensor operation, and model inference tasks are preferably classified as GPU execution tasks when the scoring results reach the preset classification threshold, since they usually have high parallelism, computational complexity and / or real-time performance indicators. Index maintenance, rule matching, logic control, and lightweight preprocessing tasks are typically classified as CPU execution tasks when the score is below the preset classification threshold, as they usually have low or medium parallelism, computational complexity, and real-time performance metrics.

[0056] S2.2: Task Collaboration and Data Interaction: A unified task description format is used to describe the tasks to be executed. The task description format includes at least: task identifier, task type, input data address, output address, priority, resource requirements, estimated execution time, and dependencies.

[0057] Data interaction between the GPU and CPU does not directly copy the entire original medical data, but instead preferably transmits data addresses, data block identifiers, index reference information, and intermediate result addresses.

[0058] In some implementation methods: For data interaction within the same storage node, shared memory or unified memory address mapping can be used; For cross-node data interaction, remote direct memory access or high-speed network transmission can be used, transmitting only the necessary data blocks and result summaries.

[0059] The above methods reduce the need to move all raw data across nodes.

[0060] S2.3: Co-location Deployment and Dynamic Scheduling: The medical event records with spatiotemporal diagnostic coordinate labels formed in step 1 are data fragmented, and the data fragments and corresponding computing tasks are co-located and deployed on the same storage node. The data fragments are preferably divided according to time windows, spatial ranges, diagnostic attribute dimensions, or data types to form data fragments to be processed that match the granularity of task processing.

[0061] In some implementations, data sharding can be performed as follows: S2.3.1 Slicing by time window: Medical event records with spatiotemporal diagnosis and treatment coordinate labels are divided into multiple time segments according to a preset time window, so that event records within the same time window are preferentially included in the same data segment. S2.3.2 Fragmentation by spatial range: Medical event records with spatiotemporal diagnostic coordinate labels are divided according to the location of the medical institution, department, equipment, imaging area, or lesion space, so that data within the same or adjacent spatial range are preferentially assigned to the same data segment. S2.3.3 Segmentation by diagnostic and treatment attributes: Medical event records with the same or similar diagnostic categories, examination item categories, treatment plan categories, or physiological indicator categories are grouped into the same data shard to facilitate subsequent attribute-related local calculations and aggregation processing. S2.3.4 Fragmentation by data type: We establish corresponding data fragments for time-series physiological data, medical imaging data, structured diagnostic and treatment data, and text structured results to adapt to the processing methods of different types of computing tasks.

[0062] To ensure efficient subsequent co-location deployment and heterogeneous scheduling, the data sharding preferably meets the following requirements: (1) Data within the same data segment have a high correlation in terms of time dimension, spatial dimension or diagnosis and treatment attribute dimension; (2) The data size of a single data shard shall not exceed the preset shard capacity threshold to avoid excessive load on a single node; (3) Data in the same data shard should preferably correspond to the same or adjacent types of computing tasks to reduce cross-node task splitting and duplicate data transmission; (4) For high-priority business data that needs to be processed in real time, smaller-granularity data fragments can be formed first to improve scheduling response speed.

[0063] After data sharding is completed, the data shards and corresponding computing tasks are co-located and deployed on the same storage node according to the data residence location. Based on the task classification results, GPU execution tasks are scheduled to GPU nodes for execution, and CPU execution tasks are scheduled to CPU nodes for execution.

[0064] When node resource usage reaches the preset conditions, low-priority tasks can be migrated, and resources can be reserved for high-priority services.

[0065] S2.3.5 Preset conditions include (the following preset conditions are OR relationships, meaning that meeting any one of them is considered a trigger): (1) CPU utilization consistently exceeds the preset CPU threshold; (2) GPU utilization consistently exceeds the preset GPU threshold; (3) The memory usage rate or video memory usage rate is consistently higher than the preset capacity threshold. (4) The task queue length exceeds the preset queue threshold; (5) The average waiting time for the task exceeds the preset latency threshold; (6) The node's input / output throughput or network transmission load is consistently higher than the preset bandwidth threshold.

[0066] In some implementations, when the preset conditions are triggered, it is preferable to first identify the low-priority tasks that can be migrated and their corresponding data shards, and then migrate the low-priority tasks to the storage and computing nodes with lower loads, while reserving or reserving the GPU resources, CPU resources, memory resources or video memory resources on the current node for high-priority business tasks to call.

[0067] S3: Flow Cytometry Multigranular Medical Tensor Recombination and Storage-Side Partial Aggregation like Figure 4 As shown, this step is used to implement tensor quantization of medical data, execution of the streaming pipeline, and partial aggregation calculation on the storage side.

[0068] S3.1: Tensorization Processing of Medical Data: Tensorization reconstruction is performed on medical event records with spatiotemporal diagnostic and treatment coordinate labels. Specifically, one-dimensional sequence tensors are constructed for time-series physiological data according to time windows; two-dimensional or three-dimensional spatial tensors are constructed for image data according to pixel matrices or voxel volumes; and high-dimensional fusion tensors are constructed for multi-source fusion data according to a multi-axis structure of "patient-time-space-attribute".

[0069] In some implementations, the tensor quantization process can be represented as:

[0070] in, Indicates the first The original medical data set within each data window

[0071] This represents the corresponding set of spatiotemporal diagnostic coordinates. This represents the quantified medical data object, f( ) represents the tensor construction mapping function.

[0072] Preferably, the tensor construction mapping function can be expressed as:

[0073] Among them, Select(X) k C k ) represents the set of spatiotemporal diagnostic coordinates C k For the original medical data set X k The selection process is performed to obtain a subset of candidate data that falls within the range of the k-th time window, spatial range, and diagnostic / treatment attributes. Align means performing time alignment, spatial alignment, and attribute alignment on each data object in the candidate data subset according to a unified spatiotemporal diagnostic coordinate. Stack represents stacking, concatenating, or rearranging aligned data objects according to a preset dimensional order, generating a corresponding tensor object T. k .

[0074] After the tensor is constructed, the tensor-quantized medical data object is written to the streaming tensor processing cache, and the index correspondence between different dimensions is maintained to support subsequent fusion processing.

[0075] S3.2: Streaming tensor processing pipeline: A streaming tensor processing pipeline is established on the storage node side. The streaming tensor processing pipeline includes a data access processing unit, a tensor operation processing unit, and a result output processing unit.

[0076] S3.2.1 Data Access Processing Unit: The data access processing unit is used to receive streaming medical data from different data sources and write the data into the processing buffer according to patient identification, time window, spatial range and diagnosis and treatment attributes.

[0077] The tensor operation processing unit is used to perform spatiotemporal tensor alignment, feature extraction, and multi-source fusion processing on tensor objects in the processing buffer. Specifically, for one-dimensional temporal tensors, resampling, time alignment, and window segmentation are performed according to a sliding time window to obtain windowed temporal tensors; for two-dimensional or three-dimensional image tensors, slicing and local block extraction are performed according to preset spatial blocks or regions of interest to obtain spatial local tensor blocks; for multi-source heterogeneous tensors, a set of fusion candidate tensors is generated based on patient identification, time dimension, spatial dimension, and diagnosis and treatment attribute dimension in a unified spatiotemporal diagnostic and treatment coordinate system.

[0078] In some implementations, the process of generating the fusion candidate tensor set includes: S3.2.1.1 Sample Individual Merging: Tensor objects from different data sources are initially merged according to their individual identifiers to filter out candidate tensor objects belonging to the same individual; S3.2.1.2 Time Window Matching: For candidate tensor objects that have completed individual merging, time dimension matching is performed according to a preset time window, and tensor objects with time differences within a preset threshold range are grouped into the same candidate set. S3.2.1.3 Spatial Range Matching: For candidate tensor objects that have completed time window matching, spatial matching is performed based on the location of medical institutions, equipment, imaging regions, lesion regions, or preset spatial proximity relationships. Tensor objects with consistent spatial locations, adjacent locations, or that meet preset spatial association conditions are grouped into the same candidate set. S3.2.1.4 Relationship of Diagnosis and Treatment Attributes: For candidate tensor objects that have completed spatial range matching, perform diagnostic and treatment attribute association based on diagnosis category, examination item category, treatment plan category, physiological indicator category, or text structured entity category, and group tensor objects with consistent, similar, or preset mapping relationships into the same candidate set. S3.2.1.5 Candidate Set Generation: Tensor objects that simultaneously meet the conditions of individual merging, time window matching, spatial range matching, and diagnosis and treatment attribute association are combined into a set of fusion candidate tensors. Each set of fusion candidate tensors is assigned a set identifier, source index, and version number for subsequent multi-source fusion processing.

[0079] The fusion candidate tensor set is used to characterize a set of multi-source tensor objects of the same individual under the same or adjacent time window, the same or related spatial range, and the same or related diagnostic and treatment attributes, and can be used as input for subsequent feature splicing, weighted convergence, or cross-modal fusion calculation.

[0080] The candidate set expression is as follows:

[0081] in: F k Denotes the set of the k-th fusion candidate tensors; T i Represents a candidate tensor object; p i =p k This indicates that the patient identification is consistent; |t i t k |≤δ t This indicates that the time difference falls within the preset time window threshold; d(s i ,s k )≤δ s This indicates that the spatial distance or spatial difference falls within a preset spatial threshold; a i a k This indicates that the diagnostic and treatment attributes are consistent, similar, or satisfy a preset attribute mapping relationship.

[0082] S3.2.2 Feature Extraction Process: In some implementations, basic feature extraction can employ conventional statistical analysis methods, image processing methods, and structured coding methods. The improvement of this invention lies in the fact that the feature extraction process is a hierarchical feature extraction process performed under the constraints of unified spatiotemporal diagnostic coordinates and a fused candidate tensor set, to ensure the correspondence and fusionability of extraction results from different data sources in the time dimension, spatial dimension, and diagnostic attribute dimension. Specifically: S3.2.2.1 Temporal Tensor Feature Extraction For a windowed time series tensor, time series features are extracted within a preset time window. In some implementations, a sliding window statistical method can be used to calculate the mean, maximum, minimum, fluctuation amplitude, and rate of change. The feature extraction rules include: the mean feature is used to characterize the average state within the current window; the extreme value feature is used to characterize the peak and valley values ​​within the current window; the fluctuation amplitude feature is used to characterize the dispersion within the current window; the rate of change feature is used to characterize the changing trend between adjacent sampling points or adjacent sub-windows; and the abnormal fluctuation marker is used to characterize time segments that exceed a preset normal range or whose rate of change exceeds a preset threshold.

[0083] In some implementations, the window mean and rate of change can be expressed as follows:

[0084]

[0085] Where, μ k V represents the mean characteristic within the k-th time window.k x represents the rate of change characteristic within the k-th time window. i This represents the i-th sampled value within the window, and n represents the number of sampled points within the window.

[0086] S3.2.2.2 Image Tensor Feature Extraction Extract image features from two-dimensional or three-dimensional image tensors within preset spatial blocks, regions of interest, or lesion regions.

[0087] In some implementations, gray-level statistical features, texture features, edge contour features, and local region morphological features can be extracted using gray-level statistics, local texture analysis, edge detection, and region morphological analysis methods.

[0088] Among them: grayscale statistical features are used to characterize the distribution of pixel values ​​or voxel values ​​in the current region; texture features are used to characterize the grayscale change pattern within the region; edge contour features are used to characterize the boundary of the lesion or the contour of the tissue structure; local region morphological features are used to characterize the area, volume, aspect ratio or compactness of the lesion region or region of interest.

[0089] In some implementations, the average gray level of a local area can be expressed as:

[0090] Among them, g k y represents the average grayscale feature of the k-th spatial region. j This represents the j-th pixel value or voxel value within the region, and m represents the total number of pixels or voxels within the region.

[0091] S3.2.2.3 Feature Extraction from Structured Diagnostic Data and Textual Structured Results For structured diagnostic and treatment data and textual structured results, under the constraint of unified spatiotemporal diagnostic and treatment coordinates, we extract diagnostic category codes, examination item codes, treatment plan codes, physiological indicator vectors, and entity co-occurrence relationship features.

[0092] In some implementation methods: The diagnostic results, examination items, and treatment plans are encoded and mapped to form discrete category features; the physiological indicators are numerically vectorized to form continuous indicator features; and entity co-occurrence matrices or co-occurrence relationship vectors are established for entity objects in the text structured results to represent the association between different diagnostic and treatment entities.

[0093] S3.2.2.4 Unified Feature Output The aforementioned temporal tensor features, image tensor features, and structured diagnostic features are written into the feature cache or fusion processing unit according to a unified feature description format.

[0094] In some implementations, the unified feature description format includes at least patient identifier, time window identifier, spatial range identifier, diagnosis and treatment attribute identifier, feature type identifier, and feature value vector, for use in subsequent multi-source fusion processing.

[0095] In this invention, the above feature extraction results are not output in isolation, but are organized into a structured feature set that can be directly used for multi-source fusion processing, constrained by unified spatiotemporal diagnostic coordinates and a fusion candidate tensor set.

[0096] S3.2.3 In the multi-source fusion process, the preferred order for fusion processing is "patient identifier merging—time window alignment—spatial range filtering—diagnosis and treatment attribute splicing or weighted aggregation," generating a fusion feature tensor or a structured intermediate feature vector. The fusion formula is shown below:

[0097] in, Indicates the first Fusion feature results within a time window Represents a time-series tensor. Represents the image tensor. Represents the diagnostic attribute tensor. This represents the fusion mapping function.

[0098] Preferably, the fusion mapping function can be expressed as:

[0099] Where Norm represents the dimensionality normalization or scale unification processing performed on tensors of different modalities, ⊕ represents feature concatenation or combination according to the corresponding dimension, and ω1, ω2, and ω3 represent the fusion weight coefficients corresponding to the temporal tensor, image tensor, and diagnostic attribute tensor, respectively, and satisfy the following:

[0100] In some implementation methods: (1) For time series tensors First, extract window-level time series features, then perform normalization processing; (2) Image tensor First, extract the region-level grayscale, texture, or morphological features, and then perform normalization processing. (3) Treatment attribute tensor First, extract the diagnostic category code, examination item code, treatment plan code, physiological indicator vector, or entity co-occurrence relationship features, and then perform normalization processing. (4) Then, based on the preset fusion weights ω1, ω2, and ω3, the features of different modalities are spliced ​​or weighted and converged to obtain the fusion feature result F. k .

[0101] In some implementations, when a structured intermediate feature vector needs to be output, the fusion result F k It can be further mapped to a unified feature description format for subsequent local aggregation calculations, real-time monitoring and analysis, or model inference calls.

[0102] S3.3: Partial Aggregation Computation on the Storage Side: Partial aggregation computation is performed on the medical data after tensor quantization and preliminary fusion processing on the storage side to reduce the pressure of subsequent cross-node centralized computation.

[0103] The aforementioned partial aggregation calculation uses combinations of time windows, spatial ranges, and diagnostic attributes as basic aggregation units, and performs corresponding local aggregation processing for different types of tensors, specifically including: S3.3.1 Local statistical aggregation of temporal tensors: For the same patient or the same monitoring subject within a preset time window, calculate the window mean, maximum value, minimum value, fluctuation amplitude, rate of change, and abnormal fluctuation markers of the time series one-dimensional tensor, and generate a window-level statistical feature vector; S3.3.2 Local Region Aggregation of Image Tensors: Local aggregation is performed on two-dimensional or three-dimensional image tensors according to preset spatial blocks, lesion regions or regions of interest, and gray-level statistical features, texture features, edge contour features and local morphological feature summaries of each region are extracted to generate region-level feature results.

[0104] In some implementations, the local aggregation includes the following steps: S3.3.2.1 Regional Division: Based on preset spatial partitioning rules, lesion annotation regions, regions of interest, or sliding window rules, the image tensor is divided into multiple local region tensor blocks; among them, the local region tensor block can be represented as R k,r , where k represents the k-th time window or the k-th image object, and r represents the r-th local region; S3.3.2.2 Feature extraction within the region: For each local region tensor block R k,r Gray-scale statistical features, texture features, edge contour features, and local morphological features are extracted respectively; The gray-level statistical features include the region's average gray level, maximum gray level, minimum gray level, and gray-level variance; the texture features include statistics on gray-level co-occurrence relationships, intensity of local gray-level changes, or descriptions of local uniformity; the edge contour features include the number of edges in the region, edge intensity, or descriptions of boundary continuity; and the local morphological features include the region's area, volume, aspect ratio, compactness, or outer boundary dimensions. S3.3.2.3 Aggregation Calculation within a Local Area: Combines multiple features extracted from the same local area in a preset order to generate a local area feature vector.

[0105] In some implementations, the local region aggregation process can be represented as:

[0106] Among them, R k,r Let G represent the r-th local region tensor block in the k-th image object, where Agg represents the local aggregation function. k,r This indicates the aggregation result for the corresponding local region.

[0107] S3.3.2.4 Regional-level result output: Multiple local region feature vectors are written into the regional-level feature result set in spatial order, and associated with the corresponding patient identifier, time window identifier, spatial region identifier and image index item, so as to be used for subsequent multi-source fusion processing or partial aggregation.

[0108] In some implementations, the local aggregation function Agg can be implemented by summarizing statistics within a region, concatenating local features, or weighted aggregation. Preferably, grayscale statistical features, texture features, edge contour features, and local morphological features within the same region can be concatenated in a preset order to form a local region feature vector.

[0109] S3.3.3 Attribute Condition Aggregation of Multi-Source Fusion Tensors: Perform local filtering, grouping summary and feature splicing on multi-source fusion high-dimensional tensors according to diagnostic category, examination item category, treatment plan category, physiological indicator category or patient grouping conditions to generate attribute-level aggregation results.

[0110] In some implementations, the attribute condition aggregation includes the following steps: S3.3.3.1 Attribute Condition Setting: Determine attribute filtering conditions based on preset aggregation targets. The attribute filtering conditions include one or more of the following: diagnostic category conditions, examination item category conditions, treatment plan category conditions, physiological indicator category conditions, and patient grouping conditions.

[0111] S3.3.3.2 Local Filtering: For each tensor object in the multi-source fusion high-dimensional tensor, filter according to the attribute filtering conditions, and retain only tensor objects that meet the attribute conditions.

[0112] In some implementations, the local filtering process can be represented as:

[0113] in, Let m represent the k-th multi-source fusion tensor set. iLet A represent the i-th tensor object. k This represents the set of attribute conditions corresponding to the current aggregation target. This represents the candidate tensor set obtained after attribute filtering.

[0114] S3.3.3.3 Grouping and Summarizing: Summarizing the filtered candidate tensor sets Group patients according to the same diagnostic category, the same examination item category, the same treatment plan category, the same patient group, or a preset attribute combination, and count the number, mean, extreme value, aggregation vector, or feature summary of each group.

[0115] S3.3.3.4 Feature Concatenation: Concatenate the statistical results, feature summaries, and intermediate vectors corresponding to each attribute group according to the preset attribute order to generate an attribute-level aggregated result vector.

[0116] S3.3.3.5 Write-back of results: Write the attribute-level aggregation results to the basic storage module and establish association relationships with the corresponding attribute condition identifiers, time window identifiers, patient group identifiers and index items for subsequent retrieval, trend analysis or model calls.

[0117] In some implementations, the attribute condition matching in the local filtering can be either a full match or a similar category match using a preset attribute mapping relationship. When similar category matching is used, different but related diagnostic categories, examination item categories, or treatment plan categories can be grouped into the same aggregation group according to the preset attribute mapping table.

[0118] S3.3.4 Write back intermediate feature results: Write back the window-level statistical feature vectors, region-level feature results and attribute-level aggregation results obtained from partial aggregation calculations to the basic storage module, and establish association relationships with the corresponding tensor identifiers, time window identifiers, spatial range identifiers and index items to support subsequent retrieval, continued calculation and model calling.

[0119] In some implementations, the local aggregation process can be represented as:

[0120] in, G represents the tensor object within the k-th time window or spatial block, Agg represents the local aggregation function, and G represents the tensor object within the k-th time window or spatial block. k This indicates the corresponding local aggregation result.

[0121] Preferably, the local aggregation function can be expressed as:

[0122] Among them, t i Represents a set of tensor objects Tk The i-th local feature vector or local statistical result, where n represents the number of features participating in the aggregation, η i Let represent the corresponding aggregate weight coefficient, and satisfy:

[0123] In some implementation methods: (1) When the importance of each local feature is the same, the local aggregation function can adopt an equal-weighted average aggregation method. ; (2) When different local features correspond to different time segments, spatial regions or attribute conditions with different importance, the local aggregation function can adopt a weighted aggregation method, by presetting weight coefficients. Weighted summation of different local features; (3) When the local aggregation result needs to retain the independent information of multiple features, multiple local feature vectors can be spliced ​​in a preset order before or after weighted aggregation to form a complete intermediate feature result.

[0124] After the local aggregation is completed, the local aggregation result G is... k The corresponding tensor identifier, time window identifier, spatial range identifier, attribute condition identifier, and version number information are written into the basic storage module for subsequent retrieval, continued calculation, and model invocation.

[0125] S3.3.5 In some implementations, the following optimization methods can be adopted to improve processing efficiency: S3.3.5.1 Block processing: Divide the large-scale image tensor or high-dimensional fusion tensor into multiple tensor blocks according to a preset size, perform local aggregation calculations in each tensor block, and summarize the results after the calculations of each tensor block are completed.

[0126] S3.3.5.2 Parallel Processing: Simultaneously perform local aggregation calculations on data from different time windows, different spatial blocks, or different patient groups to improve the throughput of local aggregation processing.

[0127] S3.3.5.3 Memory reuse: Reuse intermediate buffers for adjacent time windows or blocks in the same space, and use overwrite or version replacement methods for intermediate results that have been aggregated but still need to be retained for a short time, in order to reduce repeated memory allocation and data copying.

[0128] S4: Construct a dual-threshold elastic scheduling mechanism based on a spatiotemporal correlation prediction model like Figure 5As shown, this step is used to collect load characteristics of the storage and computing nodes, predict load change trends, generate scheduling strategies, and execute the scaling up and down of computing resources, thereby improving the system's adaptability to load fluctuations in medical data processing services.

[0129] S4.1: Load Characteristic Collection: Deploy load monitoring units on each compute node to periodically collect load characteristic data such as resource utilization, task status, input / output status, and data throughput. The load characteristic data includes at least one or more of the following: S4.1.1 Resource utilization characteristics: CPU utilization, GPU utilization, memory utilization, video memory utilization, storage capacity utilization, etc. S4.1.2 Task status characteristics: number of tasks to be executed, number of tasks in operation, task queue length, average task waiting time, average task execution time, distribution of the number of tasks with different priorities, etc. S4.1.3 Input / output status characteristics: storage read / write rate, network transmission rate, IO wait time, node input throughput, node output throughput, etc. S4.1.4 Data distribution characteristics: number of data shards held by the current node, number of newly added medical events per unit time, number of tensor objects in the current window, number of local aggregation results, etc.

[0130] In some implementations, the load monitoring unit updates the above features at preset time intervals and writes the load features at multiple consecutive time points into the load feature cache or historical feature storage area.

[0131] To ensure data consistency for subsequent predictions, it is preferable to record the collection timestamp, node identifier, and current scheduling cycle number simultaneously during each data collection.

[0132] S4.2: Load forecasting model construction: Based on spatiotemporal correlation features and time series features, a load forecasting model is constructed to predict the load change trend within a future time window.

[0133] A load prediction function can be constructed based on resource utilization, task status, and data throughput, and it is expressed as follows:

[0134] in, +Δ represents the predicted load at future time t+Δ, Ut represents the resource utilization characteristic at the current time, Qt represents the task status characteristic at the current time, Bt represents the data throughput characteristic at the current time, and g represents the load prediction model.

[0135] The load prediction model can be a prediction model that combines graph structure spatiotemporal correlation modeling with time series modeling; in some implementations, a fusion model composed of STGNN and LSTM can be used.

[0136] Preferably, the fusion model includes the following structure: S4.2.1 Graph Structure Spatiotemporal Relation Modeling Layer: Each storage node is regarded as a node in a graph, and the data sharding association, task collaboration, network connection or resource scheduling dependency between nodes are constructed as graph edges; Graph structure spatiotemporal correlation modeling is performed on the resource utilization characteristics, task status characteristics and data throughput characteristics of each node at multiple consecutive time steps to obtain the spatiotemporal correlation feature representation of each node; S4.2.2 Time Series Modeling Layer: The spatiotemporal correlation feature representation output from the graph structure spatiotemporal correlation modeling layer is input into the Long Short-Term Memory (LSTM) network in chronological order to perform time series modeling of node load change trends, thereby obtaining a load trend representation within the future prediction window. S4.2.3 Predicting Output Layer: The load trend representation output from the time series modeling layer is input into the prediction output layer to generate node load prediction values ​​for one or more future time windows.

[0137] In some implementations, the processing of the fusion model can be represented as follows:

[0138]

[0139] Where A represents the association matrix between storage nodes, H t This represents the spatiotemporal correlation feature representation of nodes obtained through spatiotemporal correlation modeling using a graph structure. LSTM represents the time series prediction process based on a Long Short-Term Memory network. H represents t The prediction results.

[0140] In some implementations, the correlation matrix A can be constructed based on the data sharding and sharing relationship between nodes, task migration frequency, network connection topology, or business collaboration relationship; the prediction output layer can output single-node load prediction values, multi-node load prediction values, or global load trend results for subsequent dual-threshold scheduling strategy generation steps to call.

[0141] S4.3: Scheduling Strategy Generation: A scheduling strategy is generated based on the load prediction results and preset expansion and contraction thresholds. The scheduling strategy includes threshold-triggered scheduling and predictive advance scheduling. In resource contraction scenarios, it is preferable to execute tasks in the order of idle resources first, followed by low-priority tasks, and to complete task migration before resource contraction. In some implementations, scheduling actions can be generated based on the predicted load and preset expansion and contraction thresholds. The scheduling action can be expressed as:

[0142]

[0143]

[0144] in, This indicates the scheduling action at the current moment. Indicates the capacity expansion threshold. This represents the contraction threshold. The scheduling strategy preferably includes one or more of the following: S4.3.1 Threshold-triggered scheduling: When the real-time load or predicted load exceeds the expansion threshold, an expansion strategy is generated immediately; when the real-time load or predicted load is below the shrinkage threshold, a resource shrinkage strategy is generated. S4.3.2 Predictive advance scheduling: When the forecast results indicate that there will be a business peak in the future time window, activate the warm-up nodes, idle nodes or reserve computing resources in advance; when the forecast results indicate that the load will continue to decline in the future time window, prepare for resource reclamation and task migration in advance. S4.3.3 Priority-based coordinated scheduling: When resources are scarce, priority is given to high-priority tasks such as emergency care, real-time monitoring, and early warning analysis; when resources are reduced, idle resources are reclaimed first, followed by resources occupied by low-priority tasks. S4.3.4 Data Resident Linkage Scheduling: When generating scheduling strategies, priority is given to the current location of data shards, and new tasks are executed on the node where the data resides or on a nearby node as much as possible to reduce cross-node data migration.

[0145] In resource contraction scenarios, it is preferable to execute tasks in the order of "idle resources first, then low-priority task resources", and complete task migration, state saving and index synchronization before resource contraction.

[0146] S4.4: Scheduling Execution: Perform horizontal scaling and / or vertical scaling of computing resources according to the scheduling strategy to adjust the resource configuration relationship between storage and computing nodes.

[0147] Horizontal scaling refers to increasing or decreasing the number of compute nodes, GPU nodes, or CPU nodes; vertical scaling refers to adjusting the number of CPU cores, GPU quota, memory quota, video memory quota, or storage bandwidth quota of a single node.

[0148] When performing expansion, the following order is preferred: identify warm-up nodes, idle nodes, or allocable idle resources; write the new nodes or resources into the resource mapping table; reallocate tasks to be executed to the new resources according to data residence location and task priority; update the task status table, node status table, and index reference relationships; and start the execution of computing tasks on the new resources.

[0149] When performing a contraction, the following sequence is preferred: identify idle resources; identify resources occupied by low-priority tasks; perform task migration for migrateable tasks and save task running status, intermediate results, and version number information; write the target node after task migration into the task mapping table; synchronously update the node resource mapping table, index status, and task status; reclaim the released resource quotas or shut down the corresponding node.

[0150] To ensure uninterrupted service during the contraction process, it is preferable to perform resource release actions only after the task migration is completed and the status verification is finished.

[0151] In the context of regional medical big data centers, this invention can be used to achieve unified access and intelligent scheduling of data from multiple hospitals. Specifically, after raw medical data is uniformly accessed to the storage and computing nodes, it undergoes event-based processing and spatiotemporal multi-dimensional alignment to build a three-level joint index on the near-data side; image data sharding and GPU execution tasks are co-deployed, with computing resources adjusted according to the data's location, prioritizing high-priority diagnosis and treatment services; a streaming tensor processing pipeline receives ECG and image data streams in real time, and performs some aggregation calculations and incremental result output on the storage side; the system continuously collects resource utilization, task status, and data throughput of each node based on a load monitoring unit, and identifies future peak treatment times through a load prediction model, pre-activating some idle nodes or reserving resources before the peak arrives, and gradually releasing resources during off-peak periods according to task migration and resource reclamation order, thereby improving resource utilization efficiency and reducing cross-node data migration overhead.

[0152] Furthermore, in yet another embodiment, this solution can also be implemented by setting up a system, which includes: The data acquisition module is used to acquire raw medical data and provide standardized input; The event processing and indexing module is used to clean, process, and align the raw medical data in a spatiotemporal manner, generating medical event records with spatiotemporal diagnosis and treatment coordinate labels, and constructing a three-level composite index consisting of a global primary index, local sub-indexes, and feature inverted indexes on the near data side; The heterogeneous computing scheduling module is used to classify medical data processing tasks, perform data sharding on medical event records with spatiotemporal diagnosis and treatment coordinate labels, and co-deploy the data shards and corresponding computing tasks on the storage and computing nodes according to the data residence location, so as to realize the linkage between GPU / CPU collaborative scheduling and data residence strategy. The streaming tensor processing module is used to perform tensor recombination on medical event records, generate a set of fusion candidate tensors, establish a streaming tensor processing pipeline, and perform some aggregation calculations on the storage side. The elastic scheduling module is used to collect load characteristic data, perform load prediction, generate scheduling strategies based on preset expansion and contraction thresholds, and perform horizontal and / or vertical expansion and contraction of computing resources. The basic storage module is used for distributed storage of raw medical data, index data, and calculation results, and provides underlying data read / write and address mapping services.

[0153] In some implementations, the event-based indexing module, heterogeneous computing scheduling module, streaming tensor processing module, and elastic scheduling module are all deployed on the storage node side, and data reading and writing and address mapping are coordinated through the basic storage module.

[0154] This system can execute the spatiotemporal multidimensional aligned medical big data storage and computing scheduling method provided in the previous embodiment. Those skilled in the art will understand that all or part of the above method's processes can be completed by computer program instructions controlling related hardware. The program can be stored in a computer-readable storage medium and implement the above method processes during execution. The storage medium may include a disk, optical disk, read-only memory (ROM), random access memory (RAM), solid-state drive (SSD), etc.

[0155] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention should be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope defined in the appended claims.

Claims

1. A spatiotemporal multidimensional aligned medical big data storage and computing scheduling method, characterized in that, include: Step 1: Standardize and clean the collected raw medical data, and generate medical event records based on time information, spatial location information, and diagnosis and treatment attribute information; The medical event records are mapped to a unified spatiotemporal diagnosis and treatment coordinate system to form medical event records with spatiotemporal diagnosis and treatment coordinate labels. On the near data side, a three-level joint index consisting of a global main index, local sub-indexes, and feature inverted index is constructed based on the medical event records with spatiotemporal diagnosis and treatment coordinate labels. The association between index items and the original medical data storage address is established to form a multi-dimensional joint retrieval mechanism oriented towards time dimension, space dimension, and diagnosis and treatment attribute dimension. Step 2: Based on the parallelism, computational complexity, and real-time requirements of the medical data processing tasks, heterogeneously classify them into GPU-executed tasks and CPU-executed tasks. For the medical event records with spatiotemporal diagnostic coordinate labels formed in Step 1, perform data sharding according to preset time windows, spatial ranges, or diagnostic attribute dimensions to form data shards to be processed corresponding to the medical data processing tasks. Deploy the data shards to be processed and the corresponding medical data processing tasks on the storage nodes according to their data residency locations. Based on the task classification results, schedule the GPU-executed tasks to GPU nodes and the CPU-executed tasks to CPU nodes. Link GPU / CPU collaborative scheduling with the data residency strategy to achieve heterogeneous computing resource allocation and task execution order adjustment oriented towards data locality. Step 3: Perform tensor-based reorganization on medical event records with spatiotemporal diagnostic coordinate labels that have undergone spatiotemporal multidimensional alignment, constructing a one-dimensional temporal tensor, a two-dimensional or three-dimensional image tensor, and a multi-source fusion high-dimensional tensor; generate a set of fusion candidate tensors based on patient identification, time dimension, spatial dimension, and diagnostic attribute dimension in the unified spatiotemporal diagnostic coordinates; establish a streaming tensor processing pipeline on the storage node side, and perform spatiotemporal tensor alignment, feature extraction, multi-source fusion, and incremental processing in the streaming tensor processing pipeline; Perform some aggregation computations on the storage side to reduce the pressure of cross-node data migration and subsequent centralized computation; Step 4: Collect load characteristic data consisting of resource utilization, task status, and data throughput at each storage node; A load prediction model is constructed based on spatiotemporal correlation features and time series features, and a scheduling strategy is generated based on the load prediction results and preset expansion and contraction thresholds. The computing resources are then scaled horizontally and / or vertically according to the scheduling strategy to achieve dynamic resource allocation oriented towards load fluctuations and data distribution states.

2. The method according to claim 1, characterized in that, The global master index is constructed based on patient identifiers and timestamps; the local sub-indexes are based on data types and spatial range divisions; the feature inverted index is constructed based on diagnostic and treatment attribute keywords, and joint retrieval and incremental updates are performed on the global master index, local sub-indexes, and feature inverted index.

3. The method according to claim 1, characterized in that, Step 2 further includes: allocating high-parallelism, high-computational-complexity tasks to GPUs for execution, allocating index maintenance, logic control, and preprocessing tasks to CPUs for execution, and adjusting the allocation of computing resources between GPUs and CPUs according to task priorities and resource usage.

4. The method according to claim 1, characterized in that, The tensor recombination in step 3 includes: constructing a one-dimensional sequence tensor for time-series physiological data, constructing a two-dimensional or three-dimensional spatial tensor for image data, constructing a high-dimensional fusion tensor for multi-source fusion data, and maintaining the correspondence between the dimensions during the tensor construction process.

5. The method according to claim 1, characterized in that, The streaming tensor processing pipeline includes data access processing, tensor operation processing, and result output processing. The data access processing is used to receive multi-source streaming data, the tensor operation processing is used to perform tensor alignment, feature extraction, and fusion processing, and the result output processing is used to output structured results in an incremental manner.

6. The method according to claim 1, characterized in that, The load prediction model is a fusion model based on graph structure spatiotemporal correlation modeling and time series modeling.

7. The method according to claim 1, characterized in that, The scheduling strategy includes threshold-triggered scheduling and predictive advance scheduling; resource contraction is performed in the order of first idle resources and then low-priority task resources, and task migration is completed before resource contraction.

8. A spatiotemporally multidimensional aligned medical big data storage and scheduling system, characterized in that, include: The data acquisition module is used to acquire raw medical data and provide standardized input; The event processing and indexing module is used to clean, process, and align the raw medical data in a spatiotemporal manner, generating medical event records with spatiotemporal diagnosis and treatment coordinate labels, and constructing a three-level composite index consisting of a global primary index, local sub-indexes, and feature inverted indexes on the near data side; The heterogeneous computing scheduling module is used to classify medical data processing tasks, perform data sharding on medical event records with spatiotemporal diagnosis and treatment coordinate labels, and co-deploy the data shards and corresponding computing tasks on the storage and computing nodes according to the data residence location, so as to realize the linkage between GPU / CPU collaborative scheduling and data residence strategy. The streaming tensor processing module is used to perform tensor recombination on medical event records, generate a set of fusion candidate tensors, establish a streaming tensor processing pipeline, and perform some aggregation calculations on the storage side. The elastic scheduling module is used to collect load characteristic data, perform load prediction, generate scheduling strategies based on preset expansion and contraction thresholds, and perform horizontal and / or vertical expansion and contraction of computing resources. The basic storage module is used for distributed storage of raw medical data, index data, and calculation results, and provides underlying data read / write and address mapping services.

9. A spatiotemporally multidimensional aligned medical big data storage and scheduling device, characterized in that, The method includes a processor and a memory, wherein the memory stores computer instructions, and the processor executes the computer instructions to implement the method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores program code that, when executed by a processor, implements the method described in any one of claims 1-7.

Images