Medical data processing method and device, equipment and storage medium

Abstract

The invention discloses a medical data processing method and device, equipment and a storage medium, and the method comprises the steps: obtaining medical queue data, carrying out the data preprocessing of the medical queue data, and obtaining to-be-processed data; based on the to-be-processed data, generating a medical data analysis workflow by utilizing a visual tool of a preset workflow engine and a workflow description language; splitting the medical data analysis workflow into a plurality of sub-tasks according to the load condition of each computing node and the characteristics of the medical data analysis workflow, and distributing each sub-task to different computing nodes; and monitoring the execution state of the sub-tasks in each computing node in real time, and outputting a medical data analysis result until all the sub-tasks are completed. The medical data analysis workflow is split according to the load condition of each computing node and the characteristics of the medical data analysis workflow, and each subtask is allocated to different computing nodes, so that the efficiency of resource scheduling and data processing of medical queue data analysis is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of workflow processing, and in particular to a medical data processing method, device, equipment and storage medium. Background Art

[0002] Workflow is a computational model of the workflow, which means that the logic and rules of how the work in the workflow is organized together are identified in the computer with an appropriate model and calculated. Hadoop and Spark are currently the most widely used big data processing frameworks, providing distributed data storage and computing capabilities. Hadoop mainly uses HDFS for distributed file storage and performs parallel computing through the MapReduce model; Spark processes data in memory, provides higher computing speed, and supports a variety of data processing tasks, such as SQL queries, machine learning, and graph computing. These frameworks can be used for distributed processing of large-scale medical data and provide a certain degree of task scheduling and resource management functions. They also have data sharding and parallel processing capabilities, and can process large-scale data in batches.

[0003] However, Hadoop and Spark are general big data processing frameworks that are not optimized for the particularities of medical data (such as data standardization, privacy protection, and queue analysis). Their task scheduling mechanisms are relatively fixed and lack dynamic resource scheduling strategies for specific scenarios. In addition, these frameworks do not have a visual workflow engine and a standardized description language suitable for medical data.

[0004] Therefore, there is an urgent need for a medical data processing method that can effectively improve the resource scheduling and data processing efficiency of medical cohort data analysis. Summary of the invention

[0005] The main purpose of the present invention is to provide a medical data processing method, device, equipment and storage medium, aiming to solve the technical problems of low resource scheduling and data processing efficiency in medical queue data analysis in the prior art.

[0006] To achieve the above object, the present invention provides a medical data processing method, the method comprising the following steps:

[0007] Acquire medical queue data, and perform data preprocessing on the medical queue data to obtain data to be processed;

[0008] Based on the data to be processed, a medical data analysis workflow is generated using a visualization tool and a workflow description language of a preset workflow engine;

[0009] Splitting the medical data analysis workflow into a plurality of subtasks according to the load conditions of each computing node and the characteristics of the medical data analysis workflow, and assigning each of the subtasks to a different computing node;

[0010] Monitor the execution status of subtasks in each computing node in real time until all subtasks are completed and output the medical data analysis results.

[0011] Optionally, the step of acquiring medical queue data and performing data preprocessing on the medical queue data to obtain data to be processed includes:

[0012] Obtain medical cohort data from different sources, analyze the format and structure of medical cohort data from different sources, and obtain data analysis results;

[0013] generating a data mapping rule according to the data analysis result, and standardizing the data format of the medical queue data based on the data mapping rule to obtain standardized medical queue data;

[0014] The standardized medical cohort data is subjected to data quality inspection, and data cleaning and data filling are performed on the standardized medical cohort data according to the quality inspection result to obtain data to be processed.

[0015] Optionally, after the step of generating a medical data analysis workflow based on the data to be processed by using a visualization tool and a workflow description language of a preset workflow engine, the step further includes:

[0016] Collecting system historical data, including system load, task type and resource usage;

[0017] According to the system historical data and the medical data analysis workflow, using a preset prediction model to predict the computing resource demand of the medical data analysis workflow to obtain a prediction result;

[0018] According to the prediction result, a resource quota corresponding to the medical data analysis workflow is reserved.

[0019] Optionally, the step of reserving a resource quota corresponding to the medical data analysis workflow according to the prediction result includes:

[0020] If the prediction result indicates that the computing resource demand of the medical data analysis workflow will exceed the current system capacity, then the capacity expansion operation of the current system is triggered;

[0021] If the prediction result indicates that the computing resource demand of the medical data analysis workflow will be less than the current system capacity, a resource release operation of the current system is triggered.

[0022] Optionally, the step of generating a medical data analysis workflow based on the data to be processed using a visualization tool and a workflow description language of a preset workflow engine includes:

[0023] Determine the analysis requirements for the data to be processed, and based on the analysis requirements, use a visualization tool of a preset workflow engine to drag and connect corresponding data processing components to build an initial workflow;

[0024] Defining task nodes in the initial workflow, and determining the execution order of each task node in the initial workflow according to the logical relationship between the task nodes;

[0025] After determining the execution order of each of the task nodes, the initial workflow is optimized and adjusted using the workflow description language of a preset workflow engine to generate a medical data analysis workflow.

[0026] Optionally, the step of splitting the medical data analysis workflow into a plurality of subtasks according to the load of each computing node and the characteristics of the medical data analysis workflow, and allocating each of the subtasks to a different computing node includes:

[0027] Obtaining the load status of each computing node in the current system, and determining a task allocation strategy according to the load status of each computing node;

[0028] Based on the task allocation strategy and the characteristics of the medical data analysis workflow, the data in the medical data analysis workflow is sliced ​​by row or column to obtain a plurality of subtasks;

[0029] Allocate the subtasks to different computing nodes so that the subtasks are processed in parallel on different computing nodes.

[0030] Optionally, the step of monitoring the execution status of subtasks in each computing node in real time until all subtasks are completed and then outputting the medical data analysis results includes:

[0031] Use monitoring tools to record the resource usage and task execution status of each computing node in real time to obtain real-time monitoring data;

[0032] Predicting load change trends based on the real-time monitoring data, and adjusting task allocation strategies based on the load change trends;

[0033] When a computing node fails, the subtasks corresponding to the computing node are migrated to other healthy nodes for continued execution until all subtasks are completed and the medical data analysis results are output.

[0034] In addition, to achieve the above-mentioned purpose, the present invention also provides a medical data processing device, the device comprising:

[0035] A data acquisition module is used to acquire medical queue data and perform data preprocessing on the medical queue data to obtain data to be processed;

[0036] A workflow generation module, used to generate a medical data analysis workflow based on the data to be processed using a visualization tool and a workflow description language of a preset workflow engine;

[0037] A task allocation module, used to split the medical data analysis workflow into multiple subtasks according to the load of each computing node and the characteristics of the medical data analysis workflow, and allocate each of the subtasks to different computing nodes;

[0038] The result output module is used to regularly monitor the execution status of subtasks in each computing node until all subtasks are completed and then output the medical data analysis results.

[0039] In addition, to achieve the above-mentioned objectives, the present invention also proposes a medical data processing device, which includes: a memory, a processor, and a medical data processing program stored on the memory and executable on the processor, wherein the medical data processing program is configured to implement the steps of the medical data processing method described above.

[0040] In addition, to achieve the above-mentioned purpose, the present invention also proposes a storage medium, on which a medical data processing program is stored, and when the medical data processing program is executed by a processor, the steps of the medical data processing method described above are implemented.

[0041] The present invention discloses obtaining medical queue data, and performing data preprocessing on the medical queue data to obtain data to be processed; based on the data to be processed, generating a medical data analysis workflow using a visualization tool and a workflow description language of a preset workflow engine; splitting the medical data analysis workflow into multiple subtasks according to the load conditions of each computing node and the characteristics of the medical data analysis workflow, and assigning each of the subtasks to different computing nodes; monitoring the execution status of the subtasks in each computing node in real time until all subtasks are completed, and then outputting the medical data analysis results. Since the present invention is based on the data to be processed, generating a medical data analysis workflow using a visualization tool and a workflow description language of a preset workflow engine; then splitting the medical data analysis workflow into multiple subtasks according to the load conditions of each computing node and the characteristics of the medical data analysis workflow, and assigning each subtask to a different computing node, compared to the prior art, the present invention effectively improves the efficiency of resource scheduling and data processing for medical queue data analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 It is a flowchart of the first embodiment of the medical data processing method of the present invention;

[0043] Figure 2 It is a schematic diagram of the technical principle of the medical data processing method of the present invention;

[0044] Figure 3 It is a flowchart of a second embodiment of a medical data processing method of the present invention;

[0045] Figure 4 It is a flowchart of a third embodiment of a medical data processing method of the present invention;

[0046] Figure 5 is a structural block diagram of a first embodiment of a medical data processing device of the present invention;

[0047] Figure 6 It is a structural diagram of a medical data processing device in a hardware operating environment involved in an embodiment of the present invention.

[0048] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0049] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0050] The embodiment of the present invention provides a medical data processing method, referring to Figure 1 , Figure 1 FIG. 4 is a flow chart of the first embodiment of the medical data processing method of the present invention.

[0051] In this embodiment, the medical data processing method includes steps S10 to S40:

[0052] Step S10: Acquire medical queue data, and perform data preprocessing on the medical queue data to obtain data to be processed.

[0053] It should be noted that the execution subject of this embodiment can be a computer server device with data processing, network communication and program running functions used in medical data analysis scenarios, such as a personal computer, a laptop computer, a mobile phone, a smart watch, etc., or an electronic device capable of realizing the above functions, a medical data processing device, etc. The following takes a system including a medical data processing device (hereinafter referred to as the system) as an example to illustrate this embodiment and the following embodiments.

[0054] It should be understood that medical cohort data refers to a series of related data from a group of individual patients from a specific disease population, which usually have some common attributes or characteristics. The characteristics of these data are large data volume, many types, high value but low density, and fast generation and processing speed. Medical cohort data can be divided into two categories: structured data and unstructured data. Structured data mainly includes basic information, diagnostic information, treatment information, etc. of patients, which are usually stored in the form of relational databases; unstructured data includes medical images, pathology reports, doctor's notes, etc., which are usually stored in the form of files.

[0055] It should be noted that the above-mentioned data preprocessing of the medical queue data may include operations such as multi-source data fusion and data cleaning.

[0056] In the specific implementation, in order to ensure the accuracy of the analysis results, a multi-source data fusion module can be built to obtain medical cohort data from multiple data sources such as hospital information systems, imaging databases, and gene sequencing data, and clean and integrate them to provide multi-dimensional data support for the medical data analysis workflow.

[0057] Step S20: Based on the data to be processed, a medical data analysis workflow is generated using a visualization tool and a workflow description language of a preset workflow engine.

[0058] It should be noted that the above-mentioned preset workflow engine can be a workflow engine designed and implemented on the basis of building a distributed computing framework.

[0059] The visualization tool of the preset workflow engine can be a visual workflow design tool based on a graphical interface. Through the drag-and-drop interface, the preset analysis components (such as data cleaning, feature extraction, model training, etc.) can be dragged into the workflow, and the dependencies between components can be defined through wiring. This tool supports a variety of commonly used data analysis operations, reducing the difficulty for users to design workflows.

[0060] The workflow description language of the preset workflow engine can be a general workflow description language (such as based on XML or JSON) designed to ensure the flexibility and scalability of the workflow, which is used to describe the relationships between the various steps, parameter configurations and components in the workflow. This language must also be scalable to facilitate the addition of new analysis components and data processing modules in the future.

[0061] In a specific implementation, the analysis requirements for the data to be processed can be determined, and based on the analysis requirements, the corresponding data processing components can be dragged and connected using the visualization tools of the preset workflow engine to construct an initial workflow; task nodes are defined in the initial workflow, and the execution order of each task node in the initial workflow is determined based on the logical relationship between the task nodes; after determining the execution order of each task node, the initial workflow is optimized and adjusted using the workflow description language of the preset workflow engine to generate a medical data analysis workflow.

[0062] It should be added that in order to support the dynamic execution of medical data analysis workflows, the preset workflow engine can also include a task execution engine, which can parse the workflow description language, convert it into executable subtasks, and schedule them according to the dependencies between tasks. To enhance stability, the task execution engine can also introduce an automatic retry mechanism and breakpoint resumption function to ensure that the medical data analysis workflow can continue to execute after an unexpected interruption.

[0063] Step S30: split the medical data analysis workflow into multiple subtasks according to the load conditions of each computing node and the characteristics of the medical data analysis workflow, and assign each subtask to a different computing node.

[0064] It should be noted that according to the characteristics of the medical data analysis workflow, the medical data analysis workflow can be split into multiple subtasks using advanced scheduling algorithms (such as load balancing and dynamic priority scheduling), and these subtasks can be assigned to different computing nodes. Then, the load of each computing node can be monitored in real time, and the task allocation strategy can be dynamically adjusted according to resource utilization to ensure that the resources of each computing node are maximized.

[0065] In order to achieve flexible resource scheduling and elastic expansion and meet the diverse needs of medical queue data analysis tasks, the resource management module can be rebuilt in the system to monitor and dynamically allocate CPU, memory, storage, network and other resources in the entire cluster in real time. This module uses containerization technology (such as Docker and Kubernetes) to virtualize computing resources.

[0066] Furthermore, to support efficient processing of large-scale medical cohort data, a distributed data management module can be redesigned and deployed in the system. This module uses a distributed file system (such as HDFS) to store and manage medical cohort data, and uses data sharding technology to divide data by rows or columns to different computing nodes to optimize data reading and processing performance.

[0067] In a specific implementation, the load conditions of each computing node in the current system can be obtained, and a task allocation strategy can be determined based on the load conditions of each computing node; based on the task allocation strategy and the characteristics of the medical data analysis workflow, the data in the medical data analysis workflow can be sliced ​​by rows or columns to obtain multiple subtasks; each of the subtasks can be allocated to a different computing node so that each of the subtasks can be processed in parallel on different computing nodes.

[0068] It should be added that in order to speed up data processing, a distributed cache system (such as Redis, Memcached) can be introduced. During the analysis process (i.e., during the execution of each subtask), the distributed cache system will cache the intermediate results and commonly used data to avoid repeated calculations. At the same time, a data sharing mechanism is designed to ensure that intermediate data can be shared between subtasks, thereby further improving the overall computing efficiency.

[0069] Step S40: monitor the execution status of subtasks in each computing node in real time until all subtasks are completed, and then output the medical data analysis results.

[0070] In order to ensure the reliability of the medical data analysis results, in the specific implementation, a monitoring tool can be used to record the resource usage and task execution status of each computing node in real time to obtain real-time monitoring data; based on the real-time monitoring data, the load change trend is predicted, and the task allocation strategy is adjusted according to the load change trend; when the computing node fails, the subtask corresponding to the computing node is migrated to other healthy nodes for continued execution until all subtasks are completed, and the medical data analysis results are output.

[0071] For example, refer to Figure 2 , Figure 2The figure is a technical principle diagram of the medical data processing method of the present invention. In the figure, input and user interaction represent the interface for the user to interact with the workflow engine. The user can input data, start the workflow, monitor the progress of the workflow, etc. through this interface. The core workflow engine and task scheduling represent the brain of the entire workflow system, which is responsible for coordinating and managing the execution of the entire workflow. It includes task scheduling, that is, determining the order of task execution and allocating resources. Execution and output represent the data processing and result output of each computing node. Data input module: responsible for receiving the medical queue data input by the user. These data may be manually input through the user interface or imported from an external system in an automated way. Computing node: a node that performs actual data processing and computing tasks. These nodes may be physical servers, virtual machines or cloud services. Data standardization module: responsible for converting the input data into a unified format so that the subsequent data processing module can process the data more easily. Data processing module: further processing the data, such as cleaning, conversion, analysis, etc. Resource management: managing computing resources, such as CPU, memory, storage, etc., to ensure the efficient execution of the workflow. User interface: an interface provided to users for operation, through which users can interact with the workflow engine. Result output module: outputs the results of workflow processing to users, which may be reports, charts, data files, etc. Data sharding and caching module: responsible for splitting data into smaller parts for easier processing, and caching these data to improve processing efficiency. Workflow description module: defines and describes the structure and logic of the workflow, including the dependencies and execution order of each task.

[0072] This embodiment discloses obtaining medical queue data, and performing data preprocessing on the medical queue data to obtain data to be processed; based on the data to be processed, generating a medical data analysis workflow using a visualization tool and workflow description language of a preset workflow engine; splitting the medical data analysis workflow into multiple subtasks according to the load conditions of each computing node and the characteristics of the medical data analysis workflow, and assigning each of the subtasks to different computing nodes; monitoring the execution status of the subtasks in each computing node in real time until all subtasks are completed, and then outputting the medical data analysis results. Since this embodiment is based on the data to be processed, generating a medical data analysis workflow using a visualization tool and workflow description language of a preset workflow engine; then splitting the medical data analysis workflow into multiple subtasks according to the load conditions of each computing node and the characteristics of the medical data analysis workflow, and assigning each subtask to a different computing node, compared with the prior art, this embodiment effectively improves the efficiency of resource scheduling and data processing for medical queue data analysis.

[0073] refer to Figure 3 , Figure 3FIG. 4 is a flow chart of a second embodiment of a medical data processing method according to the present invention.

[0074] Based on the above first embodiment, in this embodiment, step S10 includes steps S101 to S103:

[0075] Step S101: Obtain medical cohort data from different sources, and analyze the formats and structures of the medical cohort data from different sources to obtain data analysis results.

[0076] Step S102: Generate a data mapping rule according to the data analysis result, and standardize the data format of the medical queue data based on the data mapping rule to obtain standardized medical queue data.

[0077] It is understandable that since medical cohort data can be obtained from a variety of different data sources such as hospital information systems, imaging databases, gene sequencing data, etc., the format and structure are not exactly the same. In order to ensure that the data can be processed in batches, the medical data from different sources need to be converted into a unified format. Therefore, the format and structure of medical cohort data from different sources can be analyzed, and data mapping rules can be generated based on the analysis results. Then, the data format of the medical cohort data can be standardized according to the data mapping rules.

[0078] In the specific implementation, in order to ensure that patient privacy is not leaked, data desensitization and encryption technologies can also be integrated into the system to ensure that patient privacy is not leaked during data transmission and analysis.

[0079] Step S103: Performing data quality detection on the standardized medical cohort data, and performing data cleaning and data filling on the standardized medical cohort data according to the quality detection result to obtain data to be processed.

[0080] It should be noted that in order to ensure that the data input into the medical data analysis workflow is of high quality and provide reliable guarantee for subsequent analysis results, data quality detection and verification functions can be introduced, including data integrity check, outlier detection, missing value filling, etc.

[0081] Furthermore, in order to improve the scalability and usability of the system, (1) the various analysis function modules in the system can be designed as plug-in components. Users can choose to load and configure different components according to their own needs. At the same time, it supports the development of custom plug-ins to facilitate future upgrades and functional expansion of the system. (2) Develop a one-click deployment tool to support flexible deployment of the system in a variety of computing environments (such as local servers, cloud platforms, HPC clusters), simplify the installation and configuration process, and improve the availability of the system. (3) Design and open standardized API interfaces to facilitate data interaction between other systems and the system of this embodiment, such as automatically importing data from hospital information systems, feeding back analysis results to other business systems, etc., to achieve seamless integration, etc.

[0082] This embodiment discloses obtaining medical cohort data from different sources, analyzing the format and structure of the medical cohort data from different sources, and obtaining data analysis results; generating data mapping rules according to the data analysis results, standardizing the data format of the medical cohort data based on the data mapping rules, and obtaining standardized medical cohort data; performing data quality detection on the standardized medical cohort data, and performing data cleaning and data filling on the standardized medical cohort data according to the quality detection results to obtain data to be processed. Since this embodiment standardizes the data format of the medical cohort data and performs data quality detection on the standardized medical cohort data, compared with the prior art, this embodiment not only ensures that the data can be processed in batches, but also ensures the reliability of subsequent medical data analysis results.

[0083] refer to Figure 4 , Figure 4 FIG. 4 is a flow chart of a third embodiment of a medical data processing method according to the present invention.

[0084] Based on the above embodiments, in this embodiment, after step S10, steps S121 to S123 are further included:

[0085] Step S121: Collect system historical data, which includes system load, task type and resource usage.

[0086] Step S122: Based on the system historical data and the medical data analysis workflow, a preset prediction model is used to predict the computing resource demand of the medical data analysis workflow to obtain a prediction result.

[0087] Step S123: According to the prediction result, reserve the resource quota corresponding to the medical data analysis workflow.

[0088] It should be noted that the above historical data may be the system load situation, task type situation and resource usage situation in the system in the past preset time period.

[0089] It should be explained that the above-mentioned preset prediction model can be a time series prediction model such as ARIMA, LSTM, etc., or it can be a deep learning model such as a neural network, a convolutional neural network (CNN) or a recurrent neural network (RNN), and this embodiment is not limited to this.

[0090] In a specific implementation, if the prediction result indicates that the computing resource demand of the medical data analysis workflow will exceed the current system capacity, the expansion operation of the current system is triggered; if the prediction result indicates that the computing resource demand of the medical data analysis workflow will be less than the current system capacity, the resource release operation of the current system is triggered.

[0091] The present embodiment discloses collecting system historical data, wherein the historical data includes system load, task type and resource usage; based on the system historical data and the medical data analysis workflow, a preset prediction model is used to predict the computing resource demand of the medical data analysis workflow to obtain a prediction result; based on the prediction result, a corresponding resource quota of the medical data analysis workflow is reserved. Compared with the prior art, the present embodiment uses a preset prediction model to estimate in advance the computing resources required for upcoming tasks and reserve corresponding resource quotas for them, thereby realizing dynamic expansion and contraction of the system and ensuring efficient use of system resources.

[0092] In addition, an embodiment of the present invention further proposes a storage medium, on which a medical data processing program is stored. When the medical data processing program is executed by a processor, the steps of the medical data processing method described above are implemented.

[0093] Reference Figure 5 , Figure 5 This is a structural block diagram of the first embodiment of the medical data processing device of the present invention.

[0094] like Figure 5 As shown, the medical data processing device proposed in the embodiment of the present invention includes: a data acquisition module 501, a workflow generation module 502, a task allocation module 503 and a result output module 504.

[0095] The data acquisition module 501 is used to acquire medical queue data and perform data preprocessing on the medical queue data to obtain data to be processed.

[0096] The workflow generation module 502 is used to generate a medical data analysis workflow based on the data to be processed by using a visualization tool and a workflow description language of a preset workflow engine.

[0097] The task allocation module 503 is used to split the medical data analysis workflow into multiple subtasks according to the load conditions of each computing node and the characteristics of the medical data analysis workflow, and allocate each subtask to a different computing node.

[0098] The result output module 504 is used to regularly monitor the execution status of subtasks in each computing node until all subtasks are completed, and then output the medical data analysis results.

[0099] The workflow generation module 502 is also used to determine the analysis requirements for the data to be processed, and based on the analysis requirements, use the visualization tool of the preset workflow engine to drag and connect the corresponding data processing components to build an initial workflow; define task nodes in the initial workflow, and determine the execution order of each task node in the initial workflow based on the logical relationship of the task nodes; after determining the execution order of each task node, use the workflow description language of the preset workflow engine to optimize and adjust the initial workflow to generate a medical data analysis workflow.

[0100] The task allocation module 503 is also used to obtain the load status of each computing node in the current system, and determine the task allocation strategy according to the load status of each computing node; based on the task allocation strategy and the characteristics of the medical data analysis workflow, the data in the medical data analysis workflow is sliced ​​by row or column to obtain multiple subtasks; each of the subtasks is allocated to different computing nodes so that each of the subtasks can be processed in parallel on different computing nodes.

[0101] The result output module 504 is also used to use monitoring tools to record the resource usage and task execution status of each computing node in real time to obtain real-time monitoring data; based on the real-time monitoring data, predict the load change trend, and adjust the task allocation strategy according to the load change trend; when the computing node fails, migrate the subtask corresponding to the computing node to other healthy nodes for continued execution until all subtasks are completed, and then output the medical data analysis results.

[0102] The embodiment of the device discloses obtaining medical queue data, and performing data preprocessing on the medical queue data to obtain data to be processed; based on the data to be processed, generating a medical data analysis workflow using the visualization tool and workflow description language of a preset workflow engine; splitting the medical data analysis workflow into multiple subtasks according to the load conditions of each computing node and the characteristics of the medical data analysis workflow, and assigning each of the subtasks to different computing nodes; real-time monitoring of the execution status of the subtasks in each computing node until all subtasks are completed, and then outputting the medical data analysis results. Since the embodiment of the device is based on the data to be processed, generating a medical data analysis workflow using the visualization tool and workflow description language of a preset workflow engine; then splitting the medical data analysis workflow into multiple subtasks according to the load conditions of each computing node and the characteristics of the medical data analysis workflow, and assigning each subtask to a different computing node, compared with the prior art, the embodiment of the device effectively improves the efficiency of resource scheduling and data processing for medical queue data analysis.

[0103] Based on the above-mentioned first embodiment of the medical data processing device of the present invention, a second embodiment of the medical data processing device of the present invention is proposed.

[0104] In this embodiment, the data acquisition module 501 is also used to acquire medical cohort data from different sources, and analyze the format and structure of the medical cohort data from different sources to obtain data analysis results; generate data mapping rules according to the data analysis results, and standardize the data format of the medical cohort data based on the data mapping rules to obtain standardized medical cohort data; perform data quality detection on the standardized medical cohort data, and perform data cleaning and data filling on the standardized medical cohort data according to the quality detection results to obtain data to be processed.

[0105] Other embodiments or specific implementations of the medical data processing device of the present invention may refer to the above-mentioned method embodiments and will not be described in detail here.

[0106] The present application provides a medical data processing device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the medical data processing method in the above-mentioned embodiment one.

[0107] Reference below Figure 6, which shows a schematic diagram of the structure of a medical data processing device suitable for implementing the embodiments of the present application. The medical data processing device in the embodiments of the present application may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 6 The medical data processing device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0108] like Figure 6 As shown, the medical data processing device may include a processing device 1001 (e.g., a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the medical data processing device are also stored. The processing device 1001, ROM1002, and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the medical data processing device to communicate with other devices wirelessly or by wire to exchange data. Although the figure shows a medical data processing device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or provided instead.

[0109] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0110] The medical data processing device provided by the present application adopts the medical data processing method in the above embodiment, which can solve the technical problems of low resource scheduling and data processing efficiency in the analysis of medical queue data in the prior art. Compared with the prior art, the beneficial effects of the medical data processing device provided by the present application are the same as the beneficial effects of the medical data processing method provided by the above embodiment, and the other technical features in the medical data processing device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0111] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0112] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0113] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0114] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0115] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as a read-only memory / random access memory, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0116] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A medical data processing method, characterized in that: The method comprises: Acquire medical queue data, and perform data preprocessing on the medical queue data to obtain data to be processed; Based on the data to be processed, a medical data analysis workflow is generated using a visualization tool and a workflow description language of a preset workflow engine; Splitting the medical data analysis workflow into a plurality of subtasks according to the load conditions of each computing node and the characteristics of the medical data analysis workflow, and assigning each of the subtasks to a different computing node; Monitor the execution status of subtasks in each computing node in real time until all subtasks are completed and output the medical data analysis results.

2. The medical data processing method according to claim 1, characterized in that: The step of obtaining medical queue data and preprocessing the medical queue data to obtain data to be processed includes: Obtain medical cohort data from different sources, analyze the format and structure of medical cohort data from different sources, and obtain data analysis results; generating a data mapping rule according to the data analysis result, and standardizing the data format of the medical queue data based on the data mapping rule to obtain standardized medical queue data; The standardized medical cohort data is subjected to data quality inspection, and data cleaning and data filling are performed on the standardized medical cohort data according to the quality inspection result to obtain data to be processed.

3. The medical data processing method according to claim 1, characterized in that: After the step of generating a medical data analysis workflow based on the data to be processed by using a visualization tool and a workflow description language of a preset workflow engine, the method further includes: Collecting system historical data, including system load, task type and resource usage; According to the system historical data and the medical data analysis workflow, using a preset prediction model to predict the computing resource demand of the medical data analysis workflow to obtain a prediction result; According to the prediction result, a resource quota corresponding to the medical data analysis workflow is reserved.

4. The medical data processing method according to claim 3, characterized in that: The step of reserving a resource quota corresponding to the medical data analysis workflow according to the prediction result comprises: If the prediction result indicates that the computing resource demand of the medical data analysis workflow will exceed the current system capacity, then the capacity expansion operation of the current system is triggered; If the prediction result indicates that the computing resource demand of the medical data analysis workflow will be less than the current system capacity, a resource release operation of the current system is triggered.

5. The medical data processing method according to claim 1, characterized in that: The step of generating a medical data analysis workflow based on the data to be processed by using a visualization tool and a workflow description language of a preset workflow engine includes: Determine the analysis requirements for the data to be processed, and based on the analysis requirements, use a visualization tool of a preset workflow engine to drag and connect corresponding data processing components to build an initial workflow; Defining task nodes in the initial workflow, and determining the execution order of each task node in the initial workflow according to the logical relationship between the task nodes; After determining the execution order of each of the task nodes, the initial workflow is optimized and adjusted using the workflow description language of a preset workflow engine to generate a medical data analysis workflow.

6. The medical data processing method according to claim 1, characterized in that: The step of splitting the medical data analysis workflow into a plurality of subtasks according to the load conditions of each computing node and the characteristics of the medical data analysis workflow, and allocating each of the subtasks to a different computing node includes: Obtaining the load status of each computing node in the current system, and determining a task allocation strategy according to the load status of each computing node; Based on the task allocation strategy and the characteristics of the medical data analysis workflow, the data in the medical data analysis workflow is sliced ​​by row or column to obtain a plurality of subtasks; Allocate the subtasks to different computing nodes so that the subtasks are processed in parallel on different computing nodes.

7. The medical data processing method according to claim 1, characterized in that: The step of monitoring the execution status of subtasks in each computing node in real time until all subtasks are completed and then outputting the medical data analysis results includes: Use monitoring tools to record the resource usage and task execution status of each computing node in real time to obtain real-time monitoring data; Predicting load change trends based on the real-time monitoring data, and adjusting task allocation strategies based on the load change trends; When a computing node fails, the subtasks corresponding to the computing node are migrated to other healthy nodes for continued execution until all subtasks are completed and the medical data analysis results are output.

8. A medical data processing device, characterized in that: The device comprises: A data acquisition module is used to acquire medical queue data and perform data preprocessing on the medical queue data to obtain data to be processed; A workflow generation module, used to generate a medical data analysis workflow based on the data to be processed using a visualization tool and a workflow description language of a preset workflow engine; A task allocation module, used to split the medical data analysis workflow into multiple subtasks according to the load of each computing node and the characteristics of the medical data analysis workflow, and allocate each of the subtasks to different computing nodes; The result output module is used to regularly monitor the execution status of subtasks in each computing node until all subtasks are completed and then output the medical data analysis results.

9. A medical data processing device, characterized in that: The device comprises: a memory, a processor, and a medical data processing program stored in the memory and executable on the processor, wherein the medical data processing program is configured to implement the steps of the medical data processing method according to any one of claims 1 to 7.

10. A storage medium, characterized in that: The storage medium stores a medical data processing program, and when the medical data processing program is executed by the processor, the steps of the medical data processing method according to any one of claims 1 to 7 are implemented.

Images