A batch size determination method for distributed heterogeneous data in the medical industry

Abstract

The present application relates to a kind of medical industry distributed heterogeneous data batch size determination method, belong to medical information processing technical field, solve the problem that current technology cannot effectively balance data contrast efficiency and source database system stability.It is method that includes: the current state information of source system is collected, based on current state information, the suggested batch size, risk level and confidence are obtained by constructing prompt word input large model;Current decision is determined based on the risk level and confidence;If attenuation decision is contained in current decision, the suggested batch size is attenuated to obtain final batch size;If exploration is contained in current decision, the suggested batch size is optimized to obtain final batch size based on reinforcement learning model;If the suggestion is contained in current decision, the suggested batch size is used as final batch size.Effectively balance data contrast efficiency and the stability of source database system.

Description

Technical Field

[0001] This invention relates to the field of medical information processing technology, and in particular to a method for determining the batch size of distributed heterogeneous data in the medical industry. Background Technology

[0002] With the deepening development of medical informatization, medical institutions have accumulated massive amounts of heterogeneous data, including basic patient information, test results, imaging data, electronic medical records, and drug information. This data is scattered and stored in different types of databases, such as distributed relational databases for storing structured medical data, distributed document databases for storing semi-structured medical records, and distributed index databases for storing medical documents and retrieval data, forming a serious data silo problem.

[0003] In the context of comparing heterogeneous data systems in the medical data field, a scientific batch processing mechanism must be established for comparison operations involving heterogeneous data sources such as distributed relational databases, distributed document databases, and distributed index databases. Due to the unique characteristics of medical data—high business continuity requirements, time sensitivity, and low data quality tolerance—traditional fixed-time-window partitioning strategies (such as partitioning by fixed months / years) have significant drawbacks: firstly, coarse-grained time-dimensional partitioning cannot achieve optimal scheduling of system resources, leading to a negative correlation between processing efficiency and system stability; secondly, static batch partitioning standards are difficult to adapt to the dynamic growth characteristics of medical data, easily triggering cascading risks in the source database system. Existing methods cannot effectively balance data extraction efficiency and the stability of the source database system. Summary of the Invention

[0004] Based on the above analysis, the embodiments of the present invention aim to provide a method for determining the batch size of distributed heterogeneous data in the medical industry, in order to solve the problem that existing methods cannot effectively balance data comparison efficiency and source database system stability.

[0005] On one hand, embodiments of the present invention provide a method for determining the batch size of distributed heterogeneous data in the medical industry, including the following steps: Collect the current state information of the source system, and construct a large model of prompt words input based on the current state information to obtain the suggested batch size, risk level and confidence level; The current decision is determined based on the aforementioned risk level and confidence level; If the current decision includes a decay decision, the suggested batch size is decayed to obtain the final batch size; if the current decision includes exploration, the suggested batch size is optimized based on a reinforcement learning model to obtain the final batch size; if the current decision includes adoption of a suggestion, the suggested batch size is used as the final batch size.

[0006] Based on further improvements to the above method, the state space of the reinforcement learning model includes current state information, historical state information, risk level, and confidence level.

[0007] Based on a further improvement of the above method, the reward function of the reinforcement learning model is expressed as: ; in, , , and These represent the weighting coefficients for efficiency, stability, error penalty, and health, respectively. Indicates an efficiency reward. This indicates a stability reward. Indicates punishment for mistakes. This indicates a health reward.

[0008] Based on the further improvement of the above method, the efficiency bonus is calculated using the following formula: ; ; ; in, This indicates the actual throughput of this operation. Indicates the baseline throughput. Represents the hyperbolic tangent function. This indicates the average CPU load during this operation. This indicates the average memory usage during this operation. This indicates the average I / O latency during this operation. , and This represents the weighting coefficient.

[0009] Based on the above method, a further improvement is made to calculate the error penalty in the following way: ; in, This indicates the error type that represents the most serious error that occurred during this operation. This indicates the number of retries performed for the most serious error in this operation. Indicates the retry penalty weight. This represents the basic penalty function.

[0010] Based on further improvements to the above method, the confidence level is obtained in the following way: The variance is calculated based on the probability of each batch size predicted by the large model. The confidence level is calculated based on the variance.

[0011] Based on further improvements to the above method, the confidence level is calculated using the following formula based on the variance: ; ; ; in, Indicates the number of risk levels. Indicates the maximum variance. Represents the normalized variance. Indicates the confidence level. Indicates variance.

[0012] Based on further improvements to the above method, the risk levels include severe, high risk, medium risk, and low risk; Determining the current decision based on the aforementioned risk level and confidence level includes: If the risk level is severe, the current decision will be made manually. If the risk level is high, and the confidence level is greater than or equal to the first threshold, the current decision is to adopt the recommendation; if the confidence level is less than the first threshold but greater than or equal to the second threshold, the current decision is to use a decay strategy with a high decay intensity; if the confidence level is less than the second threshold, the current decision is to make a manual decision. If the risk level is medium risk, and the confidence level is greater than or equal to the first threshold, the current decision is an exploratory strategy; if the confidence level is less than the first threshold but greater than or equal to the second threshold, the current decision is a decay strategy with a medium decay strength; if the confidence level is less than the second threshold, the current decision is an empirical strategy. If the risk level is low and the confidence level is greater than or equal to the second threshold, the current decision is to explore the strategy; otherwise, the current decision is to adopt the recommendation.

[0013] Based on a further improvement to the above method, the suggested batch size is attenuated to obtain the final batch size in the following manner: ; ; in, Indicates the final batch size. Indicates the suggested batch size. Indicates the attenuation coefficient. Indicates the confidence level. The confidence decay function is represented by the function. The weights representing the confidence level. This represents the real-time risk decay value. Indicates real-time risk weights. This represents the risk decay value based on historical experience. This represents the risk weight based on historical experience.

[0014] Based on a further improvement of the above method, the confidence decay function is expressed as: ; in, This indicates the maximum decay rate of the confidence level.

[0015] Compared with existing technologies, this invention collects the current state information of the source database system, constructs a prompt word input model based on the current state information to obtain suggested batch size, risk level, and confidence level, and then determines the current decision based on the risk level and confidence level. Based on the suggested batch size of the large model and different decision results, the final batch size is determined, and the data corresponding to the final batch size is extracted for the current data comparison. By combining the advanced judgment ability of the large model and the adaptive optimization ability of reinforcement learning, intelligent dynamic adjustment of the data extraction batch is achieved in the process of medical heterogeneous data comparison. This fundamentally eliminates the source database system crash or severe performance fluctuations caused by data extraction and comparison operations, ensures the high availability of the medical business system, and effectively balances the data comparison efficiency and the stability of the source database system.

[0016] In this invention, the above-described technical solutions can be combined with each other to achieve more preferred combinations. Other features and advantages of this invention will be set forth in the following description, and some advantages may become apparent from the description or be learned by practicing the invention. The objects and other advantages of this invention can be realized and obtained from what is particularly pointed out in the description and drawings. Attached Figure Description

[0017] The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Throughout the drawings, the same reference numerals denote the same parts. Figure 1 This is a flowchart illustrating a method for determining the batch size of distributed heterogeneous data in the medical industry, as described in an embodiment of the present invention. Detailed Implementation

[0018] Preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings, which form part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not intended to limit the scope of the present invention.

[0019] A specific embodiment of the present invention discloses a method for determining the batch size of distributed heterogeneous data in the medical industry, such as... Figure 1 As shown, it includes the following steps: S1. Collect the current status information of the source system, and construct a large model of prompt word input based on the current status information to obtain the suggested batch size, risk level and confidence level; S2. Determine the current decision based on the risk level and confidence level; S3. If the current decision includes a decay decision, the suggested batch size is decayed to obtain the final batch size; if the current decision includes exploration, the suggested batch size is optimized based on a reinforcement learning model to obtain the final batch size; if the current decision includes adoption suggestions, the suggested batch size is used as the final batch size.

[0020] The source database system is the database system to be compared.

[0021] The current status information of the source database system includes the following key performance indicators: CPU utilization monitoring includes: total system CPU utilization, database / index service process CPU utilization, CPU load balancing, CPU temperature, etc.

[0022] Memory usage monitoring: Total system memory usage, database / index service process memory usage, memory overflow (OOM) events, memory fragmentation, cache hit rate, etc.

[0023] Disk I / O monitoring: Read / write throughput, I / O wait queue length, disk space utilization, disk response time, IOPS performance metrics, etc.

[0024] Network I / O monitoring: Incoming and outgoing traffic between the data source and the extraction service, network latency, packet loss rate, number of connections, etc.

[0025] Internal database metrics include: active connection count, transaction / query lock status, internal queue depth, slow query log, exception / warning log, and index usage.

[0026] Business metrics monitoring: Key business-level metrics such as data processing speed, error rate, success rate, and average response time.

[0027] During implementation, continuous indices are discretized or normalized as necessary to construct standardized state vectors, which can then be used as inputs for large models and reinforcement learning algorithms.

[0028] Then, prompt words are constructed based on the current state information.

[0029] During implementation, a general large model can be used directly (without fine-tuning). Therefore, it is necessary to encapsulate all the necessary background information (such as historical operations, real-time indicators, database configuration, etc.) and the problem to be solved in a context through sophisticated cue word engineering, and hand it over to the general large model (such as localized versions of Llama 3, GPT-4, and Claude 3) for inference. This method is also known as "contextual learning".

[0030] In order to put "contextual learning" into practice, it is essential to construct a cue word with high information density and clear structure.

[0031] Below is a prompt word template that can be directly generated by the system and input into the LLM.

[0032] # 1. Role and Task Definition # Define the identity, target, and output format of the LLM to ensure its behavior meets expectations.

[0033] You are a top-tier database architect and operations specialist (SRE) for healthcare information systems, with expertise in performance tuning and risk control for heterogeneous data systems such as Greenplum, Elasticsearch, and MongoDB.

[0034] Your task is to accurately assess the potential risks of an upcoming data operation based on the extremely detailed "decision context" provided below, and to provide specific, actionable optimization recommendations.

[0035] Your analysis must be data-driven and clearly explain your decision-making logic.

[0036] # 2. Decision Context # Contains all the information needed to perform this assessment.

[0037] # 2.1 Business and Task Profile # Describe the business context of the operation, which determines the risk tolerance.

[0038] # 2.2 Operations to be evaluated # Clearly state what we want to achieve; this is the core objective of the assessment. This application requires the large model to provide a suggested batch size and a quantified risk level.

[0039] # 2.3 Current System Real-time Status # A snapshot of the system at the moment before the operation is executed, showing the current status information collected in real time.

[0040] # 2.4 Relevant Historical Operation Records.

[0041] # Provides the most recent operation record that is most similar to the current state, including the previous state -> action -> subsequent state -> result.

[0042] # 3. Evaluation Request # Issue specific instructions to the LLM.

[0043] Based on the complete decision-making context above, please conduct a comprehensive risk assessment of the "operation to be evaluated".

[0044] When implementing this, the risk level is divided into four levels: severe, high risk, medium risk, and low risk.

[0045] During implementation, prompts can be used to instruct the large model to output its assessment confidence level. Alternatively, confidence levels can be calculated based on the large model's predicted probabilities. Specifically, prompts can be used to instruct the large model to output its calculated batch size, corresponding probability, risk level, and suggested batch size, allowing confidence levels to be calculated based on the probability corresponding to each batch size.

[0046] Specifically, the confidence level is obtained using the following method: The variance is calculated based on the probability of each batch size predicted by the large model. The confidence level is calculated based on the variance.

[0047] Within the large model, the selection of each digital token (for each batch size) is accompanied by a probability distribution. The degree of certainty can be quantified by analyzing these probabilities. If the model's selection probabilities are similar across multiple alternative parameters, it indicates high uncertainty, and the confidence level should be lowered.

[0048] Specifically, the raw, unnormalized prediction scores of a large language model before the final output layer (usually a Softmax activation function layer) represent its probability of obtaining each digital token.

[0049] Calculate the variance of these probability distributions .

[0050] The confidence level is calculated based on the variance using the following formula: ; ; ; in, Indicates the number of risk levels. Indicates the maximum variance. Represents the normalized variance. Indicates the confidence level. Indicates variance.

[0051] The smaller the variance, the more concentrated the probability distribution, and the higher the confidence level of the model should be; conversely, the larger the variance, the more dispersed the probability distribution, and the lower the confidence level of the model should be.

[0052] Then, the current decision is determined based on the risk level and confidence level, that is, based on the suggested batch size, corresponding risk level and confidence level given by the large model.

[0053] Specifically, if the risk level is severe, the current decision will be made manually. If the risk level is high, and the confidence level is greater than or equal to the first threshold, the current decision is to adopt the recommendation; if the confidence level is less than the first threshold but greater than or equal to the second threshold, the current decision is to use a decay strategy with a high decay intensity; if the confidence level is less than the second threshold, the current decision is to make a manual decision. If the risk level is medium risk, and the confidence level is greater than or equal to the first threshold, the current decision is an exploratory strategy; if the confidence level is less than the first threshold but greater than or equal to the second threshold, the current decision is a decay strategy with a medium decay strength; if the confidence level is less than the second threshold, the current decision is an empirical strategy. If the risk level is low and the confidence level is greater than or equal to the second threshold, the current decision is to explore the strategy; otherwise, the current decision is to adopt the recommendation.

[0054] In practice, the first threshold can be set to 0.85 and the second threshold can be set to 0.6.

[0055] Once the large model determines that the risk has reached the "severe" level, it means that a catastrophic event such as service interruption may occur. Therefore, the most conservative strategy must be adopted, requiring manual intervention. In implementation, the highest level alert can be sent to the DBA and operations team, along with the complete decision context, requiring manual analysis and decision-making.

[0056] If the large model indicates a high risk level, the corresponding decision is determined based on the confidence level. If the confidence level is high (greater than or equal to the first threshold), the recommended value given by the LLM is fully adopted, and no optimization exploration is conducted. Simultaneously, due to the high risk, an advanced alert can be triggered to notify operations personnel to pay attention to this batch of tasks. If the confidence level is medium (less than the first threshold but greater than or equal to the second threshold), the large model considers there to be a high risk, but lacks confidence in this judgment. Therefore, the current decision is a decay strategy, that is, the recommended batch size given by the large model is reduced to obtain execution parameters that are more conservative than the large LLM recommended value. If the confidence level is low, the large model vaguely senses a high risk, which is a very dangerous situation. Therefore, the current decision is a manual decision, sending an alert to operations personnel.

[0057] If the large model assigns a medium risk level, the corresponding decision is determined based on the confidence level. If the confidence level is high, the large model clearly identifies a medium risk and provides a clear suggestion. Since the risk level is not high, further numerical optimization can be performed; that is, the current decision is an exploratory strategy, and subsequent optimization exploration will be carried out using an RL agent. If the confidence level is medium (the most common scenario), the large model provides a suggestion but there is some uncertainty. Therefore, the batch size of the suggestion given by the large model is conservatively adjusted; that is, the decision is a decay strategy, and the decay strength can be moderate. If the confidence level is low, the judgment of the large model is very ambiguous. To ensure safety, the current strategy is an empirical strategy. During implementation, the batch size corresponding to operations similar to the current state information can be retrieved from the historical records as the final batch size.

[0058] If the risk level is low and the confidence level exceeds the second threshold, the current environment is considered safe, and the decision is to adopt an exploratory strategy. If the confidence level is less than the second threshold, the current decision is to adopt the recommendation and not to proceed with exploration.

[0059] Then, the final batch size is determined based on the current decision.

[0060] Specifically, if the current decision includes a decay decision, the suggested batch size is decayed to obtain the final batch size.

[0061] Specifically, the suggested batch size is attenuated to obtain the final batch size using the following method: ; ; in, Indicates the final batch size. Indicates the suggested batch size. Indicates the attenuation coefficient. Indicates the confidence level. The confidence decay function is represented by the function. The weights representing the confidence level. This represents the real-time risk decay value. Indicates real-time risk weights. This represents the risk decay value based on historical experience. This represents the risk weight based on historical experience.

[0062] Specifically, the confidence decay function is expressed as follows: ; in, This indicates the maximum decay rate of the confidence level.

[0063] The confidence level of a large model reflects its self-awareness. Within the medium confidence range [0.6, 0.85), the lower the score, the more ambiguous the scenario encountered by the large model and the less historical data support it has. Therefore, the system should apply greater conservatism.

[0064] The maximum attenuation percentage that may result solely from insufficient confidence is defined (e.g., 0.4, indicating a maximum reduction of 40%). In practice, the maximum attenuation percentage varies depending on the intensity of attenuation; the maximum attenuation percentage for medium-intensity attenuation is less than that for high-intensity attenuation.

[0065] No matter how credible the recommendations of a large model are, the current actual health of the system cannot be ignored. A system under high load and pressure is fragile, and any large operation could be the straw that breaks the camel's back. Therefore, the impact of real-time risks needs to be considered.

[0066] Specifically, the real-time risk decay value is obtained using the following method: Calculate the risk score for each key indicator based on the current status information; The real-time risk decay value is obtained based on the risk score of each key indicator.

[0067] During implementation, health and danger thresholds can be set for each key performance indicator, and then the risk score of each key indicator in the current status information can be calculated.

[0068] Specifically, the risk score for each key indicator is calculated using the following formula: ; in, This indicates the value of the key indicator in the current status information. This indicates the health threshold for this key indicator. This indicates the danger threshold for this key indicator.

[0069] The real-time risk decay value is obtained by weighted averaging of the risk scores of all key indicators.

[0070] In order to learn from past failures, if the current operating environment is highly similar to a historical operating environment that caused system delays or even failures, a strong "punitive" attenuation must be applied to avoid repeating the same mistakes.

[0071] Specifically, the historical risk attenuation value is obtained using the following method: Filter high-risk operations from the historical operation database that have a similarity to the current status information exceeding a preset threshold; Calculate the average similarity between the selected high-risk operations and the current state information; The historical experience risk decay value is obtained based on the average similarity and the maximum historical risk decay ratio.

[0072] In implementation, the current state information and the state information of high-risk operations in historical operation control can be converted into vectors, and the similarity between the two state information sets can be calculated using a similarity calculation method. For example, the cosine similarity method can be used to calculate the similarity.

[0073] In practice, the average similarity is multiplied by the maximum historical risk attenuation ratio to obtain the historical experience risk attenuation value.

[0074] If the current decision includes adopting the recommendations, then the recommended batch size given by the large model will be used as the final batch size.

[0075] If the current decision involves exploration, the proposed batch size is optimized based on a reinforcement learning model to obtain the final batch size.

[0076] In practice, the state space of a reinforcement learning model includes current state information, historical state information, risk level, and confidence level.

[0077] During implementation, historical status information includes the system status information of the last operation, as well as the risk level, risk level and confidence level obtained from the large model.

[0078] The action space defines all possible operations that a reinforcement learning model can perform. In this application, the action space is a discrete set of actions from which the agent of the reinforcement learning model needs to select an optimal action at each decision point. For example, the action space may include INCREASE_BATCH (+2000 actions), MAINTAIN_BATCH (unchanged), and DECREASE_BATCH (-1000 actions).

[0079] Input the current state information, historical state information, risk level, and confidence level into the reinforcement learning model, and the model will provide the current action. If the action is MAINTAIN_BATCH (unchanged), the suggested batch size will be used as the final batch size; if the action is INCREASE_BATCH (+2000 items), the suggested batch size will be increased by 2000 items, and the final batch size will be calculated by decreasing the suggested batch size by 1000 items.

[0080] In practice, reinforcement learning models can employ reinforcement learning algorithms such as deep Q-networks and Q-Learning.

[0081] To accurately and effectively guide the behavior of reinforcement learning models and train them, this invention designs a comprehensive multi-dimensional reward function. This function quantifies multiple expected outcomes (efficiency, stability, error-free, health) of a single operation into a single scalar reward value. After each operation, the batch size is determined based on the action given by the reinforcement learning model, and the reward value is calculated after comparing the data according to this batch size.

[0082] Specifically, the reward function of a reinforcement learning model is expressed as: ; in, , , and These represent the weighting coefficients for efficiency, stability, error penalty, and health, respectively. For example, they could be set to 0.4, 0.3, 0.2, and 0.1. During implementation, these values ​​can be adjusted based on the priorities of specific business scenarios. Indicates an efficiency reward. This indicates a stability reward. Indicates punishment for mistakes. This indicates a health reward.

[0083] Specifically, the goal of efficiency rewards is to reward higher data processing volume per unit time (throughput) while also taking into account lower costs.

[0084] Specifically, the efficiency bonus is calculated using the following formula: ; ; ; in, This represents the actual throughput of this operation, calculated as batch size / operation time. This represents the baseline throughput, a value pre-set based on business data. The hyperbolic tangent function is represented. When the throughput is equal to the benchmark, the reward is 50 points; when it is significantly higher than the benchmark, the reward approaches 100 points; and when it is lower than the benchmark, the reward approaches 0 points. This indicates the average CPU load during this operation. This indicates the average memory usage during this operation. This indicates the average I / O latency during this operation. , and This represents the weighting coefficient.

[0085] This represents the resource consumption cost factor, which measures how many system resources are consumed to achieve the current throughput. The more resources consumed, the larger this factor becomes, and the greater the "discount" on the basic efficiency reward. This represents the normalization function.

[0086] The target of the stability reward is the smooth behavior that has a small impact on the source system during operation.

[0087] ; ;

[0088] in, Indicates CPU load volatility. The weighting coefficients can be represented by... The calculation measures the fluctuation of CPU peak values ​​relative to the average value. This indicates the highest CPU load value during this operation, representing the maximum instantaneous pressure this operation exerted on CPU resources. This represents the average CPU load during this operation. Indicates I / O wait volatility. This represents the corresponding weighting coefficient, which can be determined according to... The calculated value is used to measure the jitter level of I / O wait. This indicates the maximum I / O wait time during this operation, representing the average workload of the disk system during this operation. This represents the average I / O wait time during this operation, indicating the maximum instantaneous blocking effect this operation has on the disk system.

[0089] This indicates the internal pressure penalty item of the database.

[0090] This represents the total database lock wait time during this operation. Lock wait affects online business response and is therefore included in the reward function. The weighting coefficient represents the database lock wait time.

[0091] This represents the number of queries queued in the database. This represents the weighting coefficient for the number of sorted queries.

[0092] This indicates a fluctuation penalty.

[0093] The goal of error penalty is to severely punish operational failures and data errors. Specifically, error penalties are calculated as follows: ; in, This indicates the error type that represents the most serious error that occurred during this operation. This indicates the number of retries performed for the most serious error in this operation. Indicates the retry penalty weight. This represents the basic penalty function, used to map different error types to penalty values.

[0094] In practice, the basic penalty function can be defined as follows: ; In practice, a base score is found through a basic penalty function, and then the base score is weighted and amplified according to the number of retries to obtain a comprehensive penalty score that accurately reflects the "worst outcome" and "recovery cost".

[0095] The goal of health rewards is to reward behaviors that allow the system to quickly return to a healthy state after an action is performed.

[0096] ; ; ; in, This indicates the average CPU load after this operation. This indicates the CPU load health threshold. This indicates the average memory usage after this operation. This indicates the healthy threshold for average memory utilization. Indicates CPU health. Indicates memory health status.

[0097] In practice, the reinforcement learning model can be trained using existing training methods based on samples collected from historical operation information.

[0098] The efficient comparison method for distributed heterogeneous data in the medical industry provided in this embodiment combines the advanced judgment capabilities of large models with the adaptive optimization capabilities of reinforcement learning to achieve intelligent dynamic adjustment of data extraction batches during the comparison of heterogeneous medical data. This fundamentally eliminates the source database system crashes or severe performance fluctuations caused by data extraction and comparison operations, ensuring the high availability of the medical business system and effectively balancing data comparison efficiency and the stability of the source database system.

[0099] Those skilled in the art will understand that all or part of the processes of the methods described in the above embodiments can be implemented by a computer program instructing related hardware, and the program can be stored in a computer-readable storage medium. The computer-readable storage medium may be a disk, optical disk, read-only memory, or random access memory, etc.

[0100] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for determining the batch size of distributed heterogeneous data in the medical industry, characterized in that, Includes the following steps: Collect the current state information of the source system, and construct a large model of prompt words input based on the current state information to obtain the suggested batch size, risk level and confidence level; The current decision is determined based on the aforementioned risk level and confidence level; If the current decision includes a decay decision, then the suggested batch size is decayed to obtain the final batch size; If the current decision involves exploration, the proposed batch size is optimized based on a reinforcement learning model to obtain the final batch size; If the current decision includes a recommendation to be adopted, then the batch size of that recommendation will be used as the final batch size. The state space of the reinforcement learning model includes current state information, historical state information, risk level, and confidence level; The reward function of the reinforcement learning model is expressed as: ； in, , , and These represent the weighting coefficients for efficiency, stability, error penalty, and health, respectively. Indicates an efficiency reward. Indicates a stability reward. Indicates punishment for mistakes. Indicates a health reward; The suggested batch size is decreased to obtain the final batch size using the following method: ； ； in, Indicates the final batch size. Indicates the suggested batch size. Indicates the attenuation coefficient. Indicates the confidence level. The confidence decay function is represented by the function. The weights representing the confidence level. This represents the real-time risk decay value. Indicates real-time risk weights. This represents the risk decay value based on historical experience. This represents the risk weight based on historical experience.

2. The method for determining batch size of distributed heterogeneous data in the medical industry according to claim 1, characterized in that, The efficiency bonus is calculated using the following formula: ； ； ； in, This indicates the actual throughput of this operation. Indicates the baseline throughput. Represents the hyperbolic tangent function. This indicates the average CPU load during this operation. This indicates the average memory usage during this operation. This indicates the average I / O latency during this operation. , and Indicates the weighting coefficient. Indicates the resource consumption cost factor. This represents the normalization function.

3. The method for determining the batch size of distributed heterogeneous data in the medical industry according to claim 1, characterized in that, Error penalties are calculated as follows: ； in, This indicates the error type that represents the most serious error that occurred during this operation. This indicates the number of retries performed for the most serious error in this operation. Indicates the retry penalty weight. This represents the basic penalty function.

4. The method for determining batch size of distributed heterogeneous data in the medical industry according to claim 1, characterized in that, The confidence level is obtained in the following way: The variance is calculated based on the probability of each batch size predicted by the large model. The confidence level is calculated based on the variance.

5. The method for determining the batch size of distributed heterogeneous data in the medical industry according to claim 4, characterized in that, The confidence level is calculated based on the variance using the following formula: ； ； ； in, Indicates the number of risk levels. Indicates the maximum variance. Represents the normalized variance. Indicates the confidence level. Indicates variance.

6. The method for determining batch size of distributed heterogeneous data in the medical industry according to claim 1, characterized in that, The risk levels include severe, high risk, medium risk, and low risk; Determining the current decision based on the aforementioned risk level and confidence level includes: If the risk level is severe, the current decision will be made manually. If the risk level is high, and the confidence level is greater than or equal to the first threshold, the current decision is to adopt the recommendation; if the confidence level is less than the first threshold but greater than or equal to the second threshold, the current decision is to use a decay strategy with a high decay intensity; if the confidence level is less than the second threshold, the current decision is to make a manual decision. If the risk level is medium risk, and the confidence level is greater than or equal to the first threshold, the current decision is an exploratory strategy; if the confidence level is less than the first threshold but greater than or equal to the second threshold, the current decision is a decay strategy with a medium decay strength; if the confidence level is less than the second threshold, the current decision is an empirical strategy. If the risk level is low and the confidence level is greater than or equal to the second threshold, the current decision is to explore the strategy; otherwise, the current decision is to adopt the recommendation.

7. The method for determining batch size of distributed heterogeneous data in the medical industry according to claim 1, characterized in that, The confidence decay function is expressed as: ； in, This indicates the maximum decay rate of the confidence level.

Images