Medical aid decision-making method based on convolutional neural network and SARSA algorithm

Abstract

The invention discloses a medical aid decision-making method based on a convolutional neural network and an SARSA algorithm, and the method comprises the steps: data preparation and feature extraction: obtaining information data related to a hospital, carrying out the preprocessing of the information data, and carrying out the feature extraction of the processed information data, and belongs to the technical field of Internet of Things. The method comprises the following steps: modeling an environment through an SARSA algorithm, initializing SARSA algorithm parameters, iteratively updating a strategy, carrying out model training and optimization by using the SARSA algorithm through interaction with the environment, learning an optimal resource allocation strategy, applying the learned optimal strategy to an actual medical resource allocation scene, and evaluating the effect and performance of the scene. The efficient management of hospital operation and the optimal configuration of resources are realized. According to the method, the satisfaction degree of the patient is improved, the operation cost is reduced, and multiple goals of hospital operation are balanced through flexible reward function design.

Description

Technical Field

[0001] The present invention relates to the technical field of Internet of Things, and more specifically discloses a medical auxiliary decision-making method based on a convolutional neural network and a SARSA algorithm. Background Art

[0002] At present, the research on smart hospital decision-making support systems has attracted widespread attention, and many scholars and research institutions have made some important contributions in this field. On the one hand, the application of convolutional neural networks (CNN) in medical image analysis has made significant progress. For example, in tumor detection, disease classification and lesion segmentation, deep learning CNN models such as U-Net and ResNet have been widely used and achieved good results; on the other hand, the SARSA reinforcement learning algorithm is used to solve complex decision-making problems. Some scholars have tried to use it in emergency department resource allocation, operating room scheduling and medical equipment maintenance management, and achieve continuous optimization of resource allocation strategies through iterative learning. Although there have been certain research results at home and abroad, there is still broad research space and development potential in this field. In the actual application process, the traditional smart hospital decision-making support system has poor model performance, which leads to reduced patient satisfaction, increased hospital operating costs, and reduced medical service quality and efficiency. Summary of the invention

[0003] Purpose of the invention: The purpose of the present invention is to solve the problem that the traditional smart hospital decision support system has poor model performance in actual application, which leads to reduced patient satisfaction, increased hospital operating costs, and reduced medical service quality and efficiency.

[0004] To solve the above technical problems, according to one aspect of the present invention, more specifically, a medical decision-making assistance method based on a convolutional neural network and a SARSA algorithm comprises the following steps:

[0005] S1. Data preparation and feature extraction: Obtain hospital-related information data, pre-process the information data, and extract features from the processed information data;

[0006] S2. CNN model training: Use the CNN model to train hospital data and learn the feature representation and pattern recognition capabilities of the data;

[0007] S3, Environment Modeling: Model the hospital decision-making problem into a reinforcement learning environment, define the state space, action space and reward function;

[0008] S4, state representation and action selection: represent the state in the hospital decision problem as the state of the environment and define the selectable actions to make decisions;

[0009] S5. Reward design: Design a suitable reward function to evaluate the quality of each action to guide the algorithm to learn the optimal decision-making strategy;

[0010] S6, model training and optimization: by interacting with the environment, use the SARSA algorithm to train and optimize the model and learn the optimal decision-making strategy;

[0011] S7. Decision application and evaluation: Apply the learned optimal strategy to actual hospital decision-making scenarios and evaluate its effectiveness and performance.

[0012] Furthermore, in S1, the hospital-related data includes: patient information, patient satisfaction, medical resource allocation, hospital operating costs and medical imaging data.

[0013] Furthermore, in S1, after obtaining the medical image data, the CNN model is used to pre-process the data by normalization, and the processed pixel value is:

[0014] px new =(px-px min ) / (px max -px min )

[0015] Among them, px new is the normalized pixel value, px is the original pixel value, px min is the minimum value in the image dataset, px max is the maximum value in the image dataset.

[0016] Furthermore, in S1, after the data is preprocessed, the CNN model is used to extract features of the processed medical images. The process is as follows: the convolution kernel slides on the input feature map, and the pixels at each position are weighted and summed to obtain a new output value. The weight of the convolution kernel is obtained through training and learning, and can capture local features in the input data. After the convolution operation, a nonlinear activation function is applied to transform the output feature map to increase the expressive power of the model. By stacking multiple convolutional layers, higher-level abstract features of the input data can be extracted layer by layer. Each layer increases the number of features and captures more complex feature patterns.

[0017] Furthermore, in S3, in the process of modeling the environment, the state space is: current patient demand, department load and available resources; the action space is: the amount of resources allocated to a specific department or patient; and the reward function is: reflecting the quality of resource allocation results.

[0018] Furthermore, in S3, in the process of modeling the hospital decision-making problem into a reinforcement learning environment, the key information in the hospital operation is abstracted into a state space, and then the hospital management system takes measures to adjust the medical staff scheduling and allocate medical resources to optimize the hospital operation.

[0019] Furthermore, in S5, the reward function is set according to the hospital operation objectives. The hospital operation objectives of this application are: patient satisfaction and operating costs, and the reward function is designed accordingly:

[0020]

[0021] Among them, R t is the reward obtained at time step t, PS t is the patient satisfaction at time step t, PS e is the baseline value of patient satisfaction, which is the average satisfaction of hospital patients within one year, a is the preset weight coefficient of patient satisfaction, OC t is the operating cost of the hospital at time step t, OC e is the benchmark value of hospital operating costs, which is the average operating cost of the hospital in one year, and b is the preset weight coefficient of hospital operating costs.

[0022] Furthermore, in S7, the learned optimal strategy is applied to the actual hospital decision-making scenario, and the comprehensive performance index of the strategy is obtained through evaluation. If:

[0023]

[0024] This indicates that the optimal strategy learned is effective in actual hospital decision-making scenarios, where E is the comprehensive performance index of the strategy, PS past is patient satisfaction before strategy optimization, PS new is the patient satisfaction after strategy optimization, a is the preset weight coefficient of patient satisfaction, OC past is the hospital operating cost before strategy optimization, OC new is the hospital operating cost after strategy optimization, b is the preset weight coefficient of hospital operating cost, and δ is the preset threshold parameter of comprehensive performance of strategy.

[0025] The beneficial effects of the medical auxiliary decision-making method based on convolutional neural network and SARSA algorithm of the present invention are as follows: feature extraction and data preprocessing are performed by convolutional neural network and CNN model is constructed, medical image data are annotated, including lesion location, disease type or other information to be predicted, CNN model is trained using annotated data, model parameters are updated by back propagation algorithm, loss function (such as cross entropy loss) is used to measure the gap between prediction result and true label, and gradient descent optimization algorithm is applied to minimize loss function and improve model performance. Environment modeling is performed by SARSA algorithm and SARSA algorithm parameters are initialized, strategy is updated iteratively, and model training and optimization are performed by using SARSA algorithm through interaction with environment, and the optimal resource allocation strategy is learned, and the optimal strategy learned is applied to the actual medical resource allocation scenario, and its effect and performance are evaluated, so as to realize efficient management of hospital operation and optimal allocation of resources. This method not only improves patient satisfaction and reduces operating costs, but also balances multiple goals of hospital operation through flexible reward function design. In practical applications, the optimal strategy learned significantly improves the comprehensive performance of hospital decision-making scenarios, and provides strong technical support for improving the quality and efficiency of medical services. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The present invention is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0027] Figure 1 A schematic diagram of the method flow chart. DETAILED DESCRIPTION

[0028] The present invention will be described in detail below with reference to the accompanying drawings and in combination with embodiments. It should be noted that the embodiments and features in the embodiments of the present application can be combined with each other without conflict.

[0029] According to one aspect of the present invention, Figure 1 As shown, a medical decision-making assistance method based on convolutional neural network and SARSA algorithm is provided:

[0030] First, data collection and preprocessing are carried out. A large amount of medical imaging data is collected, including patient information, patient satisfaction, medical resource allocation, hospital operating costs, X-rays, CT, MRI, ultrasound, and images under different imaging modes. The original images are normalized, enhanced, denoised, and preprocessed for model learning. In the process of preprocessing the data, after obtaining the medical imaging data, the CNN model is used to preprocess the data through normalization. The pixel values ​​after processing are:

[0031] px new =(px-px min ) / (px max -pxmin )

[0032] Among them, px new is the normalized pixel value, px is the original pixel value, px min is the minimum value in the image dataset (usually 0 for 8-bit grayscale images), px max is the maximum value in the image dataset (usually 255 for 8-bit grayscale images).

[0033] Then, build a CNN model, select the appropriate network structure according to the task requirements, such as the classic LeNet, AlexNet, VGG, ResNet, U-Net, or improve or combine these basic models to adapt to specific tasks, define the convolution layer, pooling layer, fully connected layer and activation function key components, and determine the depth and width of the network. Finally, perform training and optimization, and extract features from medical imaging data. In the process of extracting features from processed medical images using the CNN model, the convolution kernel slides on the input feature map, and weighted sums the pixels at each position to obtain a new output value. The weight of the convolution kernel is obtained through training and learning, which can capture the local features in the input data. After the convolution operation, a nonlinear activation function (such as ReLU) is applied to transform the output feature map to increase the expressive power of the model. By stacking multiple convolutional layers, higher-level abstract features of the input data can be extracted layer by layer. Each layer increases the number of features (i.e., the number of channels) and captures more complex feature patterns, thereby annotating the medical imaging data, including the location of the lesion, the type of disease, or other information that needs to be predicted. The annotated data is used to train the CNN model, and the model parameters are updated through the back-propagation algorithm. The loss function (such as cross entropy loss) is used to measure the gap between the predicted result and the true label, and optimization algorithms such as gradient descent are applied to minimize the loss function and improve the model performance.

[0034] Then, the environment is modeled and the Markov decision process model of the medical resource allocation problem is defined, including the state space (such as current patient demand, department load, available resources), action space (such as the number of resources allocated to a specific department or patient), and reward function (reflecting the quality of resource allocation results, such as patient satisfaction, reduced waiting time, and improved resource utilization). The key information in hospital operations (such as bed usage, medical staff scheduling, and patient waiting time) is abstracted into the state space. Then, the hospital management system takes measures to adjust the medical staff scheduling and allocate medical resources to optimize hospital operations. After that, a suitable reward function is designed to evaluate the quality of each action to guide the algorithm to learn the optimal decision-making strategy. The reward function is set according to the hospital operation goals. The hospital operation goals of this application are: patient satisfaction and operating costs, and the reward function is designed accordingly:

[0035]

[0036] Among them, R t is the reward obtained at time step t, PS t is the patient satisfaction at time step t, PS e is the baseline value of patient satisfaction, which is the average satisfaction of hospital patients within one year, a is the preset weight coefficient of patient satisfaction, OC t is the operating cost of the hospital at time step t, OC e is the benchmark value of hospital operating costs, which is the average operating cost of the hospital in one year, and b is the preset weight coefficient of hospital operating costs. t Higher than the baseline PS e When the operating cost OC t Below the baseline OC e When , a positive reward will be obtained; otherwise, a negative reward will be obtained. By adjusting the values ​​of a and b, the hospital can flexibly balance the trade-off between patient satisfaction and operating costs. Secondly, initialize the SARSA algorithm parameters, set the learning rate α, discount factor γ and exploration rate ε, which are used to control the learning speed of the strategy, the importance of future rewards and the degree of random exploration; then, iteratively update the strategy, take action according to the current state at each time step t, and observe the next state and the corresponding reward, and update the Q-value function (i.e., state-action value function) based on this information; finally, train and optimize the model, and use the SARSA algorithm to train and optimize the model through interaction with the environment, learn the optimal resource allocation strategy, and apply the learned optimal strategy to the actual medical resource allocation scenario, evaluate its effect and performance, and apply the learned optimal strategy to the actual hospital decision-making scenario. The comprehensive performance index of the strategy is obtained through evaluation. If:

[0037]

[0038] If E>δ, it means that the optimal strategy learned is effective in actual hospital decision-making scenarios. On the contrary, if E>δ, it means that the optimal strategy learned is effective in actual hospital decision-making scenarios. E is the comprehensive performance index of the strategy, PS past is patient satisfaction before strategy optimization, PS new is the patient satisfaction after strategy optimization, a is the preset weight coefficient of patient satisfaction, OC past is the hospital operating cost before strategy optimization, OC newis the hospital operating cost after strategy optimization, b is the preset weight coefficient of hospital operating cost, δ is the preset threshold parameter of comprehensive performance of strategy, and its value is 0.7.

[0039] The above-mentioned embodiment only expresses one implementation mode of the present invention, and its description is relatively specific and detailed, but it cannot be understood as limiting the scope of the patent of the present invention. It should be pointed out that for ordinary technicians in this field, several modifications and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be based on the attached claims.

Claims

1. A medical decision-making assistance method based on convolutional neural network and SARSA algorithm, characterized in that: The following steps are involved: S1. Data preparation and feature extraction: Obtain hospital-related information data, pre-process the information data, and extract features from the processed information data; S2. CNN model training: Use the CNN model to train hospital data and learn the feature representation and pattern recognition capabilities of the data; S3, Environment Modeling: Model the hospital decision-making problem into a reinforcement learning environment, define the state space, action space and reward function; S4, state representation and action selection: represent the state in the hospital decision problem as the state of the environment and define the selectable actions to make decisions; S5. Reward design: Design a suitable reward function to evaluate the quality of each action to guide the algorithm to learn the optimal decision-making strategy; S6, Model training and optimization: Through interaction with the environment, the SARSA algorithm is used to train and optimize the model and learn the optimal decision-making strategy; S7. Decision application and evaluation: Apply the learned optimal strategy to actual hospital decision-making scenarios and evaluate its effectiveness and performance.

2. The medical decision-making assistance method based on convolutional neural network and SARSA algorithm according to claim 1, characterized in that: In S1, the hospital-related data includes: patient information, patient satisfaction, medical resource allocation, hospital operating costs and medical imaging data.

3. The medical decision-making assistance method based on convolutional neural network and SARSA algorithm according to claim 1, characterized in that: In S1, after obtaining the medical image data, the CNN model is used to pre-process the data by normalization, and the pixel value after processing is: px new =(px-px min ) / (px max -px min ) Among them, px new is the normalized pixel value, px is the original pixel value, px min is the minimum value in the image dataset, px max is the maximum value in the image dataset.

4. The medical decision-making assistance method based on convolutional neural network and SARSA algorithm according to claim 3, characterized in that: In S1, after the data is preprocessed, the CNN model is used to extract features from the processed medical images. The process is as follows: the convolution kernel slides on the input feature map, and the pixels at each position are weighted and summed to obtain a new output value. The weight of the convolution kernel is obtained through training and learning, and can capture local features in the input data. After the convolution operation, a nonlinear activation function is applied to transform the output feature map to increase the expressive power of the model. By stacking multiple convolutional layers, higher-level abstract features of the input data can be extracted layer by layer. Each layer increases the number of features and captures more complex feature patterns.

5. The medical decision-making assistance method based on convolutional neural network and SARSA algorithm according to claim 1, characterized in that: In S3, in the process of modeling the environment, the state space is: current patient demand, department load and available resources; the action space is: the amount of resources allocated to a specific department or patient; and the reward function is: reflecting the quality of resource allocation results.

6. The medical decision-making assistance method based on convolutional neural network and SARSA algorithm according to claim 1, characterized in that: In S3, the hospital decision-making problem is modeled into a reinforcement learning environment, and the key information in the hospital operation is abstracted into a state space. Then, the hospital management system takes measures to adjust the medical staff schedule and allocate medical resources to optimize the hospital operation.

7. The medical decision-making assistance method based on convolutional neural network and SARSA algorithm according to claim 1, characterized in that: In S5, the reward function is set according to the hospital operation objectives. The hospital operation objectives of this application are: patient satisfaction and operation cost, and the reward function is designed accordingly: Among them, R t is the reward obtained at time step t, PS t is the patient satisfaction at time step t, PS e is the baseline value of patient satisfaction, which is the average satisfaction of hospital patients within one year, a is the preset weight coefficient of patient satisfaction, OC t is the operating cost of the hospital at time step t, OC e is the benchmark value of hospital operating costs, which is the average operating cost of the hospital in one year, and b is the preset weight coefficient of hospital operating costs.

8. The medical decision-making assistance method based on convolutional neural network and SARSA algorithm according to claim 1, characterized in that: In S7, the learned optimal strategy is applied to the actual hospital decision-making scenario, and the comprehensive performance index of the strategy is obtained through evaluation. If: This indicates that the optimal strategy learned is effective in actual hospital decision-making scenarios, where E is the comprehensive performance index of the strategy, PS past is patient satisfaction before strategy optimization, PS new is the patient satisfaction after strategy optimization, a is the preset weight coefficient of patient satisfaction, OC past is the hospital operating cost before strategy optimization, OC new is the hospital operating cost after strategy optimization, b is the preset weight coefficient of hospital operating cost, and δ is the preset threshold parameter of comprehensive performance of strategy.

Images