Wound treatment sequence recommendation method and device based on reinforcement learning and medium

Abstract

The invention belongs to the technical field of medical detection, and discloses a wound treatment sequence recommendation method and device based on reinforcement learning and a medium, and the method comprises the steps: constructing a wound decision knowledge base, carrying out the unified coding of multi-modal wound data, mapping the data to a same dimensional space, and generating a feature vector; constructing a reinforcement learning model, training the reinforcement learning model by using the historical trauma data, and optimizing a decision strategy of the reinforcement learning model through composite reward calculation and adversarial training; identifying and processing the trauma image to complete trauma severity evaluation; and according to the evaluation result, generating a treatment priority recommendation, and feeding back the wound evaluation result and the treatment priority in real time. By introducing the reinforcement learning model, the wound type and severity of the patient are automatically evaluated, and the personalized treatment priority is generated based on the wound type and severity. According to the automatic decision-making process based on data driving, subjectivity and delay of manual judgment are avoided, and the wound treatment efficiency is remarkably improved.

Description

Technical Field

[0001] The present invention belongs to the field of medical detection technology, and in particular, relates to a method, device and medium for recommending trauma treatment sequence based on reinforcement learning. Background Art

[0002] With the development of medical imaging technology, the diagnosis and treatment process of trauma patients has been significantly improved. However, in emergency situations, trauma assessment and treatment priority determination still rely on manual judgment, which is easily affected by factors such as doctor experience and emotions, leading to delayed treatment decisions or misjudgment of priorities. In addition, existing trauma assessment systems are mostly static rule-based decisions that lack flexibility and adaptability, and cannot effectively respond to the diverse trauma conditions of different patients, and cannot dynamically adjust treatment plans according to actual conditions.

[0003] Based on this, there is an urgent need for an innovative method that can automatically assess the type and severity of trauma based on trauma images, patient conditions, and clinical data, and intelligently recommend treatment priorities, so as to improve treatment efficiency, reduce medical errors, and provide intelligent decision-making support for emergency doctors. Summary of the invention

[0004] In view of this, the present invention provides a method, device and medium for recommending trauma treatment sequence based on reinforcement learning to solve the above problems.

[0005] In order to solve the above technical problems, the present invention provides a method for recommending trauma treatment sequence based on reinforcement learning, comprising: Construct a trauma decision-making knowledge base, uniformly encode multimodal trauma data and map them to the same dimensional space to generate feature vectors that meet the input requirements of the reinforcement learning model; Build a reinforcement learning model, use historical trauma data to train the reinforcement learning model, and optimize its decision-making strategy through compound reward calculation and adversarial training; Convolutional neural networks are used to identify and process trauma images to complete trauma severity assessments. Based on the assessment results, treatment priority recommendations are generated, and trauma assessment results and treatment priorities are provided in real time.

[0006] As an optional method, constructing a trauma decision-making knowledge base includes: organizing the correspondence between trauma types and complications, establishing a database of golden treatment times for different injuries, and recording the imaging patterns of organ damage on human signs; among them, the results of the correspondence between trauma types and complications are also used to initialize the Q value table of the reinforcement learning model.

[0007] As an optional approach, the trauma decision knowledge base is also used to embed into the fusion layer as a standardized mapping framework for multimodal trauma data, and the fusion layer is used to map the multimodal trauma data into a joint feature space that can be processed by reinforcement learning.

[0008] As an optional approach, building a reinforcement learning model involves: Define the state space, fill in the patient's imaging data, vital signs, historical medical history and trauma type, and set the organ function damage entropy value as a constraint; Define the action space and pre-input the output result types of the reinforcement learning model, which include high-priority treatment, medium-priority treatment, and low-priority treatment; and set the golden treatment time and treatment plan cost as constraints; Design a reward function, which includes basic priority reward, dynamic medical reward and risk penalty.

[0009] As an optional approach, the basic priority reward parameters of the reward function include the standard priority and emergency trauma coefficient preset by the ATLS guidelines; the dynamic medical reward parameters include the complication probability and trauma entropy value based on the trauma decision knowledge base; and the risk penalty includes the idle state of medical personnel and medical equipment.

[0010] As an optional method, compound rewards include basic rewards and additional rewards. The basic rewards include whether the treatment priority selected based on the current multimodal trauma data is correct; the additional rewards include early intervention bonuses for high-risk complications, rewards for controlling the golden treatment time window, and rewards for optimizing strategies for multi-organ coordinated treatment.

[0011] As an optional approach, adversarial training includes setting up a verification system and expert review to provide feedback on the output of the reinforcement learning model; The verification system includes a case library storing typical treatment cases and an anatomical and / or biological constraint rule logic library, which optimizes the decision quality by confronting the output results of the reinforcement learning model; Expert review is used to compare the consistency of the output decision with the standard treatment guidelines and adjust the weights of the reward function.

[0012] As an optional method, real-time feedback of trauma assessment results and treatment priorities is also provided, including: A dynamic monitoring mechanism for multimodal trauma data is set up to detect the time-varying characteristics of the biochemical indicators of any vital sign of the patient; when the change rate of the time-varying characteristics of the biochemical indicators of any vital sign exceeds a preset threshold, the reward function parameters of the reinforcement learning model are dynamically adjusted.

[0013] On the other hand, the present invention also provides an electronic device, comprising: a memory for storing a computer program; and a processor for implementing the steps of any of the above-mentioned methods for recommending a trauma treatment sequence based on reinforcement learning when executing the computer program.

[0014] On the other hand, the present invention also provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned method for recommending a trauma treatment sequence based on reinforcement learning are implemented.

[0015] The beneficial effects of the present invention are: By introducing a reinforcement learning model, the present invention enables the system to automatically assess the type and severity of a patient's trauma, and generate personalized treatment priorities based on this. This data-driven automated decision-making process avoids the subjectivity and delays of manual judgment, thereby significantly improving the efficiency of trauma treatment. The system can update priorities in real time to ensure that critically ill patients receive timely treatment and optimize the allocation of overall resources. Compared with traditional manual assessment methods, automated trauma assessment not only improves accuracy, but can also process large amounts of patient data in real time, identify potential critical symptoms, and reduce the risk of misdiagnosis or missed diagnosis. This is especially important for trauma patients in the emergency department, and can effectively reduce the waste of medical resources and improve treatment outcomes. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 A schematic diagram of the steps of a method for recommending trauma pre-examination and treatment sequence provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0017] In order to enable those skilled in the art to better understand the technical solution of the present invention, the present invention is further described in detail below in conjunction with specific implementation methods.

[0018] See also Figure 1 The present embodiment provides a method for recommending trauma treatment order based on reinforcement learning, including: constructing a trauma decision knowledge base, uniformly encoding multimodal trauma data and mapping them to the same dimensional space, generating a feature vector that meets the input requirements of the reinforcement learning model; constructing a reinforcement learning model, training the reinforcement learning model with historical trauma data, and optimizing its decision-making strategy through compound reward calculation and adversarial training; using a convolutional neural network to identify and process trauma images to complete trauma severity assessment; generating treatment priority recommendations based on the assessment results, and providing real-time feedback on the trauma assessment results and treatment priorities.

[0019] Thus, through the above scheme, this embodiment can automatically analyze trauma images, evaluate the type and severity of trauma, and dynamically recommend treatment priorities based on the patient's trauma condition. This method solves the accuracy, efficiency and flexibility problems in traditional manual evaluation and static decision-making methods by continuously optimizing learning and adapting to different trauma situations. In this embodiment, this embodiment is implemented as follows: First, a trauma decision-making knowledge base is constructed, the correspondence between trauma types and complications is sorted out, a database of golden treatment times for different injuries is established, and the imaging patterns of organ damage on human signs are recorded. As an option, the correspondence between trauma types and complications is used to construct the probability of complications. This relationship is based on industry medical data and past historical data. For example, if a patient suffers from craniocerebral trauma, the probability of brain herniation is about 46%, so the optimal intervention window is within two hours; if a patient suffers from a penetrating chest injury, the incidence of hemothorax is about 78%, so the golden treatment time is within one hour. In addition, the resulting probability value of the correspondence between trauma types and complications can also be used to initialize the Q value table of the reinforcement learning model.

[0020] In this embodiment, the trauma decision knowledge base is also used to be embedded in the fusion layer as a standardized mapping framework for multimodal trauma data. Multimodal trauma data includes imaging data of trauma patients, physiological data of patients, personal conditions and medical history, etc. In this way, the fusion layer will uniformly encode the multimodal trauma data and map it to the same dimensional space, so as to generate a feature vector that meets the input requirements of the reinforcement learning model. In addition to completing the mapping of trauma types and complications and using the trauma decision knowledge base as the basis for constructing the state space of subsequent models, this step can also use the medical knowledge graph as a decision constraint to avoid the model output violating medical common sense; it can also provide a priori decision framework to greatly shorten the model training convergence time.

[0021] After that, the trauma imaging data is collected and preprocessed. The system obtains imaging data of trauma patients through CT scans, X-rays, ultrasound equipment, etc. The imaging data may include a single image or multiple images. The data is preprocessed, including normalization, denoising and size unification, to improve the quality of the model's input parameters. After the data preprocessing is completed, the reinforcement learning model training and optimization are started.

[0022] Before training the reinforcement learning model, the state space is first defined, which contains the patient's imaging data, the patient's vital signs, historical medical history and other information. Each state reflects the current assessment of the trauma. Then the action space is defined, which includes various treatment priority decisions that the system may take. For example, high priority treatment, medium priority treatment and low priority treatment can be optionally set as different actions. Finally, a reward function is designed, and the reward function is designed according to the treatment effect. The reward function described in this embodiment includes basic priority rewards, dynamic medical rewards and risk penalties.

[0023] Among them, the basic priority reward parameters of the reward function include the standard priority and emergency trauma coefficient preset by the ATLS guidelines; the dynamic medical reward parameters include the complication probability and trauma entropy value based on the trauma decision knowledge base; the risk penalty includes the idle state of medical personnel and medical equipment. Optionally, the basic priority reward is confirmed by constructing an association between the emergency trauma coefficient (such as 1.5 for penetrating injuries and 1.0 for blunt injuries) and the predicted priority and standard priority difference; the dynamic medical reward can be confirmed by calculating the association between the complication probability and the organ function damage rate and the golden treatment time; the risk penalty can adopt a double constraint and be adjusted through parameters such as the investment ratio of medical costs and the department occupancy rate. Since hospitals in different regions and at different levels have different emergency needs, the relevant weights can be set according to the actual situation, and this embodiment will not be repeated.

[0024] As an optional method, this embodiment uses the Q-learning algorithm and the deep Q network to train the reinforcement learning model, and iterates the optimal strategy by updating the Q value; after repeated training and iteration, the action with the largest Q value in the action space is selected as the optimal strategy.

[0025] The reinforcement learning model is trained using historical trauma data. The model continuously optimizes its decision-making strategy based on reward feedback through interaction with the environment (trauma assessment and treatment process), and learns how to make optimal priority recommendations based on trauma type, severity, and other clinical parameters. This embodiment uses the Q-learning algorithm to learn the optimal strategy by updating the Q value. And through repeated training and updating, the strategy is continuously adjusted so that the most appropriate treatment priority can be automatically generated in different trauma situations to ensure that the most critical trauma is treated first. After training, the action that maximizes the Q value is selected as the optimal strategy.

[0026] In addition, in addition to the self-learning of the model, this embodiment is also provided with compound reward calculation and adversarial training to optimize its decision-making strategy. In this embodiment, the compound reward includes basic rewards and additional rewards. The basic reward includes whether the treatment priority selected according to the current multimodal trauma data is correct; the additional reward includes early intervention bonus for high-risk complications, control reward for the golden treatment time window, and optimization strategy reward for multi-organ collaborative treatment. Adversarial training includes setting up a verification system and expert review to provide feedback on the output results of the reinforcement learning model; the verification system includes a case library storing typical treatment cases and an anatomical and / or biological constraint rule logic library, which optimizes the decision quality by confronting the output results of the reinforcement learning model; the expert review is used to compare the consistency of the output decision with the standard treatment guidelines and adjust the weight of the reward function.

[0027] After the model is trained and optimized through the above scheme to achieve the expected output effect, it will be put into use on site. During the training process of the reinforcement learning model, it is necessary to learn how to identify different types of trauma. This implementation uses a convolutional neural network (CNN) to automatically identify trauma types such as fractures, soft tissue injuries, and internal organ injuries. After that, a trauma severity assessment is performed to automatically assess the severity of the trauma based on the type of trauma and the degree of impact on the trauma area. Severe trauma, such as internal organ damage and extensive bleeding, will be marked as high priority for treatment. Based on the trauma assessment function, the severity of the trauma is assessed based on the type of trauma and the degree of injury in the trauma area (such as the amount of bleeding and the size of the trauma).

[0028] Based on the above-mentioned trauma type and other attribute recognition results, the priority of treatment is evaluated, and the priority of treatment is automatically evaluated and recommended based on the trauma type, severity, location of the trauma area, patient vital signs and other clinical data. The reinforcement learning model of this embodiment determines the adjustment of priority based on the reward feedback of each trauma scenario. For example, visceral bleeding may require priority treatment than simple fractures.

[0029] Among them, the reinforcement learning model of this embodiment is provided with a public time tag, which is used to drive the reinforcement learning model to adjust the priority increment of treatment in real time according to the current patient's condition. As the patient's condition changes, the priority can be updated and adjusted dynamically in real time. When visceral injury causes massive bleeding, the system will automatically increase the priority of treatment of visceral injury until the bleeding is controlled.

[0030] In this embodiment, a dynamic monitoring mechanism of multimodal trauma data is set to detect the time-varying characteristics of the biochemical indicators of any vital sign of the patient; when the change rate of the time-varying characteristics of the biochemical indicators of any vital sign exceeds a preset threshold, for example, when the heartbeat, pulse, blood pressure, etc. fluctuate violently, it indicates an inappropriate rescue logic, and the reward function parameters of the reinforcement learning model are dynamically adjusted at this time.

[0031] Finally, the method described in this embodiment will also evaluate the patient's trauma in real time, automatically generate a trauma assessment report, and provide doctors with accurate assessments of trauma type and severity. The system generates a trauma assessment report in real time based on the patient's trauma images and other clinical data, including information such as trauma type and severity: Based on the real-time assessment results, the priority of trauma treatment can be recommended to emergency doctors to ensure that the trauma that needs the most treatment receives priority attention. Based on the trauma assessment report, the system recommends trauma treatment priorities to doctors.

[0032] After completing the above-mentioned treatment priority feedback, the doctor can start trauma treatment according to the recommended treatment priority. The method described in this embodiment can adjust the treatment plan according to real-time feedback and continuously monitor the patient's vital signs and treatment effects. According to the recommended priority, the doctor starts to treat the trauma. The trauma assessment results and priority recommendations are shared with experts through the remote consultation platform. Doctors and experts make collaborative decisions based on the shared trauma data to optimize the treatment plan.

[0033] In this way, by dynamically adjusting the treatment strategy, the trauma treatment plan is always targeted at the patient's most urgent needs, thereby maximizing the success rate of treatment. The system dynamically optimizes the treatment plan based on real-time feedback during the treatment process.

[0034] On the other hand, the present embodiment also provides a trauma pre-examination and treatment sequence recommendation system based on reinforcement learning, including: an image acquisition module, used to collect trauma image data, establish an index identifier for each trauma image, and perform input preprocessing; a model training module, used to build a reinforcement learning model, define the state space, action space and reward function; use historical trauma data to train the reinforcement learning model, and optimize its decision-making strategy based on reward feedback; an identification and evaluation module, used to use a convolutional neural network to identify and process trauma images, and perform trauma severity assessment; based on the evaluation results, generate treatment priority recommendations, and provide real-time feedback on the trauma assessment results and treatment priorities.

[0035] Based on the above scheme, this embodiment automatically evaluates the type and severity of the patient's trauma by introducing a reinforcement learning model, and generates personalized treatment priorities based on this to replace manual trauma pre-examination and treatment priority recommendations. This data-driven automated decision-making process avoids the subjectivity and delays of manual judgment, thereby significantly improving the efficiency of trauma treatment. The most suitable treatment priority is automatically recommended through a reinforcement learning algorithm. Compared with traditional manual assessment methods, automated trauma assessment not only improves accuracy, but also can process large amounts of patient data in real time, identify potential critical symptoms, and reduce the risk of misdiagnosis or missed diagnosis. This is especially important for trauma patients in the emergency department, which can effectively reduce the waste of medical resources and improve treatment outcomes.

[0036] On the other hand, this embodiment further provides an electronic device, comprising: a memory for storing a computer program; and a processor for implementing the steps of any of the above-mentioned methods for recommending a trauma treatment sequence based on reinforcement learning when executing the computer program.

[0037] On the other hand, this embodiment further provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned method for recommending a trauma treatment sequence based on reinforcement learning are implemented.

[0038] The above are only preferred embodiments of the present invention. It should be noted that the above preferred embodiments should not be regarded as limiting the present invention, and the protection scope of the present invention should be based on the scope defined by the claims. For ordinary technicians in this technical field, several improvements and modifications can be made without departing from the spirit and scope of the present invention, and these improvements and modifications should also be regarded as the protection scope of the present invention.

Claims

1. A method for recommending trauma treatment sequence based on reinforcement learning, characterized in that: include: Construct a trauma decision-making knowledge base, uniformly encode multimodal trauma data and map them to the same dimensional space to generate feature vectors that meet the input requirements of the reinforcement learning model; Build a reinforcement learning model, use historical trauma data to train the reinforcement learning model, and optimize its decision-making strategy through compound reward calculation and adversarial training; Convolutional neural networks are used to identify and process trauma images to complete trauma severity assessments. Based on the assessment results, treatment priority recommendations are generated, and trauma assessment results and treatment priorities are provided in real time.

2. The method for recommending trauma treatment sequence based on reinforcement learning according to claim 1, characterized in that: The construction of the trauma decision-making knowledge base includes: organizing the correspondence between trauma types and complications, establishing a database of golden treatment times for different injuries, and recording the imaging laws of organ damage on human signs; wherein the results of the correspondence between trauma types and complications are also used to initialize the Q value table of the reinforcement learning model.

3. The method for recommending trauma treatment sequence based on reinforcement learning according to claim 1, characterized in that: The trauma decision knowledge base is also used to be embedded in a fusion layer as a standardized mapping framework for multimodal trauma data, and the fusion layer is used to map the multimodal trauma data to a joint feature space that can be processed by reinforcement learning.

4. The method for recommending trauma treatment sequence based on reinforcement learning according to claim 1, characterized in that: The construction of the reinforcement learning model includes: Define the state space, fill in the patient's imaging data, vital signs, historical medical history and trauma type, and set the organ function damage entropy value as a constraint; Define an action space, pre-input the output result type of the reinforcement learning model, the output result type includes high priority treatment, medium priority treatment and low priority treatment; and set the golden treatment time and treatment plan cost as constraints; A reward function is designed, which includes a basic priority reward, a dynamic medical reward, and a risk penalty.

5. The method for recommending trauma treatment sequence based on reinforcement learning according to claim 4, characterized in that: The basic priority reward parameters of the reward function include the standard priority and emergency trauma coefficient preset in the ATLS guidelines; the dynamic medical reward parameters include the complication probability and trauma entropy value based on the trauma decision knowledge base; the risk penalty includes the idle state of medical personnel and medical equipment.

6. The method for recommending trauma treatment sequence based on reinforcement learning according to claim 1, characterized in that: The composite reward includes a basic reward and additional rewards. The basic reward includes whether the treatment priority selected based on the current multimodal trauma data is correct; the additional reward includes an early intervention bonus for high-risk complications, a reward for controlling the golden treatment time window, and a reward for optimizing strategies for multi-organ coordinated treatment.

7. The method for recommending trauma treatment sequence based on reinforcement learning according to claim 1, characterized in that: The adversarial training includes setting up a verification system and expert review to provide feedback on the output results of the reinforcement learning model; The verification system includes a case library storing typical treatment cases and an anatomical and / or biological constraint rule logic library, which optimizes the decision quality by confronting the output results of the reinforcement learning model; The expert review is used to compare the consistency of the output decision with the standard treatment guidelines and adjust the weight of the reward function.

8. The method for recommending trauma treatment sequence based on reinforcement learning according to claim 1, characterized in that: The real-time feedback of trauma assessment results and treatment priorities also includes: A dynamic monitoring mechanism for multimodal trauma data is set up to detect the time-varying characteristics of the biochemical indicators of any vital sign of the patient; when the change rate of the time-varying characteristics of the biochemical indicators of any vital sign exceeds a preset threshold, the reward function parameters of the reinforcement learning model are dynamically adjusted.

9. An electronic device, characterized in that: include: Memory for storing computer programs; A processor is used to implement the steps of a method for recommending a trauma treatment sequence based on reinforcement learning as described in any one of claims 1 to 8 when executing the computer program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of a method for recommending a trauma treatment sequence based on reinforcement learning as described in any one of claims 1 to 8 are implemented.

Images