Reinforcement learning-based clinical decision support device and method

Abstract

The application provides a clinical decision support method and device based on reinforcement learning. The method comprises: a device receiving medical information associated with a user. Based on the medical information associated with the user and a reinforcement learning model, inquiry information is determined. The inquiry information is provided to allow response information to be received. The response information is received based on the provision of the inquiry information. Using a machine learning model, based on the medical information and the response information, diagnostic information is determined. The diagnostic information is provided to a group of devices through a network.

Description

[0001] This application claims priority to U.S. Application No. 16 / 409,293, filed May 10, 2019, the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the medical field, and in particular to a clinical decision support device and method based on reinforcement learning. Background Technology

[0003] With the introduction of electronic health records, additional digital data has become available for analysis and decision support. Therefore, when physicians diagnose patients, they need to consider and evaluate a wide range of vast amounts of data, making clinical decision-making increasingly complex. Machine learning-based clinical decision support systems can offer solutions to this data challenge.

[0004] A major challenge for clinical decision support systems is long-term predictability. In the early stages of many diseases, symptoms that may be very common, such as fever and skin redness, are present. However, even routine clinical tests may fail to detect these symptoms. In such cases, some existing clinical decision support systems may offer recommendations corresponding to common, minor illnesses, leading to misdiagnosis.

[0005] Current clinical decision support systems often lack foresight, passively receiving user information and offering recommendations based solely on known observations. However, due to oversight, some informative features may go undetected. Summary of the Invention

[0006] According to an embodiment of this application, a clinical decision support method based on reinforcement learning includes: receiving medical information associated with a user; the device determining inquiry information based on the medical information associated with the user and a reinforcement learning model; providing the inquiry information to allow receiving response information; receiving the response information based on providing the inquiry information; determining diagnostic information using a machine learning model based on the medical information and the response information; and providing the diagnostic information to a group of devices via a network.

[0007] This application provides a reinforcement learning-based clinical decision support device, comprising: a first receiving module for receiving medical information associated with a user; a first determining module for determining inquiry information based on the user-associated medical information and a reinforcement learning model; a first providing module for providing the inquiry information to allow receiving response information; a second receiving module for receiving the response information based on the provided inquiry information; a second determining module for determining diagnostic information using a machine learning model based on the medical information and the response information; and a second providing module for providing standardized format diagnostic information to a group of devices in real time via a network.

[0008] According to an embodiment of this application, a device includes: at least one memory for storing program code; and at least one processor for reading the program code and operating according to the instructions of the program code, the program code including: first receiving code for causing the at least one processor to receive medical information associated with a user; first determining code for causing the at least one processor to determine inquiry information based on the medical information associated with the user and a reinforcement learning model; providing code for causing the at least one processor to provide the inquiry information to allow receiving response information; second receiving code for causing the at least one processor to receive the response information based on providing the inquiry information; second determining code for causing the at least one processor to determine diagnostic information based on the medical information and the response information using a machine learning model; and providing code for causing the at least one processor to provide the diagnostic information to a group of devices via a network.

[0009] According to an embodiment of this application, a non-volatile computer-readable storage medium stores instructions, the instructions including: one or more instructions that, when executed by one or more processors of a device, cause the one or more processors to: receive medical information associated with a user; determine query information based on the medical information associated with the user and a reinforcement learning model; provide the query information to allow receiving response information; receive the response information based on providing the query information; determine diagnostic information using a machine learning model based on the medical information and the response information; and provide the diagnostic information to a group of devices via a network.

[0010] The reinforcement learning-based clinical decision support method, device, and non-volatile computer-readable storage medium of this application embodiment can determine the final decision based on raw information and query-based information. This application embodiment can discover unobserved clinical information, provide doctors with accurate clinical advice, reduce the risk of human error and the workload of medical staff, improve diagnostic accuracy, and reduce the time required for decision-making. Furthermore, this application embodiment can provide users with comprehensive information, improve the long-term predictive ability of the clinical decision support system, and enhance medication safety and efficiency. Finally, this application embodiment can transmit messages in a standardized format via a network to all medical staff and / or other users who have access to patient information. In this way, any changes can be quickly communicated to all users without requiring users to manually search for such information. Attached Figure Description

[0011] Figure 1 This is a schematic diagram summarizing the exemplary embodiments described in this application;

[0012] Figure 2 This is a schematic diagram of an example environment in which the systems and / or methods described in this application can be implemented;

[0013] Figure 3 yes Figure 2 A schematic diagram of example components for one or more devices;

[0014] Figure 4 This is a flowchart of an example process for determining diagnostic information using a reinforcement learning model;

[0015] Figure 5a , 5b This is a schematic diagram of a reinforcement learning-based clinical decision support device in an embodiment of this application. Detailed Implementation

[0016] A reliable clinical decision support system should proactively guide users to identify as much informative information as possible. This application provides novel techniques, including reinforcement learning and heterogeneity learning, to improve the long-term predictive ability of clinical decision support systems.

[0017] This application utilizes reinforcement learning techniques to identify potential, unobserved clinical information and provide accurate clinical recommendations. For example, it employs a reinforcement learning-based algorithm that, based on currently observed clinical presentations, determines which patient information should be confirmed to provide the most informative and valuable recommendations. The reward function within the reinforcement learning module is designed based on various clinical data formats, including electronic medical records (EMRs) / electronic health records (EHRs), rules, and / or other types of knowledge bases. Therefore, this application can be widely applied to various types of clinical recommendation tasks.

[0018] Figure 1 This is a schematic diagram summarizing the embodiments described in this application. For example... Figure 1 As shown by reference numeral 110, the information understanding module can detect valuable information from clinical data in text format. For example, in a patient's medical description, especially in their previous medical history and symptoms, there are important clues that can be used for the analysis of cardiac abnormalities. Therefore, the information understanding module identifies this informative information from the medical record information. The information understanding module includes components such as named entity recognition (NER) and semantic role labeling.

[0019] like Figure 1 As further illustrated by reference numeral 120, the potential problem collection module is designed to extract appropriate questions that a physician may ask to interact with a patient or that prompt action based on currently known clinical observations. For example, these questions could be about the current illness, medical history, the patient's medication history, the patient's test results that the physician should assess, and so on. These questions can cover any behavior or information within the clinical diagnostic setting.

[0020] like Figure 1 As further illustrated by reference numeral 130, after collecting potential questions, a question determination module estimates the importance of each question and, based on this importance, determines which question to ask. This estimation is based on reinforcement learning, where a reward function scores the importance of each question. This reward function is trained on a large amount of EMR / HER data or is based on a knowledge base or rule definition. For example, a question that is not clarified may lead to terrible consequences such as death or long-term effects, thus its importance is high. Finally, by considering known information, the impact of the conversation, and input from healthcare professionals, the system determines the final question to ask.

[0021] like Figure 1 As further shown in reference numeral 140, the system uses a machine learning module to provide a final decision to the user based on known observations learned from the text or known observations determined during the inquiry process.

[0022] The training framework proposed in this application is designed as an end-to-end framework. Compared with other clinical decision support systems, this framework can learn and extract information from both original descriptions and information collected through questioning. In this way, this application determines the final decision based on both original information and question-based information, rather than passively waiting for input like traditional clinical decision support systems. Furthermore, this framework can collect different types of clinical questions (e.g., current disease, family history, previous diseases, etc.) and provide different types of suggestions (e.g., test, examination, medication recommendations, etc.), providing users with comprehensive information. Finally, this framework can transmit messages in a standardized format over a network to all healthcare professionals and / or other users who have access to patient information. In this way, any changes can be quickly communicated to all users without requiring users to manually search for such information.

[0023] In other implementations, the information understanding module utilizes various machine learning algorithms, such as recurrent neural networks (RNNs), convolutional neural networks (CNNs), and support vector machines (SVMs). Furthermore, the reward function in the problem-determination module is designed based on EHR / EMR, knowledge bases, and rules.

[0024] Furthermore, the framework of this application is designed as an end-to-end process, with the entire framework being optimized and modified simultaneously. In an alternative embodiment, the framework includes a stepwise training process, where each module can be trained independently.

[0025] Figure 2 This is a schematic diagram of an example environment 200 in which the systems and / or methods described in this application can be implemented. For example... Figure 2 As shown, environment 200 may include user equipment 210, platform 220, and network 230. Devices in environment 200 can be interconnected via wired connections, wireless connections, or a combination of wired and wireless connections.

[0026] User equipment 210 includes one or more devices capable of receiving, generating, storing, processing, and / or providing information related to platform 220. For example, user equipment 210 may include computing devices (e.g., desktop computers, laptop computers, tablet computers, handheld computers, smart speakers, servers, etc.), mobile phones (e.g., smartphones, cordless phones, etc.), wearable devices (e.g., smart glasses or smartwatches), or similar devices. In some embodiments, user equipment 210 may receive information from and / or send information to platform 220.

[0027] Platform 220 includes one or more devices as described elsewhere in this application, capable of determining diagnostic information using a reinforcement learning model. In some embodiments, platform 220 may include a cloud server or a group of cloud servers. In some embodiments, platform 220 may be designed to be modular so that certain software components can be swapped in or out as needed. This allows platform 220 to be easily and / or quickly reconfigured for different uses.

[0028] In some implementations, such as Figure 2 As shown, platform 220 can be hosted in cloud computing environment 222. In particular, although the embodiments described in this application depict platform 220 as hosted in cloud computing environment 222, in some embodiments, platform 220 is not cloud-based (i.e., it can be implemented outside of a cloud computing environment), or it can be partially cloud-based.

[0029] The cloud computing environment 222 includes the environment of the hosting platform 220. The cloud computing environment 222 can provide services such as computing, software, data access, and storage, without requiring the end user (e.g., user device 210) to know the physical location and configuration of the systems and / or devices of the hosting platform 220. Figure 2 As shown, the cloud computing environment 222 may include a set of computing resources 224 (collectively referred to as “computing resources 224” and individually referred to as “computing resources 224”).

[0030] Computing resource 224 includes one or more personal computers, workstations, server devices, or other types of computing and / or communication devices. In some embodiments, computing resource 224 may host platform 220. Cloud resources may include: computing instances executing in computing resource 224, storage devices provided in computing resource 224, data transmission devices provided by computing resource 224, etc. In some embodiments, computing resource 224 may communicate with other computing resources 224 via wired connections, wireless connections, or a combination of wired and wireless connections.

[0031] like Figure 2 As further shown, computing resources 224 include a set of cloud resources, such as one or more applications (“APP”) 224-1, one or more virtual machines (“VM”) 224-2, virtualized storage (“VS”) 224-3, one or more hypervisors (“HYP”) 224-4, etc.

[0032] Application 224-1 includes one or more software applications that may be provided to user device 210 and / or sensor device 220, or accessed by user device and / or platform. Application 224-1 can eliminate the need to install and execute software applications on user device 210. For example, application 224-1 may include software associated with platform 220 and / or any other software that can be provided via cloud computing environment 222. In some implementations, an application 224-1 may send / receive information to / from one or more other applications 224-1 via virtual machine 224-2.

[0033] Virtual machine 224-2 includes a software implementation of a machine (e.g., a computer) that executes programs, similar to a physical machine. Virtual machine 224-2 can be a system virtual machine or a process virtual machine, depending on the extent to which virtual machine 224-2 uses and corresponds to any real machine. A system virtual machine can provide a complete system platform, supporting the execution of a complete operating system (“OS”). A process virtual machine can execute a single program and can support a single process. In some implementations, virtual machine 224-2 can execute on behalf of a user (e.g., user device 210) and can manage the infrastructure of cloud computing environment 222, such as data management, synchronization, or long-term data transfer.

[0034] Virtualized storage 224-3 includes one or more storage systems and / or one or more devices that utilize virtualization technology within the storage system or device of computing resource 224. In some embodiments, the type of virtualization, in the context of the storage system, may include block virtualization and file virtualization. Block virtualization may refer to abstracting (or separating) logical memory from physical memory, enabling access to the storage system without regard to physical memory or heterogeneous architectures. This separation allows storage system administrators flexibility in how they manage end-user storage. File virtualization can eliminate the dependency between data accessed at the file level and the physical storage location of the file. This optimizes memory usage, server consolidation, and / or the performance of uninterrupted file migration.

[0035] Hypervisor 224-4 can provide hardware virtualization technology, which allows multiple operating systems (e.g., "guest operating systems") to run simultaneously on a host computer, such as computing resource 224. Hypervisor 224-4 can provide a virtual operating platform for guest operating systems and manage the execution of guest operating systems. Multiple instances of various operating systems can share virtualization hardware resources.

[0036] Network 230 includes one or more wired and / or wireless networks. For example, network 230 may include cellular networks (e.g., fifth-generation (5G) networks, long-term evolution (LTE) networks, third-generation (3G) networks, code division multiple access (CDMA) networks, etc.), public land mobile networks (PLMNs), local area networks (LANs), wide area networks (WANs), metropolitan area networks (MANs), telephone networks (e.g., public switched telephone networks (PSTNs)), private networks, hybrid networks, intranets, the Internet, fiber-optic networks, etc., and / or combinations of these or other types of networks.

[0037] Figure 2 The number and arrangement of devices and networks shown are provided as an example. In reality, with... Figure 2 Compared to the equipment and / or network shown, there may be additional equipment and / or networks, fewer equipment and / or networks, different equipment and / or networks, or equipment and / or networks with different arrangements. Furthermore, Figure 2 The two or more devices shown can be implemented within a single device, or Figure 2 The single device shown can be implemented as multiple distributed devices. Alternatively, a group of devices in environment 200 (e.g., one or more devices) can perform one or more functions described as being performed by another group of devices in environment 200.

[0038] Figure 3 This is a schematic diagram of example components of device 300. Device 300 may correspond to user device 210 and / or platform 220. Figure 3 As shown, device 300 may include bus 310, processor 320, memory 330, storage component 340, input component 350, output component 360 and communication interface 370.

[0039] Bus 310 includes components that allow communication between components of device 300. Processor 320 is implemented in hardware, firmware, or a combination of hardware and software. Processor 320 is a central processing unit (CPU), graphics processing unit (GPU), accelerated processing unit (APU), microprocessor, microcontroller, digital signal processor (DSP), field-programmable gate array (FPGA), application-specific integrated circuit (ASIC), or another type of processing component. In some embodiments, processor 320 includes one or more processors that can be programmed to perform functions. Memory 330 includes random access memory (RAM), read-only memory (ROM), and / or another type of dynamic or static storage device (e.g., flash memory, magnetic storage, and / or optical storage) that stores information and / or instructions for use by processor 320.

[0040] Storage component 340 stores information and / or software related to the operation and use of device 300. For example, storage component 340 may include hard disks (e.g., magnetic disks, optical disks, magneto-optical disks, and / or solid-state disks), compact discs (CDs), digital versatile discs (DVDs), floppy disks, cassette tapes, magnetic tapes, and / or other types of non-volatile computer-readable storage media, and corresponding drives.

[0041] Input component 350 includes components that allow device 300 to receive information, such as via user input (e.g., a touchscreen display, keyboard, keypad, mouse, button, switch, and / or microphone). Alternatively, input component 350 may include sensors for sensing information (e.g., a Global Positioning System (GPS) component, accelerometer, gyroscope, and / or actuator). Output component 360 includes components that provide output information from device 300 (e.g., a display, speaker, and / or one or more light-emitting diodes (LEDs)).

[0042] The communication interface 370 includes transceiver-like components (e.g., a transceiver and / or separate receiver and transmitter) that enable the device 300 to communicate with other devices, for example, via a wireless connection, a wired connection, or a combination of wireless and wired connections. The communication interface 370 may include an Ethernet interface, an optical interface, a coaxial interface, an infrared interface, a radio frequency interface, a universal serial bus interface, a Wi-Fi interface, a cellular network interface, etc.

[0043] Device 300 can execute one or more processing procedures described in this application. Device 300 can execute these processing procedures in response to processor 320 executing software instructions stored in a non-volatile computer-readable storage medium such as memory 330 and / or storage component 340. Computer-readable storage medium is defined in this application as a non-volatile memory device. Memory devices include storage space within a single physical storage device or storage space distributed across multiple physical storage devices.

[0044] Software instructions can be read into memory 330 and / or storage component 340 from another computer-readable storage medium or another device via communication interface 370. When the software instructions stored in memory 330 and / or storage component 340 are executed, the processor 320 can perform one or more processing procedures described in this application. Alternatively or additionally, hard-wired circuitry can be used in place of, or in combination with, software instructions to perform one or more processing procedures described in this application. Therefore, the embodiments described in this application are not limited to any specific combination of hardware circuitry and software.

[0045] Figure 3 The number and arrangement of components shown are provided as an example. In reality, with... Figure 3 Compared to the components shown, device 300 may include additional components, fewer components, different components, or components arranged differently. Alternatively, a set of components of device 300 (e.g., one or more components) may perform one or more functions described as being performed by another set of components of device 300.

[0046] Figure 4 This is a flowchart of an example process 400 for determining diagnostic information using a reinforcement learning model. In some implementations, Figure 4 One or more processing blocks can be executed by platform 220. In some implementations, Figure 4 One or more processing blocks can be executed by another device or a group of devices that are separate from or include the platform 220, such as user equipment 210.

[0047] As in Figure 4 As shown, process 400 may include: the device receiving medical information associated with the user (box 410).

[0048] For example, platform 220 can receive user-related medical information, such as EMR data, EHR data, and / or other types of formatted medical data. This medical information can identify disease history, symptoms, treatment history, biometrics, medication information, and so on. This medical information can be obtained from an automated database containing the patient's historical and current medical records, as well as other key information, including all prescriptions and personal allergy documents.

[0049] Platform 220 can receive medical information and use technologies such as Named Entity Recognition (NER), semantic role labeling, data mining, and parsing to identify specific information.

[0050] like Figure 4 As further shown in the diagram, process 400 may include: the device determining inquiry information based on medical information (box 420).

[0051] Platform 220 can identify potential questions to be provided to doctors, healthcare professionals, etc., based on medical information, in order to determine additional information relevant to the final diagnosis. For example, these questions could be about current illness, previous illnesses, medication history, test results, etc.

[0052] Platform 220 can identify a set of potential problems and determine the importance score for each potential problem. For example, highly relevant or highly probative problems can be associated with a high importance score, while less relevant or less probative problems can be associated with a lower importance score.

[0053] Platform 220 can use reinforcement learning techniques to determine the set of importance scores. For example, platform 220 can use EMR, EHR, knowledge base data, rule data, etc., to train the reward function of the reinforcement learning model.

[0054] like Figure 4 As further shown, process 400 may include determining whether the query information meets a threshold score (box 430). Platform 220 may determine whether the importance score included in a potential question meets a threshold score. In this way, platform 220 can identify specific questions to be provided to doctors, healthcare professionals, etc.

[0055] like Figure 4 As further shown, if the query information does not meet the threshold score (box 430 - No), process 400 may include returning to box 420. In this case, platform 220 can identify another potential problem to be addressed.

[0056] like Figure 4 As further shown, if the query information meets the threshold score (box 430 - Yes), then process 400 may include: the device providing the query information to allow receiving response information (box 440).

[0057] Platform 220 may provide the query information (e.g., questions, information requests, etc.) to another device or output component to allow the receipt of response information.

[0058] As in Figure 4 As further shown in the diagram, process 400 may include: the device receiving the response information based on the provided inquiry information (block 450).

[0059] Platform 220 can receive response information based on query information from another device (e.g., the device may have already received input from a doctor, healthcare professional, etc.).

[0060] like Figure 4 As further illustrated, process 400 may include: the device using a machine learning model to determine diagnostic information based on the medical information and the response information (box 460).

[0061] Platform 220 can determine diagnostic information based on medical and response information, such as identifying diagnoses, treatment options, and prescription drugs.

[0062] Platform 220 can use models such as Recurrent Neural Network (RNN), Convolutional Neural Network (CNN), and Support Vector Machine (SVM) to determine diagnostic information.

[0063] like Figure 4 As further shown, process 400 may include: the device providing diagnostic information to a group of devices via a network (box 470).

[0064] Platform 220 can provide diagnostic information to a group of other devices in real time via a network. In some implementations, platform 220 can provide diagnostic information in a standardized format to allow for the updating of various databases and records based on the diagnostic information.

[0065] Platform 220 can provide standardized diagnostic information to a group of devices in real time via a network. Platform 220 can use standardization technology to standardize the diagnostic information, so that each device in the group can utilize the standardized diagnostic information.

[0066] Platform 220 can collect medical information and convert and merge medical information from various physicians and healthcare providers into a standardized format. Furthermore, platform 220 can generate diagnostic information associated with the standardized format. Platform 220 can store the standardized medical and / or diagnostic information in a set of network-based storage devices (e.g., platform 220 itself), and generate messages to notify healthcare providers, doctors, medical staff, patients, etc., whenever medical and / or diagnostic information is generated or updated.

[0067] Furthermore, platform 220 can provide diagnostic information to the group of devices in real time (e.g., substantially simultaneously with the generation of diagnostic information), allowing the group of devices to update and / or utilize the diagnostic information in real time. In this way, individual users of the group of devices can immediately access the latest diagnostic information.

[0068] In this way, compared with non-standardized medical information associated with different medical providers, some embodiments of this application allow the generation of standardized medical and / or diagnostic information in real time and provide it to multiple different devices, thereby allowing different users to share medical and / or diagnostic information.

[0069] Furthermore, in this way, some implementations of this application allow for the real-time provision of complete and accurate medical and / or diagnostic information. Compared to situations where multiple different healthcare professionals have incomplete or inaccurate medical or diagnostic information, some embodiments of this application allow for the dissemination and easy sharing of complete and accurate medical and diagnostic information among healthcare professionals.

[0070] Although Figure 4 Example blocks of process 400 are shown, but in some implementations, process 400 may include additional blocks, fewer blocks, different blocks, or blocks similar to those in other implementations. Figure 4 The blocks shown are arranged differently. Alternatively, two or more blocks of process 400 can be executed in parallel.

[0071] This application also provides a clinical decision support device based on reinforcement learning, such as... Figure 5a As shown, the device includes: a first receiving module 510 for receiving medical information associated with a user; a first determining module 520 for determining query information based on the medical information associated with the user and a reinforcement learning model; a first providing module 530 for providing the query information to allow receiving response information; a second receiving module 540 for receiving the response information based on providing the query information; a second determining module 550 for determining diagnostic information using a machine learning model based on the medical information and the response information; and a second providing module 560 for providing standardized format diagnostic information to a group of devices in real time via a network.

[0072] like Figure 5bAs shown, the device may further include: an execution module 570, configured to perform named entity recognition technology using the medical information; in this case, the second determining module 550 is configured to determine the diagnostic information based on the execution of the named entity recognition technology. The execution module 570 may also be configured to perform semantic role labeling technology using the medical information; in this case, the second determining module 550 is configured to determine the diagnostic information based on the semantic role labeling technology.

[0073] The device further includes a third determining module 580, configured to: determine a set of potential inquiries based on the medical information; determine individual scores associated with the set of potential inquiries; and determine the inquiry with the highest score from the set of potential inquiries; in this case, the first determining module 520 is configured to determine the inquiry information based on the inquiry with the highest score.

[0074] The device further includes a training module for training the reinforcement learning model using at least one of electronic medical record (EMR) data and electronic health record (EHR) data. Specifically, the training module is used to train the reward function of the reinforcement learning model using at least one of the EMR data and the EHR data.

[0075] The second determining module 550 determines the diagnostic information by using at least one of a recurrent neural network, a convolutional neural network, and a support vector machine.

[0076] For a description of the above device embodiments, please refer to the preceding method embodiments.

[0077] This application embodiment also provides a clinical decision support device based on reinforcement learning, comprising: at least one memory for storing program code; and at least one processor for reading the program code and operating according to the instructions of the program code, the program code comprising: a first receiving code for causing the at least one processor to receive medical information associated with a user; a first determining code for causing the at least one processor to determine inquiry information based on the medical information associated with the user and a reinforcement learning model; a providing code for causing the at least one processor to provide the inquiry information to allow receiving response information; a second receiving code for causing the at least one processor to receive the response information based on providing the inquiry information; a second determining code for causing the at least one processor to determine diagnostic information using a machine learning model based on the medical information and the response information; and a providing code for causing the at least one processor to provide standardized format diagnostic information to a group of devices in real time via a network.

[0078] The device may further include: execution code for causing the at least one processor to perform named entity recognition technology using the medical information; wherein the second determining code is used to cause the at least one processor to determine the diagnostic information based on the execution of the named entity recognition technology.

[0079] The device may further include: execution code for causing the at least one processor to perform semantic role labeling technology using the medical information; wherein the second determining code is used to cause the at least one processor to determine the diagnostic information based on the semantic role labeling technology.

[0080] The device may further include: third determining code for causing the at least one processor to:

[0081] Based on the medical information, a set of potential inquiries is identified; scores associated with the set of potential inquiries are determined; the inquiry with the highest score is identified from the set of potential inquiries; wherein the first determining code is used to enable the at least one processor to determine the inquiry information based on the inquiry with the highest score.

[0082] The device may further include: training code for enabling the at least one processor to train the reinforcement learning model using at least one of electronic medical record (EMR) data and electronic health record (EHR) data.

[0083] The device may further include: training code for enabling the at least one processor to train the reward function of the reinforcement learning model using at least one of the EMR data and the EHR data.

[0084] The second determining code is used to enable the at least one processor to determine the diagnostic information by using at least one of a recurrent neural network, a convolutional neural network, and a support vector machine.

[0085] The foregoing description provides an exposition and description, but is not intended to be exhaustive or to limit the embodiments to the precise forms disclosed. Modifications and variations are possible based on the foregoing description, or may be obtained from practical implementations.

[0086] As used in this application, the term "component" is intended to be interpreted broadly as hardware, firmware, or a combination of hardware and software.

[0087] It is evident that the systems and / or methods described in this application can be implemented in various forms, including hardware, firmware, or a combination of hardware and software. The actual dedicated control hardware or software code used to implement these systems and / or methods does not limit the implementation. Therefore, while the operation and performance of the systems and / or methods are described in this application without reference to specific software code—it should be understood that software and hardware can be designed to implement the said systems and / or methods based on the descriptions in this application.

[0088] Even if combinations of features are recited in the claims and / or disclosed in the specification, these combinations are not intended to limit the disclosure of possible embodiments. In fact, many of these features can be combined in ways not specifically recited in the claims and / or disclosed in the specification. While each dependent claim may be directly subordinated to only one claim, the disclosure of possible embodiments includes combinations of each dependent claim in the claim set with each of the other claims.

[0089] The elements, actions, or instructions used in this application should not be construed as critical or essential unless explicitly stated otherwise. Furthermore, as used in this application, the articles “a” and “an” are intended to include one or more items and may be used interchangeably with “one or more.” Additionally, as used in this application, the term “set” is intended to include one or more items (e.g., related items, unrelated items, combinations of related and unrelated items, etc.) and may be used interchangeably with “one or more.” The term “one” or similar language is used where only one item is referred to. Furthermore, as used in this application, the terms “has,” “have,” “having,” etc., are intended to be open-ended terms. Additionally, the phrase “based on” is intended to mean “at least partially based on” unless explicitly stated otherwise.

Claims

1. A clinical decision support method based on reinforcement learning, characterized in that, It includes: Receive medical information associated with the user; Based on the user-associated medical information and reinforcement learning model, the query information is determined; Provide the query information to allow receiving response information; Based on the provided inquiry information, the response information is received; Using a machine learning model, diagnostic information is determined based on the medical information and the response information; as well as Standardized diagnostic information is provided to a group of devices in real time via the network; Among them, determining the inquiry information includes: Determine whether the importance score of a potential problem meets a threshold score, wherein the importance score is determined using a reinforcement learning model, wherein the reward function in the reinforcement learning model scores the importance of each potential problem. If a potential problem may lead to death or sequelae, its importance score is higher, and if a potential problem is highly correlated with or highly probative of the final diagnosis, its importance score is higher. When the importance score meets the threshold score, the potential question is determined to be the query information.

2. The method according to claim 1, characterized in that, It further includes: The medical information is used to perform named entity recognition technology; The determination of the diagnostic information includes: determining the diagnostic information based on the named entity recognition technology.

3. The method according to claim 1, characterized in that, It further includes: The medical information is used to perform semantic role labeling technology; The determination of the diagnostic information includes: determining the diagnostic information based on the semantic role labeling technology.

4. The method according to claim 1, characterized in that, It further includes: Based on the aforementioned medical information, a set of potential inquiries is identified; Determine the individual scores associated with the set of potential inquiries; Identify the question with the highest score from the set of potential questions; The determination of the query information includes: determining the query information based on the query with the highest score.

5. The method according to any one of claims 1 to 4, characterized in that, It further includes: The reinforcement learning model is trained using at least one of electronic medical record (EMR) data and electronic health record (EHR) data.

6. The method according to claim 5, characterized in that, It further includes: The reward function of the reinforcement learning model is trained using at least one of the EMR data and the EHR data.

7. The method according to any one of claims 1 to 4, characterized in that, Determining the diagnostic information includes using at least one of a recurrent neural network, a convolutional neural network, and a support vector machine.

8. A clinical decision support device based on reinforcement learning, characterized in that, It includes: The first receiving module is used to receive medical information associated with the user; The first determining module is used to determine the query information based on the medical information associated with the user and the reinforcement learning model; A first providing module is used to provide the query information to allow receiving response information; The second receiving module is used to receive the response information based on the provided inquiry information; The second determining module is used to determine diagnostic information based on the medical information and the response information using a machine learning model; as well as The second providing module is used to provide standardized diagnostic information to a group of devices in real time via a network; The first determining module determines the query information, including: Determine whether the importance score of a potential problem meets a threshold score, wherein the importance score is determined using a reinforcement learning model, wherein the reward function in the reinforcement learning model scores the importance of each potential problem. If a potential problem may lead to death or sequelae, its importance score is higher, and if a potential problem is highly correlated with or highly probative of the final diagnosis, its importance score is higher. When the importance score meets the threshold score, the potential question is determined to be the query information.

9. The device according to claim 8, characterized in that, It further includes: An execution module is used to perform named entity recognition technology using the medical information; The second determining module is used to determine the diagnostic information based on the execution of the named entity recognition technology.

10. The device according to claim 8, characterized in that, It further includes: The execution module is used to perform semantic role labeling technology using the medical information; The second determining module is used to determine the diagnostic information based on the semantic role labeling technology.

11. The device according to claim 8, characterized in that, It further includes: The third determining module is used for: Based on the aforementioned medical information, a set of potential inquiries is identified; Determine the individual scores associated with the set of potential inquiries; Identify the question with the highest score from the set of potential questions; The first determining module is used to determine the query information based on the query with the highest score.

12. The device according to any one of claims 8 to 11, characterized in that, It further includes: A training module is used to train the reinforcement learning model using at least one of electronic medical record (EMR) data and electronic health record (EHR) data.

13. The device according to claim 12, characterized in that, It further includes: A training module is used to train the reward function of the reinforcement learning model using at least one of the EMR data and the EHR data.

14. The device according to any one of claims 8 to 11, characterized in that, The second determining module is used to determine the diagnostic information by using at least one of a recurrent neural network, a convolutional neural network, and a support vector machine.

15. A non-volatile computer-readable storage medium storing instructions, characterized in that, The instructions include: one or more instructions that, when executed by one or more processors of the device, cause the one or more processors to perform the method according to any one of claims 1 to 7.

Images