Artificial intelligence-based risk management for digital health technologies

Abstract

Risk analysis of digital health technologies is provided by an artificial intelligence (AI)-driven risk management framework. User input data that contain an indication of a target digital health technology (DHT) and a user prompt are received and processed by a machine learning model, such as a large language model (LLM) subject to retrieval-augmented generation (RAG), to generate risk context data. Risk data may be retrieved from a risk library containing hazard and harm definitions associated with the target DHT. Risk management data are generated based on the risk context data and risk data. A risk management report is generated from the risk management data and output to a user. The risk management report may include one or more risk cards that visually depict risk analysis components in the risk management data.

Description

Mayo 2024-421 Q&B Docket: 630666.01596 ARTIFICIAL INTELLIGENCE-BASED RISK MANAGEMENT FOR DIGITAL HEALTH TECHNOLOGIES CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims the benefit of U.S. Provisional Patent Application Serial No. 63 / 684,329, filed on August 16, 2024, and entitled “Artificial Intelligence-Based Risk Management for Digital Health Technologies,” which is herein incorporated by reference in its entirety. BACKGROUND

[0002] Traditional risk management processes in medical device development are often inefficient, leading to increased patient risks and regulatory complications. These inefficiencies stem from manual processes and disparate tools for hazard identification, risk assessment, and documentation, which can be inconsistent, incomplete, and error prone. Existing technologies have several disadvantages, such as consistency and variability in manual processes, being incomplete and error-prone, and requiring more contextual relevance in traditional risk assessments. SUMMARY OF THE DISCLOSURE

[0003] It is an aspect of the present disclosure to provide a method for risk management of a digital health technology. The method includes receiving user input data via a client device. The user input data include an indication of a target digital health technology (DHT) and a user prompt. The user input data are processed using a machine learning model to generate risk management data as an output. Processing the user input data using the machine learning model includes: processing the user input data to retrieve risk data associated with the target DHT from a risk library; processing the user prompt with the machine learning model to generate risk context data associated with the DHT; and generating the risk management data based on the risk data and the risk context data. A risk management report is generated based on the risk management data, and may be output to a user via the client device. BRIEF DESCRIPTION OF THE DRAWINGS

[0004] FIG.1 is a block diagram of an example risk management system according to some embodiments described in the present disclosure. 1 QB\630666.01596\97915262.2Mayo 2024-421 Q&B Docket: 630666.01596

[0005] FIG. 2 is a block diagram of example components that can implement the risk management system of FIG.1.

[0006] FIG.3 illustrates an example process for using retrieval-augmented generation (RAG) and a large language model (LLM) for generating risk context using additional content.

[0007] FIGS.4A–4C illustrate examples of risk cards that can be generated to visualize risk management data for a target digital health technology (DHT). FIG. 4A illustrates an example risk card. FIG.4B illustrates an example risk card that is augmented with a sequence diagram. FIG. 4C illustrates the example sequence diagram used to augment the risk card in FIG.4B.

[0008] FIG. 5 illustrates an example of an alternative risk card format that can be generated to visualize risk management data for a targeted DHT.

[0009] FIG.6 is a flowchart of an example method for generating and visualizing risk management data in accordance with some embodiments described in the present disclosure. DETAILED DESCRIPTION

[0010] Described here are systems and methods for assessing risk in digital health technologies. The disclosed systems and methods provide an artificial intelligence (AI)-driven risk management framework for enhancing the precision and effectiveness of risk management for digital health technologies. In some embodiments, the risk management framework described in the present disclosure is tailored for the unique dynamics of healthcare institutions and / or for broad application across complex, safety-critical digital health technology products. In other examples, the framework can be adapted for other industries, such as private industry, medical device manufacturers, or the like.

[0011] Deployment of AI-enabled digital health technologies into clinical practice requires cross-functional, collaborative effort. Incorporating a risk management methodology into a design process can be used to develop safe and effective products; however, using existing risk management tools, development teams often struggle to efficiently identify risk analysis components, lengthening the translation time of products that improve patient outcomes. The disclosed risk management framework addresses these inefficiencies by including a reusable, standard library for hazard and harm definitions; using large language models (LLMs) to perform grounded retrieval-augmented generation (RAG) on specific risk context and mitigations; and visually representing system structure and behavior. These techniques can be paired with an automated and simplified format synchronization and 2 QB\630666.01596\97915262.2Mayo 2024-421 Q&B Docket: 630666.01596 standardized document generation, facilitating review by clinical stakeholders. The AI-assisted process enables accelerated deployment of safe digital health technologies by focusing critical stakeholder efforts on the most pertinent information, thereby improving patient outcomes.

[0012] As noted above, the disclosed AI-driven framework implements a reusable, standardized library for hazard and harm definitions. The reusable, standardized library allows for consistency and precision in identifying potential risks, reducing variability, and enhancing reliability.

[0013] LLMs are also utilized to perform RAG tailored to specific risk contexts and mitigations. Advantageously, using LLMs for RAG can provide detailed, contextually relevant risk assessments, accurately identifying and addressing potential risks and mitigations.

[0014] The framework also implements advanced visualization techniques for visualizing system structure and behavior in a way that engages stakeholder review by supplementing visual media to support, simplify, and scale efficient product deployment and downstream patient impact. Advanced visualization tools offer clear, intuitive insights into risk areas, facilitating better understanding and communication among stakeholders who may not be well-versed in product development risk management frameworks. As an example, the disclosed risk management framework can generate dynamic, traceable risk cards that visually represent risk analysis components using a templated display containing relevant, but technical, content that has been simplified for stakeholders unfamiliar with existing standards or frameworks.

[0015] Using the disclosed risk management framework, digital health technology development teams can facilitate risk analysis review across subject matter domains, including clinical experts and technical developers. These teams can utilize shareable risk components with contextual project information from LLMs to perform scenario-based risk analysis and evaluation. For instance, evaluation of risk analysis results can include collaborative review of risk cards created by the risk management framework.

[0016] FIG.1 illustrates an overview of an AI-driven risk management system 100 in accordance with embodiments described in the present disclosure. The risk management system 100 generally includes a client device 150, a server 152 containing at least a risk library 160 and optionally one or more knowledge stores 162, and a communication network 154. A user inputs a prompt or question pertaining to the risk analysis of a target digital health technology (DHT) via the client device 150. Those user input data (e.g., the user prompt or user question, target DHT data) are then sent to the server 152 where they are processed using 3 QB\630666.01596\97915262.2Mayo 2024-421 Q&B Docket: 630666.01596 an LLM or other machine learning model to generate risk management data. The LLM retrieves risk data from the risk library 160 based on the user input data using a RAG framework. The risk management data are then used to create a risk management report for the DHT, which may be sent from the server 152 to the client device 150 where the risk management report may be displayed to the user. As described, the risk report may include one or more risk cards that provide advanced visualization of the risk management profile for the target DHT, one or more score labels that provide efficient visualization of the risk management profile for the target DHT, and / or one or more messages sent from the client device 150 and indicating the risk management data.

[0017] As shown in FIG.1, the client device 150 can receive one or more types of data (e.g., user input data, target DHT data, medical condition or disease data) from data source 102. In some embodiments, the client device 150 can execute at least a portion of a digital health technology risk analysis system 104 to generate risk management data from data received from the data source 102.

[0018] Additionally or alternatively, in some embodiments, the client device 150 can communicate information about data received from the data source 102 to the server 152 over the communication network 154, which can execute at least a portion of the digital health technology risk analysis system 104. In such embodiments, the server 152 can return information to the client device 150 (and / or any other suitable client device) indicative of an output of the digital health technology risk analysis system 104.

[0019] In some embodiments, the client device 150 and / or the server 152 may include a machine learning controller and / or an artificial intelligence controller. In instances where the client device 150 and / or the server 152 include a machine learning controller, the machine learning controller may be integrated into and implemented by the electronic controller of the client device 150 and / or server 152 (e.g., the electronic controller may be referred to as a machine learning controller). In some embodiments, the machine learning controller is a separate controller from the electronic controller of the client device 150 and / or server 152 and includes an electronic processor and memory, similar to the machine learning controller.

[0020] The machine learning controller implements a machine learning program, algorithm or model. In some implementations, the machine learning controller is configured to construct a model (e.g., building one or more algorithms) based on example inputs, which may be done using supervised learning, unsupervised learning, reinforcement learning, ensemble learning, active learning, transfer learning, or other suitable learning techniques for machine 4 QB\630666.01596\97915262.2Mayo 2024-421 Q&B Docket: 630666.01596 learning programs, algorithms, or models. Additionally or alternatively, the machine learning controller is configured to modify a machine learning program, algorithm, or model; to activate and / or deactivate a machine learning program, algorithm, or model; to switch between different machine learning programs, algorithms, or models; and / or to change output thresholds for a machine learning program, algorithms, or model.

[0021] The machine learning controller can include a trained machine learning controller that utilizes previously collected data to profile, evaluate, or otherwise assess the risk associated with a particular digital health technology. The machine learning controller can generate risk assessment and / or management data for the target digital health technology.

[0022] The machine learning controller may be a static machine learning controller, a self-updating machine learning controller, an adjustable machine learning controller, or the like. In some embodiments, the client device 150 and / or server 152 may include more than one machine learning controller, and each machine learning controller may be of a different type.

[0023] In some embodiments, the client device 150 and / or server 152 may implement an artificial intelligence controller instead of, or in addition to, the machine learning controller. The artificial intelligence controller implements one or more AI programs, algorithms, or models different from machine learning program, algorithms, or models. In some embodiments, the AI controller is configured to implement the one or more AI programs, algorithms, or models such as an expert system, a rules engine, a symbolic logic, one or more knowledge graphs, and so on. In some embodiments, the AI controller is integrated into and implemented by the electronic controller of the client device 150 and / or server 152 (e.g., the electronic controller may be referred to as an AI controller). In some embodiments, the AI controller is a separate controller from the electronic controller of the client device 150 and / or server 152 and includes an electronic processor and memory, similar to the machine learning controller.

[0024] In some embodiments, client device 150 and / or server 152 can be any suitable client device or combination of devices, such as a desktop computer, a laptop computer, a smartphone, a tablet computer, a wearable computer, a server computer, a virtual machine being executed by a physical client device, and so on.

[0025] The client device 150 communicates with the server 152, for example, to send user prompts and other user input data to the server 152 for initiating a risk analysis of a target DHT. In some embodiments, the client device 150 may include a long-range transceiver to communicate with the server 152 and / or a short-range transceiver to communicate with other 5 QB\630666.01596\97915262.2Mayo 2024-421 Q&B Docket: 630666.01596 client devices or servers via, for example, a short-range communication protocol such as Bluetooth® or Wi-Fi®.

[0026] To perform its various functions, the client device 150 may include a device electronic control assembly having a device electronic processor, a device memory, and a transceiver (also referred to as a device transceiver). The client device 150 generally includes a user interface and / or dashboard by which users can provide user input to the client device 150. The user input data are received by the server 152 where they are processed by an LLM, which may retrieve risk data from the risk library 160 to generate an output using a RAG framework. The output of the LLM, or other machine learning model, can include risk management data generated for the target DHT based on the user input data and the risk data retrieved from the risk library 160. The user input data may be processed to determine which risk data to retrieve from the risk library 160. Additionally, the risk management data may be processed by the server 152 and / or the client device 150 to generate a risk management report, which in some embodiments may include one or more risk cards, one or more score labels, one or more messages, or combinations thereof. The risk management report, risk cards, score labels, and / or messages can include a visualization of the risk data in addition to other data and information indicating the risk analysis performed for the target DHT. The device electronic processor and device memory may collectively form a device electronic controller that is configured to perform certain methods described herein (e.g., the method of FIG.6).

[0027] In some embodiments, data source 102 can be any suitable source of data (e.g., measurement data, user input data, DHT data, patient health data), another client device (e.g., a server storing measurement data, user input data, DHT data, medical condition or disease data), and so on. In some embodiments, data source 102 can be local to client device 150. For example, data source 102 can be incorporated with client device 150 (e.g., client device 150 can be configured as part of a device for measuring, recording, estimating, acquiring, or otherwise collecting or storing data). As another example, data source 102 can be connected to client device 150 by a cable, a direct wireless link, and so on. Additionally or alternatively, in some embodiments, data source 102 can be located locally and / or remotely from client device 150, and can communicate data to client device 150 (and / or server 152) via a communication network (e.g., communication network 154).

[0028] The server 152 includes a server electronic control assembly having a server electronic processor, a server memory, and a transceiver. The transceiver allows the server 152 to communicate with the client device 152. The server electronic processor receives user input 6 QB\630666.01596\97915262.2Mayo 2024-421 Q&B Docket: 630666.01596 data (e.g., a user prompt or question, target DHT data, patient health data, etc.) from client device 150 (e.g., via the network 154) and processes the user input data to retrieve risk data from the risk library 160. As noted above, the server 152 may maintain a risk library 160 as a database (e.g., on the server memory) for containing hazard definition data and / or harm definition data related to a plurality of different DHTs.

[0029] The server 152 may also contain trained machine learning models, artificial intelligence models (e.g., rules and / or other logic implemented in an artificial intelligence model and / or algorithm), and the like. For example, the server 152 may contain one or more trained LLMs that may be used to generate risk management data from the user input data and risk data retrieved from the risk library 160. As described, the LLM(s) may be directed using RAG to retrieve relevant data from the risk library 160 or other authoritative knowledge sources stored on the server 152 (e.g., from the electronic memory of the server), another server, the client device 150, or another suitable computing device. Thus, the server 152 may also contain one or more knowledge stores 162 containing project-specific data that may be utilized in a RAG framework in connection with an LLM. For example, the knowledge store(s) 162 may contain DHT requirements, risks, intended use, and so on. The knowledge store(s) 162 may include one or more databases, such as one or more vector databases.

[0030] Although illustrated as a single device, the server 152 may be a distributed device in which the server electronic processor and server memory are distributed among two or more units that are communicatively coupled (e.g., via the network 154).

[0031] In some embodiments, communication network 154 can be any suitable communication network or combination of communication networks. For example, communication network 154 can include a Wi-Fi network (which can include one or more wireless routers, one or more switches, etc.), a peer-to-peer network (e.g., a Bluetooth network), a cellular network (e.g., a 3G network, a 4G network, etc., complying with any suitable standard, such as CDMA, GSM, LTE, LTE Advanced, WiMAX, etc.), other types of wireless network, a wired network, and so on. In some embodiments, communication network 154 can be a local area network, a wide area network, a public network (e.g., the Internet), a private or semi-private network (e.g., a corporate or university intranet), any other suitable type of network, or any suitable combination of networks. Communications links shown in FIG. 1 can each be any suitable communications link or combination of communications links, such as wired links, fiber optic links, Wi-Fi links, Bluetooth links, cellular links, and so on. 7 QB\630666.01596\97915262.2Mayo 2024-421 Q&B Docket: 630666.01596

[0032] The network 154 may therefore be a long-range wireless network such as the Internet, a local area network (LAN), a wide area network (WAN), or a combination thereof. In other embodiments, the network 154 may be a short-range wireless communication network, and in yet other embodiments, the network 154 may be a wired network using, for example, ethernet cables, USB cables, or the like. Additionally or alternatively, the network 154 may include a combination of long-range, short-range, and / or wired connections. In some embodiments, the network 154 may therefore include both wired and wireless connections.

[0033] Referring now to FIG. 2, an example of hardware 200 that can be used to implement data source 102, client device 150, and server 152 in accordance with some embodiments of the systems and methods described in the present disclosure is shown.

[0034] As shown in FIG. 2, in some embodiments, client device 150 can include a processor 202, a display 204, one or more inputs 206, one or more communication systems 208, and / or memory 210. In some embodiments, processor 202 can be any suitable hardware processor or combination of processors, such as a central processing unit (CPU), a graphics processing unit (GPU), and so on. In some embodiments, display 204 can include any suitable display devices, such as a liquid crystal display (LCD) screen, a light-emitting diode (LED) display, an organic LED (OLED) display, an electrophoretic display (e.g., an “e-ink” display), a computer monitor, a touchscreen, a television, and so on. In some embodiments, inputs 206 can include any suitable input devices and / or sensors that can be used to receive user input, such as a keyboard, a mouse, a touchscreen, a microphone, and so on.

[0035] In some embodiments, communications systems 208 can include any suitable hardware, firmware, and / or software for communicating information over communication network 154 and / or any other suitable communication networks. For example, communications systems 208 can include one or more transceivers, one or more communication chips and / or chip sets, and so on. In a more particular example, communications systems 208 can include hardware, firmware, and / or software that can be used to establish a Wi-Fi connection, a Bluetooth connection, a cellular connection, an Ethernet connection, and so on.

[0036] In some embodiments, memory 210 can include any suitable storage device or devices that can be used to store instructions, values, data, or the like, that can be used, for example, by processor 202 to present content using display 204, to communicate with server 152 via communications system(s) 208, and so on. Memory 210 can include any suitable volatile memory, non-volatile memory, storage, or any suitable combination thereof. For example, memory 210 can include random-access memory (RAM), read-only memory (ROM), 8 QB\630666.01596\97915262.2Mayo 2024-421 Q&B Docket: 630666.01596 electrically programmable ROM (EPROM), electrically erasable ROM (EEPROM), other forms of volatile memory, other forms of non-volatile memory, one or more forms of semi- volatile memory, one or more flash drives, one or more hard disks, one or more solid state drives, one or more optical drives, and so on. In some embodiments, memory 210 can have encoded thereon, or otherwise stored therein, a computer program for controlling operation of client device 150. In such embodiments, processor 202 can execute at least a portion of the computer program to present content (e.g., images, user interfaces, graphics, tables), receive content from server 152, transmit information to server 152, and so on. For example, the processor 202 and the memory 210 can be configured to perform the methods described herein (e.g., the method of FIG.6).

[0037] In some embodiments, server 152 can include a processor 212, a display 214, one or more inputs 216, one or more communications systems 218, and / or memory 220. In some embodiments, processor 212 can be any suitable hardware processor or combination of processors, such as a CPU, a GPU, and so on. In some embodiments, display 214 can include any suitable display devices, such as an LCD screen, LED display, OLED display, electrophoretic display, a computer monitor, a touchscreen, a television, and so on. In some embodiments, inputs 216 can include any suitable input devices and / or sensors that can be used to receive user input, such as a keyboard, a mouse, a touchscreen, a microphone, and so on.

[0038] In some embodiments, communications systems 218 can include any suitable hardware, firmware, and / or software for communicating information over communication network 154 and / or any other suitable communication networks. For example, communications systems 218 can include one or more transceivers, one or more communication chips and / or chip sets, and so on. In a more particular example, communications systems 218 can include hardware, firmware, and / or software that can be used to establish a Wi-Fi connection, a Bluetooth connection, a cellular connection, an Ethernet connection, and so on.

[0039] In some embodiments, memory 220 can include any suitable storage device or devices that can be used to store instructions, values, data, or the like, that can be used, for example, by processor 212 to present content using display 214, to communicate with one or more client devices 150, and so on. Memory 220 can include any suitable volatile memory, non-volatile memory, storage, or any suitable combination thereof. For example, memory 220 can include RAM, ROM, EPROM, EEPROM, other types of volatile memory, other types of non-volatile memory, one or more types of semi-volatile memory, one or more flash drives, one or more hard disks, one or more solid state drives, one or more optical drives, and so on. 9 QB\630666.01596\97915262.2Mayo 2024-421 Q&B Docket: 630666.01596 In some embodiments, memory 220 can have encoded thereon a server program for controlling operation of server 152. In such embodiments, processor 212 can execute at least a portion of the server program to transmit information and / or content (e.g., data, images, a user interface) to one or more client devices 150, receive information and / or content from one or more client devices 150, receive instructions from one or more devices (e.g., a personal computer, a laptop computer, a tablet computer, a smartphone), and so on.

[0040] In some embodiments, the server 152 is configured to perform the methods described in the present disclosure. For example, the processor 212 and memory 220 can be configured to perform the methods described herein (e.g., the method of FIG.6).

[0041] In some embodiments, data source 102 can include a processor 222, one or more inputs 224, one or more communications systems 226, and / or memory 228. In some embodiments, processor 222 can be any suitable hardware processor or combination of processors, such as a CPU, a GPU, and so on. In some embodiments, the one or more inputs 224 are generally configured to acquire data, receive user inputs, or both, and can include a user interface. Additionally or alternatively, in some embodiments, the one or more inputs 224 can include any suitable hardware, firmware, and / or software for coupling to and / or controlling operations of a user interface or other input. In some embodiments, one or more portions of the input(s) 224 can be removable and / or replaceable.

[0042] Note that, although not shown, data source 102 can include any suitable inputs and / or outputs. For example, data source 102 can include input devices and / or sensors that can be used to receive user input, such as a keyboard, a mouse, a touchscreen, a microphone, a trackpad, a trackball, and so on. As another example, data source 102 can include any suitable display devices, such as an LCD screen, an LED display, an OLED display, an electrophoretic display, a computer monitor, a touchscreen, a television, etc., one or more speakers, and so on.

[0043] In some embodiments, communications systems 226 can include any suitable hardware, firmware, and / or software for communicating information to client device 150 (and, in some embodiments, over communication network 154 and / or any other suitable communication networks). For example, communications systems 226 can include one or more transceivers, one or more communication chips and / or chip sets, and so on. In a more particular example, communications systems 226 can include hardware, firmware, and / or software that can be used to establish a wired connection using any suitable port and / or communication standard (e.g., VGA, DVI video, USB, RS-232, etc.), Wi-Fi connection, a Bluetooth connection, a cellular connection, an Ethernet connection, and so on. 10 QB\630666.01596\97915262.2Mayo 2024-421 Q&B Docket: 630666.01596

[0044] In some embodiments, memory 228 can include any suitable storage device or devices that can be used to store instructions, values, data, or the like, that can be used, for example, by processor 222 to control the one or more inputs 224, and / or receive data from the one or more inputs 224; to generate images from data; present content (e.g., data, images, a user interface) using a display; communicate with one or more client devices 150; and so on. Memory 228 can include any suitable volatile memory, non-volatile memory, storage, or any suitable combination thereof. For example, memory 228 can include RAM, ROM, EPROM, EEPROM, other types of volatile memory, other types of non-volatile memory, one or more types of semi-volatile memory, one or more flash drives, one or more hard disks, one or more solid state drives, one or more optical drives, and so on. In some embodiments, memory 228 can have encoded thereon, or otherwise stored therein, a program for controlling operation of data source 102. In such embodiments, processor 222 can execute at least a portion of the program to generate images, transmit information and / or content (e.g., data, images, a user interface) to one or more client devices 150, receive information and / or content from one or more client devices 150, receive instructions from one or more devices (e.g., a personal computer, a laptop computer, a tablet computer, a smartphone, etc.), and so on.

[0045] In some embodiments, any suitable computer-readable media can be used for storing instructions for performing the functions and / or processes described herein. For example, in some embodiments, computer-readable media can be transitory or non-transitory. For example, non-transitory computer-readable media can include media such as magnetic media (e.g., hard disks, floppy disks), optical media (e.g., compact discs, digital video discs, Blu-ray discs), semiconductor media (e.g., RAM, flash memory, EPROM, EEPROM), any suitable media that is not fleeting or devoid of any semblance of permanence during transmission, and / or any suitable tangible media. As another example, transitory computer- readable media can include signals on networks, in wires, conductors, optical fibers, circuits, or any suitable media that is fleeting and devoid of any semblance of permanence during transmission, and / or any suitable intangible media.

[0046] As used herein in the context of computer implementation, unless otherwise specified or limited, the terms “component,” “system,” “module,” “framework,” and the like are intended to encompass part or all of computer-related systems that include hardware, software, a combination of hardware and software, or software in execution. For example, a component may be, but is not limited to being, a processor device, a process being executed (or executable) by a processor device, an object, an executable, a thread of execution, a 11 QB\630666.01596\97915262.2Mayo 2024-421 Q&B Docket: 630666.01596 computer program, or a computer. By way of illustration, both an application running on a computer and the computer can be a component. One or more components (or system, module, and so on) may reside within a process or thread of execution, may be localized on one computer, may be distributed between two or more computers or other processor devices, or may be included within another component (or system, module, and so on).

[0047] As described above, the risk library 160 may be stored on the server 152 and accessed by the server 152 to retrieve risk data based on user input data (e.g., a user prompt, a selection of a target DHT, patient health data). The risk data stored in the risk library 160 can be used within a RAG framework when processing user prompts using a trained LLM.

[0048] The risk library 160 may contain hazard definition data and / or harm definition data for a plurality of different DHTs. The hazard definition data and / or harm definition data may be tailored to encompass shared DHT risk components. For example, the hazard definition data and / or harm definition data may include hazards and / or harms associated with security risks (e.g., data leaks, injection, DoS), AI risks (e.g., data quality, bias, drift, over trust), usability risks (e.g., misinterpretation, misuse, cognitive load), and so on. Additionally or alternatively, the hazard definition data and / or harm definition data may include hazards and / or harms associated with biological risks, radiation risks, chemical risks, electrical risks, mechanical risks, and / or thermal risks.

[0049] As a non-limiting example, ISO14971 offers a structured framework to manage the risk of harm for patient and operator safety throughout the lifecycle of medical devices. This includes (1) the identification of hazards and hazardous situations that may lead to harm, (2) the evaluation of the harm severity and the probability of occurrence of harm, and (3) the application of mitigations where the risk of harm may be unacceptable against predetermined criteria. The standard provides examples of risk management components, including hazards, sequence of events and circumstances, hazardous situations, and harms. The United States Food and Drug Administration (FDA) also provides medical device report (MDR) codes that categorize adverse events.

[0050] Applicable codes (e.g., MDR codes, other regulatory codes) may be modified and non-applicable codes may be omitted to create a reusable and shareable risk library 160 of DHT-focused hazards and harms. For example, in some embodiments hazards common with electrical hardware or those related to chemical agents can be omitted from the risk library 160 since software-centric devices do not directly present high voltage risks, combustion risks, or other fire risks. The risk library 160 may include data associated with hazards and harms related 12 QB\630666.01596\97915262.2Mayo 2024-421 Q&B Docket: 630666.01596 to the provisioning of a diagnostic result, such as where the hazard of an incorrect result could lead to patient misdiagnosis and disease progression, resulting in delayed treatment. Table 1 depicts an example risk with associated components evaluated on a qualitative, five-level scale for the probability of occurrence, severity, and resulting unacceptable risk of a particular harm. Table 1. Risk Analysis Excerpt of DHTrisk library 160, and Table 3 shows an example of harm definition data that may be stored in the risk library 160. Table 2. Excerpts of Hazards 13 QB\630666.01596\97915262.2Mayo 2024-421 Q&B Docket: 630666.01596 Table 3. Excerpt of Harms. Such a risk library 160 provides a common starting point for development teams to collect risks specific to their DHT use cases. Development teams can share and reuse information across various DHTs and expedite the identification of risk scenarios by prioritizing and scoring risks rather than spending time identifying base hazards. In addition to base hazards, the risk library 160 can be grown as risk analyses are performed across more diverse use cases, further reducing the time spent identifying unique hazards. In these instances, newly identified hazards and / or harms associated with a target DHT can be input to the risk library 160 by uploading the relevant risk data from the client device 150, or the like.

[0053] Using the risk library 160, the risks that are most applicable for specific use cases can be identified in a scalable, more automated fashion. As a non-limiting example, an LLM or other foundation model can be used to help teams self-identify the appropriate risks for the target DHT. LLMs are trained on public information, or other suitable training data, and can include general product development information. However, LLMs often lack current or specific information (e.g., ISO standards) and have the potential to hallucinate if not fine-tuned or grounded within specific context domains. For risk analysis, RAG is employed to ground an LLM with project-specific information like DHT requirements, risks, intended use, etc.. As described above, the RAG framework can implement the risk library 160 to provide risk data associated with the target DHT. Additionally, the RAG framework can implement one or more 14 QB\630666.01596\97915262.2Mayo 2024-421 Q&B Docket: 630666.01596 knowledge stores 162 containing other data (e.g., DHT requirements, intended use, project specifications, etc.).

[0054] In this way, users can interact with projects using a chat interface. One or more vector stores can be indexed to specific areas of knowledge depending on the user query so the relevant project context is used to get a response. An example of this process is illustrated in FIG.3.

[0055] As shown in FIG. 3, the following steps can be used to obtain risk context utilizing RAG-grounded LLMs. First, a user submits a prompt or question related to the DHT, as indicated at 302. As described above, the prompt or question may include a selection or other indication of a target DHT for which risk analysis will be performed. In this way, the prompt or question can identify the risk analysis task to be performed by the risk management system 100. Additionally or alternatively, the prompt or question may include other relevant data or information, including target DHT product requirements, technical specifications, and test protocols. In some embodiments, the prompt or question may include instructions for retrieving patient health data from a suitable knowledge store 162, such as an electronic health record (EHR) database or the like. Similarly, the prompt or question may include instructions for retrieving other relevant data from a suitable knowledge store 162.

[0056] The prompt is routed to one or more tools to generate relevant context to be used to answer the questions, including requirement information, attachments, or project metadata, as indicated at process 304. Answers from each tool are integrated into a single system answer, as indicated at process 306. If the response answers the query then a response is generated at process 308; otherwise, the user is prompted to refine the input question and ask again, as indicated at process 310. Similarly, the initial question that is posed by the user may be decomposed or rephrased before being partially answered in process 304. If the question is not entirely answered, process 310 can be used to rephrase the question, with or without the information produced from the last pass through process 304. This can result in an iterative approach to finding or answering parts of the question before accumulating them into a single, holistic answer. For example, in a non-risk scenario when referencing two vector stores, one containing documentation about Alzheimer’s disease and the other containing information about Covid-19, the question “Which was identified first, Covid-19 or Alzheimer’s disease?” could be split by the system into “When was Covid-19 identified?” and ask vector store 1. Then, the system would ask “When was Alzheimer’s disease identified?” against vector store 2. After both of these sub-questions are answered, the LLM would then generate a final 15 QB\630666.01596\97915262.2Mayo 2024-421 Q&B Docket: 630666.01596 question, utilizing the answers from the sub-questions to produce a final response that answered the original intent. With this environment, users can summarize a project, identify risk gaps specific to their expertise, and generate starting points for new risk content.

[0057] In conjunction with product-specific context, the hazards and harms from the risk library 160 are also used to improve the quality of the risk analysis by using the LLM environment to identify sequences of events and hazardous situations specific to the target DHT. Taking it a step further, the LLM tools are utilized to derive initial best guesses for probabilities within a sequence of events. For example, the sequence outlined in Table 1 is the product of the following probabilities: (1) A patient outside of the AI training parameters is processed, (2) The AI-enabled system produces a false negative result, and (3) The patient receives a misdiagnosis, delaying necessary treatment. Providing additional references or research as attachments allows the risk management system 100 to generate contextual information on probabilities for each sequential event, such as the misdiagnosis rate or how often specific treatments were performed. These outputs can be provided as part of the generated risk management data, which may be displayed in a risk management report, risk cards, score labels, and / or messages.

[0058] It is an advantage of the disclosed systems and methods that the generated risk management data may utilize tabular formatting to capture and document risk analysis. This includes hazards, sequence of events leading to a hazardous situation, initial risk evaluation of each hazardous situation against predefined acceptability criteria for severity and probability of occurrence of harm, risk controls, verification of the implementation of control measures and their effectiveness, residual risk evaluation after controls, a benefit-risk analysis for unacceptable risks, and their associated linkage between each component. Traditionally, table columns are mapped to risk components per the ISO14971 standard. Although tables are easily accessible and electronically sharable, the analysis is difficult to consume due to the format and the amount of text or unstructured data. It is an advantage of the disclosed systems and methods to make relationships between concepts and their contexts. These relationships can be more easily and quickly understood by presenting the risk management data in a visual form rather than in a purely textual form.

[0059] To accelerate targeted review across functional disciplines, a dynamic, traceable risk card is generated using the risk management data. An example risk card is illustrated in FIG.4A. The risk card visually represents risk analysis components using a templated display containing content equivalent to traditional tables, but simplified for stakeholders unfamiliar 16 QB\630666.01596\97915262.2Mayo 2024-421 Q&B Docket: 630666.01596 with existing standards or frameworks. In some cases, text-based content can be supplemented with graphics to capture these components. As one example, illustrated in FIG.4B, a sequence diagram can be provided as part of the risk card. The sequence diagram added to the risk card in FIG. 4B is illustrated in more detail in FIG. 4C. Each card captures at least one row in a typical risk table, but can also convey multiple risks grouped into one view. The content is linked and traced to other risk elements by digitally enumerating each component. The content can be synchronized between the tabular and card formats by mapping associated templates and enumerated content. Switching between formats allows the user experience to be tailored to the stakeholder, which can expedite understanding of risks, information collection, and document generation. In some embodiments, to better evaluate the initial risk, risk controls may not be provided in the initial risk card to encourage stakeholder feedback for potential mitigation strategies. Controls can be added after prioritization in such instances.

[0060] As shown in FIG. 5, in an alternative embodiment, the risk card may be implemented as a score label 500 that provides a standardized visual representation of risk management data for the target DHT. Similar to chemical hazard labels that communicate safety information at a glance, the score label 500 may present risk information in a format that enables rapid assessment and comparison across different digital health technologies. The score label 500 may incorporate the same underlying risk management data generated by the machine learning model described above, but present this information using a scoring system that categorizes risks into distinct domains for enhanced clarity and usability.

[0061] The score label 500 may feature a risk assessment diagram 502 that visually segments risk categories into distinct colored sections 504, each representing a different aspect of the risk profile for the target DHT. In the illustrated example, the risk assessment diagram 502 has a triangular shape. In other cases, the risk assessment diagram 502 may have other shapes, including diamonds, quadrants, and so on, depending on the number of risk categories to be displayed in the risk assessment diagram 502. For example, the risk assessment diagram 502 may include sections 504 for fairness and equity risks, usefulness and usability risks, and safety and reliability risks, with each section assigned a numerical score based on the risk management data. In the illustrated example, the sections 504 are each assigned a distinct color to provide visual contrast between the different risk categories. In some cases, the sections 504 can be assigned a color that indicates a severity of risk or level of concern of the risk associated with each section 504. For example, if the target DHT has a high level of risk for a particular risk category, the associated section 504 may be colored red. Similarly, a moderate risk may 17 QB\630666.01596\97915262.2Mayo 2024-421 Q&B Docket: 630666.01596 be colored as yellow and a low risk may be colored as blue. The central portion 506 of the risk assessment diagram 502 may display the total number of unmitigated risks, providing an immediate indication of the overall risk burden associated with the target DHT. This visual approach may allow stakeholders to quickly identify which risk domains require the most attention and prioritize their review efforts accordingly.

[0062] The score label 500 may also include contextual information about the target DHT, such as the tool name, version number, department, and deployment status, providing relevant identification and operational context alongside the risk assessment. Additional sections may describe the intended use of the DHT and highlight key limitations or contraindications that users should consider. The format of the score label 500 may be particularly advantageous for clinical stakeholders who need to make rapid decisions about DHT deployment or usage, as the score label 500 distills complex risk analysis components into an easily interpretable visual format. Like the dynamic risk cards described above, the score label 500 may include hyperlinks, or the like, to additional content. For example, the score label 500 may be synchronized with underlying tabular data and linked to more detailed risk information, allowing users to access comprehensive analysis when needed while maintaining the simplified visual interface for routine use. Additionally or alternatively, other additional content may be linked to the score label 500, including instructions for using the target DHT, a model card for the DHT, and so on.

[0063] Referring now to FIG. 6, a flowchart is illustrated as setting forth the steps of an example method for generating risk management data for a target DHT using the disclosed risk management framework.

[0064] The method includes receiving user input data via a computer system (e.g., the client device 150) including a selection of a target DHT to evaluate, as indicated at step 602. Additionally, the computer system may receive patient health data associated with the patient for whom the target DHT will be used. Alternatively, the user input data may include a prompt containing instructions to retrieve patient health data or other relevant data from one or more databases (e.g., knowledge stores 162). Additionally or alternatively, the user input data may include a prompt containing instructions to retrieve risk data from a risk library.

[0065] The user input data are then used by the computer system to retrieve risk data from a risk library, as indicated at step 604. Retrieving the risk data may include retrieving such data from a risk library stored on a memory or other suitable data storage device or medium. In some embodiments, the risk library may be stored on a memory or other data 18 QB\630666.01596\97915262.2Mayo 2024-421 Q&B Docket: 630666.01596 storage device or medium of a server (e.g., the server 152). Additionally, the user input data may be processed by a suitably trained machine learning model (e.g., an LLM or other foundation model) to generate answers in response to the prompt contained in the user input data, as indicated at step 606. As described above, the answers can be generated using a RAG framework that retrieves relevant information from one or more knowledge stores. The answers can include requirement information, attachments, or project metadata, or the like. Answers from each tool are integrated into a single system answer containing risk context data for the target DHT, as indicated at step 608.

[0066] Risk management data are then generated based on the risk data and the answers provided by the machine learning model (e.g., the risk context data), as indicated at step 610. In some instances, generating risk management data may include processing the risk data and / or the user input data using a suitably trained machine learning model, which in some examples may be the LLM or other foundation model used to generate answers in response to the prompt contained in the user input data. The generated risk management data can identify sequences of events and hazardous situations specific to the target DHT indicated in the user input data. Additionally or alternatively, initial best guesses for probabilities within a sequence of events can be determined. The risk management data may also include contextual information on probabilities for each sequential event, such as the misdiagnosis rate or how often specific treatments were performed.

[0067] The risk management data can then be displayed to a user, stored for later use or further processing, or both, as indicated at step 612. As described above, in some embodiments the risk management data may be visualized by constructing one or more risk management reports that are displayed to a user or stored for later use. In some embodiments, the risk management report may include one or more risk cards. As described above, the risk management report may include tabular data (e.g., the data presented in Table 1), or may include graphical data, such as one or more risk cards.

[0068] The risk management data may also be provided to the user in the form of a message rather than a comprehensive report. In some embodiments, the message may include a natural language summary of the risk management data that presents key findings in an easily digestible format. The message may include recommendations for risk mitigation strategies based on the analysis performed on the target DHT, providing actionable guidance to development teams. Additionally, the message may contain alerts that identify risks associated 19 QB\630666.01596\97915262.2Mayo 2024-421 Q&B Docket: 630666.01596 with the target DHT. These alerts may be prioritized based on severity levels determined from the risk management data to help users focus on the most critical issues first.

[0069] The message format may incorporate interactive elements that allow the user to request additional information about specific risks without requiring navigation through a full report. In some implementations, the message may be formatted for display on the client device 150 as a notification, enabling real-time communication of risk assessment results. The message may also include a confidence score that indicates the reliability of the risk management data, helping users understand the certainty level of the AI-generated analysis. In some cases, the message may include a score label, such as the score labels described above with respect to FIG. 5. This message-based approach may provide an alternative way to communicate risk information that may be particularly useful for time-sensitive decisions or when users need quick access to key risk insights without reviewing detailed documentation.

[0070] By way of example, the generation and transmission of messages containing risk management data may be facilitated by the coordinated operation of the AI-driven risk management system 100 shown in FIGS.1 and 2. The server 152 may utilize its processor 212 and memory 220 to process the risk management data and generate natural language summaries, recommendations, and alerts using the trained machine learning models stored in memory 220. The processor 212 may execute algorithms that analyze the risk management data to determine severity levels and prioritize alerts accordingly, while also generating confidence scores that indicate the reliability of the AI-generated analysis. The server 152 may format these elements into a structured message that can include interactive elements, notifications, and in some cases, embedded score labels that provide visual risk assessment information.

[0071] The formatted messages may be transmitted from the server 152 to the client device 150 via the communications systems 218 and the communication network 154. Upon receipt through the communications systems 208 of the client device 150, the processor 202 may process the incoming message data and utilize the memory 210 to temporarily store the message content for display processing. The client device 150 may render the message through the display 204, presenting the natural language summary, recommendations, and prioritized alerts in an easily digestible format. The processor 202 may also enable interactive elements within the message interface, allowing users to request additional information through the inputs 206 such as touchscreen interactions or voice commands. When the message includes a score label, the display 204 may present the triangular risk assessment diagram and associated 20 QB\630666.01596\97915262.2Mayo 2024-421 Q&B Docket: 630666.01596 contextual information as an integrated component of the message, providing immediate visual risk assessment without requiring navigation to a separate report interface.

[0072] In some embodiments, the risk management report may also include a score label (e.g., score label 500 shown in FIG.5) that provides a standardized visual representation of the risk management data for the target DHT. As a non-limiting example, the score label may be generated by the server 152 using the processor 212 and memory 220, which process the risk management data to create the visual elements of the score label. The server 152 may execute algorithms that categorize the risk management data into distinct risk domains and assign numerical scores to each domain based on the severity and probability assessments contained within the risk management data. The processor 212 may generate the triangular risk assessment diagram by mapping the categorized risk data to distinct colored sections, with each section representing different aspects of the risk profile for the DHT, such as fairness and equity risks, usefulness and usability risks, and safety and reliability risks.

[0073] The score label may be transmitted from the server 152 to the client device 150 via the communication network 154, where it is displayed to the user through the display 204. The client device 150 may utilize its processor 202 and memory 210 to render the score label interface, including the triangular risk assessment diagram with its colored sections and numerical scores. The display 204 may present the central portion of the diagram showing the total number of unmitigated risks, along with contextual information about the target DHT such as tool name, version number, department, and deployment status. The client device 150 may also display additional sections of the score label that describe the intended use of the DHT and highlight key limitations or contraindications, enabling clinical stakeholders to make rapid decisions about DHT deployment or usage based on the easily interpretable visual format.

[0074] The present disclosure has described one or more preferred embodiments, and it should be appreciated that many equivalents, alternatives, variations, and modifications, aside from those expressly stated, are possible and within the scope of the invention. 21 QB\630666.01596\97915262.2

Claims

Mayo 2024-421 Q&B Docket: 630666.01596 CLAIMS 1. A method for risk management of a digital health technology, the method comprising: receiving user input data via a client device, the user input data comprising an indication of a target digital health technology (DHT) and a user prompt; processing the user input data using a machine learning model, generating risk management data as an output, wherein processing the user input data using the machine learning model comprises: processing the user input data to retrieve risk data associated with the target DHT from a risk library; processing the user prompt with the machine learning model to generate risk context data associated with the DHT; generating the risk management data based on the risk data and the risk context data; generating a risk management report based on the risk management data; and outputting the risk management report to a user via the client device.

2. The method of claim 1, wherein the machine learning model comprises a foundation model.

3. The method of claim 2, wherein the machine learning model comprises a large language model (LLM).

4. The method of claim 3, wherein generating the risk management data comprises inputting the user prompt to the LLM using a retrieval-augmented generation (RAG) framework.

5. The method of claim 4, wherein the RAG framework queries one or more knowledge stores to retrieve data associated with instructions or requirements in the user prompt.

6. The method of claim 1, wherein the risk data comprise hazard definition data associated with the target DHT. 22 QB\630666.01596\97915262.2Mayo 2024-421 Q&B Docket: 630666.01596 7. The method of claim 1 or 6, wherein the risk data comprise harm definition data associated with the target DHT.

8. The method of claim 1, wherein the risk management data comprise a sequence of events and hazardous situations specific to the target DHT.

9. The method of claim 8, wherein generating the risk management data includes inputting the risk data to the machine learning model, generating estimates of probabilities within the sequence of events as an output.

10. The method of claim 9, wherein the risk management data further comprise contextual information on the estimated probabilities for each sequential event in the sequence of events.

11. The method of claim 1, wherein the risk management report comprises tabular data depicting the risk management data.

12. The method of claim 1, wherein the risk management report comprises a risk card visually depicting the risk management data.

13. The method of claim 12, wherein the risk card dynamically displays risk analysis components in the risk management data using a templated display.

14. The method of claim 12, wherein the risk card comprises content linked and traced to other risk elements in the risk management data by digitally enumerating each risk analysis component in the risk management data.

15. The method of claim 14, wherein the risk management report additionally includes tabular data, wherein the tabular data are synchronized with the risk card.

16. The method of claim 15, wherein the tabular data are synchronized with the risk card by mapping associated templates and enumerated content. 23 QB\630666.01596\97915262.2Mayo 2024-421 Q&B Docket: 630666.01596 17. The method of claim 12, wherein the risk card visually depicts risk analysis components comprising hazards, harms, risks, and controls associated with the target DHT.

18. The method of claim 17, wherein the risk analysis components further comprise a sequence of events specific to the target DHT.

19. The method of claim 1, wherein outputting the risk management report comprises generating a message containing the risk management data.

20. The method of claim 19, wherein the message comprises a natural language summary of the risk management data.

21. The method of claim 19, wherein the message comprises recommendations for risk mitigation strategies based on the risk management data.

22. The method of claim 19, wherein the message comprises alerts identifying risks associated with the target DHT.

23. The method of claim 22, wherein the alerts are prioritized based on severity levels determined from the risk management data.

24. The method of claim 19, wherein the message comprises interactive elements allowing the user to request additional information about specific risks.

25. The method of claim 19, wherein the message is formatted for display on the client device as a notification.

26. The method of claim 19, wherein the message comprises a confidence score indicating the reliability of the risk management data.

27. The method of claim 11, wherein the risk management report comprises a score label providing a standardized visual representation of the risk management data. 24 QB\630666.01596\97915262.2Mayo 2024-421 Q&B Docket: 630666.01596 28. The method of claim 27, wherein the score label comprises a risk assessment diagram that visually segments risk categories into distinct colored sections.

29. The method of claim 28, wherein each colored section represents a different aspect of a risk profile for the target DHT and each colored section is assigned a numerical score based on the risk management data.

30. The method of claim 28, wherein the triangular risk assessment diagram includes a first section for fairness and equity risks, a second section for usefulness and usability risks, and a third section for safety and reliability risks.

31. The method of claim 30, wherein the risk assessment diagram has a triangular shape.

32. The method of claim 31, wherein a central portion of the triangular risk assessment diagram displays a total number of unmitigated risks associated with the target DHT.

33. The method of claim 27, wherein the score label includes contextual information about the target DHT comprising at least one of tool name, version number, department, or deployment status.

34. The method of claim 27, wherein the score label includes a description of intended use of the target DHT.

35. The method of claim 27, wherein the score label includes at least one of limitations or contraindications associated with the target DHT.

36. The method of claim 27, wherein the score label comprises hyperlinks to additional content.

37. The method of claim 36, wherein the additional content comprises at least one of underlying tabular data in the risk management data, detailed risk information in the risk management data, instructions for using the target DHT, or a model card for the target DHT. 25 QB\630666.01596\97915262.2

Images