Generation of synthetic patient data

Abstract

A system for generating synthetic patient data is disclosed. The system includes one or more patient data monitoring devices, and a computing device that receives patient data from the one or more patient data monitoring devices. The system receives patient data from one or more patient data monitoring devices. The system analyzes patient data in real time to learn one or more characteristics. The system generates synthetic patient data from the one or more characteristics. The synthetic patient data includes data representative of the patient data different from the patient data. The system stores the synthetic patient data in a database.

Description

[0001] Cross-reference to related applications

[0002] This application was filed on February 26, 2024 as a PCT international application, claiming the benefit and priority of U.S. Provisional Patent Application No. 63 / 489,332, filed on March 9, 2023, the disclosure of which is hereby incorporated by reference. Background Technology

[0003] Developing accurate algorithmic models typically requires a large amount of data related to the model's inputs. Unfortunately, there are many obstacles to collecting data to develop algorithmic models for medical devices in hospital settings, such as privacy concerns, regulatory standards, and data ownership.

[0004] In some cases, these obstacles can be overcome by initiating clinical studies, which allow developers of algorithmic models to collect the necessary data in a hospital setting while meeting the aforementioned requirements. However, such studies can be costly and must be repeated frequently to develop robust algorithmic models. Summary of the Invention

[0005] In general, this disclosure relates to creating synthetic patient data for algorithm development for medical devices while mitigating privacy, regulatory, and data ownership issues. Various aspects are described in this disclosure, including but not limited to the following.

[0006] One aspect relates to a system for generating synthetic patient data, the system comprising: one or more patient data monitoring devices; and a computing device for receiving patient data from one or more patient data monitoring devices, the computing device comprising: at least one processing device; and at least one computer-readable data storage device storing software instructions that, when executed by the at least one processing device, cause the at least one processing device to: receive patient data from one or more patient data monitoring devices; perform real-time analysis of the patient data to learn one or more characteristics; generate synthetic patient data based on one or more characteristics, the synthetic patient data including representative patient data that differs from the patient data; and store the synthetic patient data in a database.

[0007] On the other hand, a method for generating synthetic patient data is involved, the method comprising: receiving patient data from one or more patient data monitoring devices; performing real-time analysis on the patient data to learn one or more characteristics; generating synthetic patient data based on the one or more characteristics, the synthetic patient data including representative patient data that differs from the patient data; and storing the synthetic patient data in a database.

[0008] On the other hand, it relates to a non-transitory computer-readable storage medium comprising computer-readable instructions that, when read and executed by a computing device, cause the computing device to: receive patient data from one or more patient data monitoring devices; perform real-time analysis of the patient data to learn one or more characteristics; generate synthetic patient data based on one or more characteristics, the synthetic patient data including representative patient data that differs from the patient data; and store the synthetic patient data in a database.

[0009] Various additional aspects will be set forth in the following description. These aspects may relate to individual features and combinations of features. It should be understood that the above general description and the following detailed description are merely exemplary and explanatory, and do not limit the broad inventive concepts on which the embodiments disclosed herein are based. Attached Figure Description

[0010] The following figures form part of this application. The figures are illustrations of the described technology and are not intended to limit the scope of this disclosure in any way.

[0011] Figure 1 An example of a system for generating synthetic patient data is shown schematically.

[0012] Figure 2 This is an isometric view of another example of a system used to generate synthetic patient data.

[0013] Figure 3 It shows the result of Figure 2 An example of a thermal image captured by a system's thermal imager.

[0014] Figure 4 It shows the result of Figure 2 Another example of thermal images captured by the system's thermal imager.

[0015] Figure 5 It shows the result of Figure 2 Another example of thermal images captured by the system's thermal imager.

[0016] Figure 6 It schematically illustrates that it can be generated by Figure 1 and Figure 2 This is an example of a system execution method for generating synthetic patient data.

[0017] Figure 7 The illustration shows the method used to achieve this. Figure 1 and Figure 2 Examples of computing hardware components of various aspects of the computing devices used in the system. Detailed Implementation

[0018] Figure 1An example of a system 100 for generating synthetic patient data is schematically illustrated. System 100 includes a computing device 104 that receives patient data 108 from multiple patient data monitoring devices 102a-102n. The computing device 104 learns features and patterns from the patient data 108 and transmits this information to a synthetic generation algorithm (SGA) 112 for generating synthetic patient data 110, without storing the patient data 108 received from the patient data monitoring devices 102a-102n.

[0019] Synthetic patient data 110 includes a distribution of data points that represents patient data 108, but does not include any actual data points from patient data 108, thereby protecting the privacy and integrity of patient data 108. For example, patient data 108 can be input into SGA 112, and SGA 112 outputs synthetic patient data 110 that is similar to but different from patient data 108 and does not exist in the real world.

[0020] As an illustrative example, patient data 108 may include a patient image input into SGA 112. The synthetic patient data 110 generated by SGA 112 looks similar to the patient image of patient data 108, but is different and does not exist in the real world.

[0021] As another illustrative example, patient data 108 may include patient position data input to SGA 112. Synthetic patient data 110 generated by SGA 112 is similar to, but different from, the patient position data and does not exist in the real world. In this example, the synthetic patient data 110 is stored and can be used to train additional algorithms, such as algorithms for monitoring patient position and / or patient sleep patterns.

[0022] Illustrative examples of SGA 112 may include, but are not limited to, generative adversarial networks (GANs), transformers, autoencoders, stream-based generative models, and other types of artificial intelligence algorithms, including machine learning frameworks / models.

[0023] In some examples, computing device 104 includes an edge computing device that processes patient data 108 in proximity to patient data monitoring devices 102a-102n, without using a cloud server, to improve response time and save bandwidth. This also ensures the privacy and integrity of patient data 108. Computing device 104 processes patient data 108 in real time to generate synthetic patient data 110.

[0024] Once synthetic patient data 110 is generated, computing device 104 stores it in database 106. This allows synthetic patient data 110 to be used to develop one or more algorithms, including artificial intelligence algorithms, including machine learning algorithms as a subset of artificial intelligence algorithms. For example, instead of actual patient data detected by multiple patient data monitoring devices 102a-102n, synthetic patient data 110 can be configured as training data for building machine learning models. Synthetic patient data 110 output from SGA 112 can be stored alone or in combination with real-world data for later use to create new algorithms, improve existing algorithms, or generate new synthetic patient data.

[0025] Synthetic patient data 110 is not combined with actual patient data, nor does it augment the actual patient data in database 106. In this way, synthetic patient data 110 allows for the development of artificial intelligence algorithms while maintaining the privacy of patient data 108 collected by patient data monitoring devices 102a-102n.

[0026] Patient data monitoring devices 102a-102n may include one or more sensors that measure physiological parameters of a patient. The one or more sensors may include sensors of any type, including sensors implanted in the patient's body, sensors in contact with the patient's body (but not implanted), and / or non-contact sensors.

[0027] Illustrative examples of patient data monitoring devices 102a-102n may include, but are not limited to: cameras that capture images and / or video streams of patients; pulse oximeters that indirectly monitor a patient's blood oxygen saturation (SpO2); radar that detects one or more vital signs of a patient (such as heart rate and respiratory rate) without contact with the patient's body; electrocardiographs (ECGs) that record cardiac electrical activity; ballistic cardiac angioplasty (BCG) devices that measure the ballistic forces generated by the heart from the patient's body surface using non-invasive methods; pressure sensors for capturing body pressure maps that measure the pressure distribution between the patient's body and a supporting surface such as a mattress on a hospital bed; sleep tracking sensors for measuring a patient's sleep patterns; microphones for capturing patient audio (i.e., speech); laser and / or infrared light emitters; thermal imagers for capturing infrared thermal imaging (IRT), thermal video, and / or thermal images of patients; fundus imagers and vision screening devices; infusion pumps; hemodialysis and peritoneal dialysis systems; analgesic delivery pumps; patient safety alerts; and so on. Figure 2 In one example shown, one or more patient data monitoring devices 102 are combined with computing device 104 in a single device package.

[0028] Figure 2This is an isometric view of another example of a system 200 for generating synthetic patient data. System 200 is a multimodal system that integrates one or more patient data monitoring devices with computing device 104 into a single device package. In this example, system 200 includes an RGB-D camera 202 and a thermal imager 204. In other examples, system 200 may include other types of patient data monitoring devices. Figure 2 In the example shown, system 200 includes a microphone and speaker unit 206 for bidirectional communication between the patient and a caregiver at a remote location.

[0029] RGB-D camera 202 is a camera that provides depth and RGB data as real-time output by providing pixel-to-pixel combination of color (RGB) data and depth (D) information to convey both in a single frame. Thermal imager 204 captures infrared thermal imaging (IRT), thermal video, and / or thermal images of patients, examples of which are shown below. Figures 3 to 5 As shown. The image generated by the RGB-D camera 202 and the thermal imager 204 is an example of patient data 108. As described above, the computing device 104 learns features and patterns from the patient data 108 and transmits this information to the SGA 112 that generates synthetic patient data 110, without storing the patient data 108 received from the RGB-D camera 202 and the thermal imager 204.

[0030] System 200 is positioned so that the RGB-D camera 202 and thermal imager 204 are within the patient's line of sight. Figure 1 Similarly, the computing device 104 of system 200 processes patient data captured by RGB-D camera 202 and / or thermal imager 204 without using the computing power of a cloud server to improve response time and save bandwidth, while also ensuring the privacy and integrity of patient data. The computing device 104 processes the patient data captured by RGB-D camera 202 and / or thermal imager 204 in real time to generate synthetic patient data.

[0031] Figure 3 An example of a thermal image 300 captured from patient P by thermal imager 204 of system 200 is shown. Thermal image 300 includes patient data 108. Thermal image 300 can be captured continuously, thereby allowing continuous measurement of patient P's temperature without contact with the patient. In this example, the temperature is measured from a region within thermal image 300, such as the corner of the eye where the upper and lower eyelids meet. System 200 can use thermal image 300 to generate synthetic temperature measurements without storing thermal image 300 or the patient data 108 associated with thermal image 300, thus protecting patient P's privacy.

[0032] Figure 4Another example of a thermal image 400 captured from patient P by thermal imager 204 of system 200 is shown. Thermal image 400 includes patient data 108. In this example, thermal image 400 includes a first frame 402 captured during patient P's exhalation and a second frame 404 captured during patient P's inhalation. Therefore, system 200 can continuously measure patient P's respiratory rate using thermal image 400 without contact with the patient. System 200 can use thermal image 400 to generate synthetic respiratory rate measurements without storing thermal image 400 or the patient data 108 associated with thermal image 400, thereby protecting patient P's privacy.

[0033] Furthermore, thermal images 300 and 400 can be used by system 200 to measure the hemodynamics of patient P (i.e., how blood flows through patient P's arteries and veins) to predict cardiovascular disease. System 200 can use thermal images 300 and 400 to generate synthetic hemodynamic data without storing the thermal images or patient data 108 associated with the thermal images, thereby protecting patient P's privacy.

[0034] Figure 5 Another example of a thermal image 500 captured by thermal imager 204 of system 200 from patient P is shown. In this example, the thermal image 500 can be used to continuously measure the patient P's position when the patient P's body is at least partially obscured by one or more objects (such as a blanket). Therefore, system 200 can use the thermal image 500 to continuously measure the patient P's position without contact with the patient. System 200 can use the thermal image 500 to generate synthetic patient position data without storing the original thermal image to protect the patient P's privacy. In one example implementation, computing device 104 can use the thermal image 500 derived from thermal imager 204 to generate a synthetic image.

[0035] Figure 6 An example of a method 600 for generating synthetic patient data, which can be executed by computing device 104, is illustrated schematically. The synthetic patient data can be used as training data to build one or more algorithms, such as artificial intelligence and machine learning models. Synthetic patient data allows for the construction of one or more algorithms while maintaining the privacy of protected health information (PHI) associated with the patient data, and does not require the transfer of ownership of the patient data or the leasing of the patient data.

[0036] Method 600 includes an operation 602 of receiving patient data. In some examples, patient data is received from one or more patient data monitoring devices 102a-102n (see...). Figure 1 In some examples, operation 602 includes receiving patient data as images generated by RGB-D camera 202 and thermal imager 204 (see...). Figures 2 to 5 ).

[0037] Upon receiving patient data, method 600 includes operation 604, which involves analyzing the patient data in real time without storing it. Operation 604 may include learning one or more characteristics, such as features and / or patterns, from the patient data.

[0038] Examples of features that can be learned from the patient's video data in operation 604 can vary. For example, SGA 112 can learn to exclude features from the video data that identify the patient, including patient boundaries (e.g., the location where the patient ends and the bed begins in the video data) and / or the location of one or more anatomical features of the patient, such as the patient's eyes, head, arms, torso, hair, nose, mouth, etc. SGA 112 can also learn motion-related features from the video data, such as getting out of bed, opening and closing of the patient's eyes, eating and drinking, chest movements during breathing, and other patient movements.

[0039] Features can also be learned from audio data and other types of audio, including segmenting background noise and sound patterns (such as waveform, pitch, amplitude, volume, dialect, stuttering, etc.) from target audio. Features can also be learned from non-communicative audio, such as coughing, snoring, and breathing sounds. Furthermore, features can be learned from non-human sounds (such as warnings). Based on these types of data, SGA 112 can create synthetic patient data 110, such as images, videos, or audio, which does not replicate patient data 108 obtained from real people, such as patients.

[0040] Method 600 includes an operation 606 that generates synthetic patient data based on the patient data analyzed in operation 604. For example, operation 606 may include inputting one or more features, such as features and / or patterns learned in operation 604, into SGA 112 to generate synthetic patient data.

[0041] In one example implementation, operation 606 may include generating synthetic patient data by inputting a distribution obtained from a radar-based non-contact sensor that measures one or more vital signs of the patient, such as heart rate, respiratory rate, and other vital signs. In another example implementation, patient data collected from a pulse oximeter is used to generate a synthetic SpO2 waveform. In another example implementation, synthetic patient data may be generated by learning patient safety warnings and related aspects such as warning of disturbances and fatigue. In another example implementation, operation 606 may include generating a multimodal synthetic patient dataset by inputting patient data such as: patient data obtained from multiple sensors, such as RGB-D camera 202; patient data obtained from thermal imager 204; body wireframe data; audio (including speech recognition, natural language processing (NLP), emotion assessment, etc.); patient data obtained from a radar-based non-contact sensor; patient data obtained from a pulse oximeter; patient data obtained from an ECG; patient data obtained from a BCG; infrared imaging, barometric pressure, and thermograms; and so on.

[0042] Next, method 600 includes an operation 608 of storing synthetic patient data in database 106. This allows the synthetic patient data to be used as training data for building one or more algorithmic models, including artificial intelligence and machine learning models. As an example, synthetic patient data can be used to build improved algorithmic models to predict early deterioration of a patient's condition. As another example, synthetic patient data can be used to build improved early warning algorithms, for example, for use with one or more types of medical devices.

[0043] Figure 7 An example of a computing hardware component 700 that can be used to implement various aspects of computing device 104 is shown schematically. The computing hardware component 700 includes a processing unit 702, a system memory 708, and a system bus 720 that couples the system memory 708 to the processing unit 702.

[0044] Processing unit 702 may include one or more processing devices such as a central processing unit (CPU). In some illustrative examples, processing unit 702 includes one or more CPUs, microcontrollers, digital signal processors, field-programmable gate arrays, and / or other types of programmable electronic circuits.

[0045] System memory 708 includes random access memory (“RAM”) 710 and read-only memory (“ROM”) 712. Basic input / output logic containing basic routines is stored in ROM 712, which facilitates the transfer of information between elements within computing hardware component 700, such as during startup.

[0046] The computing hardware component 700 may also include a mass storage device 714 capable of storing software instructions and data. The mass storage device 714 is connected to the processing unit 702 via a system bus 720. The mass storage device 714 and its associated computer-readable data storage medium provide non-volatile, non-transitory computer-readable data storage for the computing hardware component 700.

[0047] Although the description of computer-readable data storage medium contained herein refers to mass storage device 714, those skilled in the art will understand that a computer-readable data storage medium can be any available non-transitory physical device or article of art from which data and / or instructions can be read. Mass storage device 714 is an example of a computer-readable storage device.

[0048] Computer-readable data storage media include volatile and non-volatile, removable and non-removable media, implemented in any way or by any technology, for storing information such as computer-readable software instructions, data structures, program modules or other data. Examples of computer-readable data storage media include, but are not limited to, RAM, ROM, EPROM, EEPROM, flash memory or other solid-state memory technologies, or any other medium that can be used to store data accessible to computing device 104.

[0049] The computing hardware component 700 can operate in a network environment using logical connections to remote network devices. For example, the computing hardware component 700 can be connected to a communication network 120, such as a local area network, the Internet, or another type of computing network, via a network interface unit 704 connected to the system bus 720. The network interface unit 704 can also be used to connect to other types of networks and remote computing systems.

[0050] The computing hardware component 700 may also include an input / output controller 706 for receiving and processing input from multiple input devices. Similarly, the input / output controller 706 may provide output to multiple output devices.

[0051] Mass storage device 714 and / or RAM 710 may be used to store software instructions 716, operating system 718, and data for controlling the operation of computing device 104. When processing unit 702 executes software instructions 716, it causes computing hardware component 700 to perform the functions described herein.

[0052] The various embodiments described above are provided by way of example only and should not be construed as limiting in any way. Various modifications can be made to the above embodiments without departing from the true spirit and scope of this disclosure.

Claims

1. A system for generating synthetic patient data, the system comprising: One or more patient data monitoring devices; as well as A computing device that receives patient data from one or more patient data monitoring devices, the computing device comprising: At least one processing device; and At least one computer-readable data storage device stores software instructions that, when executed by the at least one processing device, cause the at least one processing device to: Receive patient data from one or more patient data monitoring devices; The patient data is analyzed in real time to learn one or more characteristics; Synthetic patient data is generated based on one or more of the aforementioned characteristics, the synthetic patient data including data representing the patient data that is different from the patient data itself; and The synthetic patient data is stored in a database.

2. The system according to claim 1, wherein, When the software instructions are executed by the at least one processing device, the at least one processing device also causes the at least one processing device to: One or more features of the patient data are input into an artificial intelligence algorithm to generate the synthetic patient data, the artificial intelligence algorithm being stored in the at least one computer-readable data storage device.

3. The system according to claim 1, wherein, The synthetic patient data is configured as training data for building machine learning models.

4. The system according to claim 1, wherein, The one or more patient data monitoring devices include a thermal imager packaged together with the computing device in a single device.

5. The system according to claim 4, wherein, When the software instructions are executed by the at least one processing device, the at least one processing device also causes the at least one processing device to: Receive thermal images captured by the thermal imager; and The synthetic patient data is generated using the thermal images.

6. The system according to claim 5, wherein, The synthetic patient data includes at least one of synthetic temperature measurements, synthetic respiratory rate measurements, and synthetic patient position data.

7. The system according to claim 1, wherein, When the software instructions are executed by the at least one processing device, the at least one processing device also causes the at least one processing device to: The synthetic patient data is generated based on patient data obtained from radar-based non-contact sensors.

8. The system according to claim 1, wherein, When the software instructions are executed by the at least one processing device, the at least one processing device also causes the at least one processing device to: A multimodal synthetic patient dataset is generated based on patient data obtained from multiple patient data monitoring devices.

9. A method for generating synthetic patient data, the method comprising: Receive patient data from one or more patient data monitoring devices; The patient data is analyzed in real time to learn one or more characteristics; Synthetic patient data is generated based on one or more of the aforementioned characteristics, wherein the synthetic patient data includes data representing the patient data that is different from the patient data itself; as well as The synthetic patient data is stored in a database.

10. The method of claim 9, further comprising: One or more features of the patient data are input into an artificial intelligence algorithm to generate the synthetic patient data.

11. The method according to claim 9, further comprising: Receive thermal images; as well as The synthetic patient data is generated using the thermal images.

12. The method according to claim 11, wherein, The synthetic patient data includes at least one of synthetic temperature measurements, synthetic respiratory rate measurements, and synthetic patient position data.

13. The method of claim 9, further comprising: The synthetic patient data is generated based on patient data obtained from radar-based non-contact sensors.

14. The method according to claim 9, further comprising: A multimodal synthetic patient dataset is generated based on patient data obtained from multiple patient data monitoring devices.

15. The method according to claim 9, wherein, The synthetic patient data is configured as training data for building machine learning models.

16. A non-transitory computer-readable storage medium, the non-transitory computer-readable storage medium comprising computer-readable instructions, which, when read and executed by a computing device, cause the computing device to: Receive patient data from one or more patient data monitoring devices; The patient data is analyzed in real time to learn one or more characteristics; Synthetic patient data is generated based on one or more of the aforementioned characteristics, the synthetic patient data including data representing the patient data that is different from the patient data itself; and The synthetic patient data is stored in a database.

17. The non-transitory computer-readable storage medium of claim 16, further comprising computer-readable instructions that, when read and executed by the computing device, cause the computing device to perform the following operations: Learn one or more characteristics of the patient data; and One or more features of the patient data are input into an artificial intelligence algorithm to generate the synthetic patient data.

18. The non-transitory computer-readable storage medium of claim 16, further comprising computer-readable instructions that, when read and executed by the computing device, cause the computing device to perform the following operations: The synthetic patient data is generated based on patient data obtained from radar-based non-contact sensors.

19. The non-transitory computer-readable storage medium of claim 16, further comprising computer-readable instructions that, when read and executed by the computing device, cause the computing device to perform the following operations: A multimodal synthetic patient dataset is generated based on patient data obtained from multiple patient data monitoring devices.

20. The non-transitory computer-readable storage medium of claim 16, further comprising computer-readable instructions that, when read and executed by the computing device, cause the computing device to perform the following operations: Receive thermal images captured by a thermal imager; and The synthetic patient data is generated using the thermal images.

Images