System and method for artificial intelligence-based blood pressure calculation based on extraocular images

Abstract

The present disclosure relates to calculating blood pressure from images of an individual's external eye. Images of an individual's external eye acquired via a high magnification camera can be analyzed using computer vision to identify features associated with the blood vessels of the external eye, such as blood vessel size or diameter, blood vessel wall thickness, distance between blood vessels or vessel segments, area between blood vessels or vessel segments, and / or blood flow velocity through the blood vessels. These vascular features derived from the images can be used to calculate a blood pressure measurement for the individual by using an artificial intelligence algorithm that relates the vascular features to blood pressure values.

Description

[Technical field]

[0001] This application claims the benefit of U.S. Provisional Patent Application No. 63 / 319,738, entitled "SYSTEMS AND METHODS FOR REMOTE AND AUTOMATED MEDICAL DIAGNOSIS," filed March 14, 2022, which is incorporated by reference in its entirety. In addition, this application claims priority to, and is a divisional application of, U.S. Design Application No. 29 / 830,662, entitled "CONSUMER ELECTRONICS DEVICE," filed March 14, 2022, which is incorporated by reference in its entirety.

[0002] The present disclosure relates generally to consumer electronics / medical devices. [Background technology]

[0003] Blood pressure is an important health measurement for diagnosing various illnesses. When combined with other signs and symptoms and / or other health sensor data, it can help narrow down potential illnesses for differential evaluation and diagnosis by a physician. Traditional blood pressure determinations by manual or "automated" blood pressure monitors are prone to inaccuracies or errors due to factors such as user expertise / error, and improper cuff size, fit, and placement. The process of taking a blood pressure measurement can be time-consuming and may take several minutes considering the time from setup (applying the cuff and stethoscope, configuring the automatic blood pressure monitor, etc.) to the time the final reading is observed, which may not be ideal in certain situations (e.g., emergency situations). Moreover, the large and bulky nature of blood pressure cuffs can be inconvenient to carry and can make it difficult to properly place the cuff on an individual. Summary of the Invention

[0004] The present invention relates, in part, to automating the collection and interpretation of a person's unique health data via a touchless device and accurately determining their blood pressure via computer vision and / or artificial intelligence (AI). The inventive concepts disclosed herein can save time in determining blood pressure and prevent errors that can occur in traditional determinations of blood pressure via manual or "automated" methods.

[0005] One novel approach to blood pressure measurement described herein involves acquiring images of an individual's external eye using a high magnification camera and processing the images using computer vision and AI analytics algorithms. The computer vision algorithm can determine at least one characteristic of the ocular vasculature present in the image, including, but not limited to, vascular characteristics such as vessel diameter, vessel wall thickness, distance between vessels or vessel segments, area between vessels or vessel segments, and blood flow velocity through at least one vessel or vessel segment. The image-derived characteristics can be used in an AI analytics algorithm trained to calculate blood pressure based on the image-derived characteristics. In one aspect, changes in image characteristics (e.g., % change from baseline or a previous time point) are used by the AI ​​analytics algorithm to calculate the current blood pressure. The AI ​​analytics algorithm may be unique to each individual whose blood pressure is measured, providing the advantage of more accurate blood pressure calculations because the algorithm is designed around the specific characteristics of each individual as opposed to a generalized model that may not account for the unique characteristics of different individuals. The unique AI analytics algorithm can be trained using a combination of an individual's external eye images and corresponding blood pressure measurements acquired at approximately the same time (or at the same time) as the images.

[0006] Currently, there are no known conventional methods for measuring blood pressure that rely on imaging-derived vascular measurements of the extraocular vasculature. This method enables non-contact, real-time blood pressure calculations from computer vision and AI processing of extraocular images whose existence is unknown to conventional technology.

[0007] Further details of the invention include at least the following:

[0008] The invention may include a computer-implemented method for providing feedback regarding an individual's blood pressure using analysis of sensor data and artificial intelligence (AI) analysis of images of the individual's external eye, the computer-implemented method including: receiving at a remote server via network communication with a handheld end-user electronic device a series of digital images of at least a portion of the individual's external eye, the handheld end-user electronic device including a high magnification camera operable to capture at least one of images and videos of blood flowing through the vascular system of the external eye; calculating, by a processor, using at least one computer vision algorithm, vascular measurements of at least one blood vessel within the portion of the individual's external eye, the computer vision algorithm extracting pixel value information from at least one pixel for each image of the series of digital images and processing the pixel value information to obtain a vascular measurement of at least one blood vessel within the portion of the individual's external eye. determining at least one image characteristic associated with the series of images, the at least one image characteristic including at least one of a blood vessel diameter, a blood vessel wall thickness, a blood vessel length, a distance between the vessels or vessel segments, an area between the vessels or vessel segments, and a blood flow velocity measurement associated with the at least one blood vessel, wherein the vascular measurement is calculated based on the at least one image characteristic; calculating, by the processor, a blood pressure measurement for the individual using an AI analytics algorithm that relates the vascular measurements to blood pressure values, the AI ​​analytics algorithm being trained using previously obtained and processed images of the individual's extraocular vasculature and previously obtained blood pressure measurements associated with the individual; and providing, via network communication, a feedback result to the handheld end-user electronic device that is displayed on the end-user electronic device, the feedback result including at least the calculated blood pressure measurement.

[0009] The computer-implemented method can include having a computer vision algorithm trained with labeled images including labeled pixels associated with at least one of a blood vessel, a blood vessel wall, a blood flow area, an inner vessel diameter, an outer vessel diameter, a distance between blood vessels or vessel segments, and a distance or area between blood vessels or vessel segments.

[0010] The computer-implemented method may include the computer vision algorithm being operable to determine at least one measure for the series of images, the at least one measure including minimum vessel diameter, maximum vessel diameter, average vessel diameter, minimum vessel wall thickness, maximum vessel wall thickness, average vessel wall thickness, minimum blood flow velocity, maximum blood flow velocity, average blood flow velocity, distance between vessels or vessel segments, and area between vessels or vessel segments.

[0011] The computer-implemented method may include a computer vision algorithm operable to perform at least one of object detection, edge detection, video tracking, object recognition, 3D pose estimation, and motion estimation.

[0012] The computer-implemented method can include the AI ​​analytics algorithm calculating an amount of change in the at least one image characteristic by comparing at least one current image characteristic with at least one previously calculated image characteristic associated with the individual, and calculating a blood pressure measurement based on the amount of change.

[0013] The computer-implemented method can include the AI ​​analysis algorithm being trained using previously acquired and processed images of the individual's external eye taken in combination with previous blood pressure measurements associated with the individual.

[0014] The computer-implemented method may include that the previously acquired image and the previously acquired blood pressure measurement are acquired simultaneously.

[0015] The computer-implemented method may include previously acquired images and previously acquired blood pressure measurements being acquired in combination with one another at a number of different time points.

[0016] The computer-implemented method can include, where blood flow velocity is determined by tracking at least one pixel within a specified region of interest, the region of interest being a region of a specified size.

[0017] The computer-implemented method may include the region of interest being a fixed size and located at a specified location within the image, the size of the region of interest being less than the total size of each image.

[0018] The computer implemented method may include a series of images or a series of time-stamped images.

[0019] The computer implemented method may include receiving the images in real time as they are acquired by the handheld end-user electronic device and providing feedback results to the handheld end-user electronic device in real time as the blood pressure measurements are calculated.

[0020] The computer-implemented method may include converting and / or communicating the feedback results in a format suitable for display on a handheld end-user electronic device.

[0021] The computer-implemented method may include a high magnification camera operable to be connected to or integral to the handheld end-user electronic device.

[0022] The computer-implemented method may include at least one of the AI ​​analytics algorithm and the computer vision algorithm being trained using at least one of a linear regression, a logistic regression, a decision tree, a random forest algorithm, a support vector machine, a Naive Bayes algorithm, a random walk algorithm, a k-nearest neighbor algorithm, k-means clustering, and a Markov model.

[0023] The computer-implemented method may include at least one of the AI ​​analytics algorithm and the computer vision algorithm being trained using at least one of supervised learning, unsupervised learning, semi-supervised learning, and reinforcement learning.

[0024] The computer-implemented method may include the vascular measurement being calculated using one or a combination of the image characteristics.

[0025] The invention may include a computing system for artificial intelligence (AI) analysis of images of an individual's external eye to calculate a blood pressure of the individual, the computing system comprising: at least one computing processor; and a memory comprising instructions, which when executed by the at least one computing processor, cause the computing system to receive, at a remote server via network communication with a handheld end-user electronic device, the handheld end-user electronic device comprising a high magnification camera operable to capture at least one of images and videos of blood flowing through the vascular system of the external eye; and calculate, by the processor, a vascular measurement of at least one blood vessel within the portion of the individual's external eye using at least one computer vision algorithm, the computer vision algorithm comprising: a memory comprising instructions comprising: extracting pixel value information from at least one pixel per image of the digital images and processing the pixel value information to determine at least one image characteristic associated with the series of images, the at least one image characteristic including at least one of blood vessel diameter, blood vessel wall thickness, distance between blood vessels or blood vessel segments, area between blood vessels or blood vessel segments, and blood flow velocity measurements associated with the at least one blood vessel; calculating, by the processor, a blood pressure measurement for the individual using an AI analytics algorithm that relates the vascular measurements to blood pressure values, the AI ​​analytics algorithm being trained using previously obtained and processed images of the individual's extraocular vasculature and previously obtained blood pressure measurements associated with the individual; and providing, via network communication, a feedback result to the handheld end-user electronic device that is displayed on the end-user electronic device, the feedback result including at least the calculated blood pressure measurement.

[0026] The invention may include a non-transitory computer readable medium comprising instructions that, when executed by a processor, enable the processor to perform a method of calculating blood pressure based on AI analysis of images of an individual's external eye, the method including: receiving at a remote server via network communication with a handheld end-user electronic device a series of digital images of at least a portion of the individual's external eye, the handheld end-user electronic device including a high magnification camera operable to capture at least one of images and videos of blood flowing through the vascular system of the external eye; calculating, by the processor, using at least one computer vision algorithm, a vascular measurement for at least one blood vessel within the portion of the individual's external eye, the computer vision algorithm generating pixel value information from at least one pixel for each image of the series of digital images. calculating, by the processor, a blood pressure measurement for the individual using an AI analytics algorithm that relates the vascular measurements to blood pressure values, the AI ​​analytics algorithm being trained using previously acquired and processed images of the individual's extraocular vasculature and previously obtained blood pressure measurements associated with the individual; and providing, via network communication, feedback results to the handheld end-user electronic device for display on the end-user electronic device, the feedback results including at least the calculated blood pressure measurements.

[0027] The accompanying drawings illustrate several embodiments and, together with the description, serve to explain the principles of the invention by way of example. Those skilled in the art should appreciate that the specific arrangements illustrated in the drawings are merely exemplary and should not be considered as limiting the scope of the invention or the claims herein. [Brief description of the drawings]

[0028] [Figure 1] 1 is a diagram of an environment for remote and / or automated medical diagnostics, according to an example embodiment of the present invention. [Diagram 2] 1 is a diagram of an example medical / consumer electronic device, in accordance with an exemplary embodiment of the present invention. [Diagram 3] 1 is a diagram of an example telemedicine platform, according to an exemplary embodiment of the present invention. [Figure 4] 1 is a flow chart for remote and / or automated medical diagnosis, according to an example embodiment of the present invention. [Diagram 5] FIG. 2 is a component diagram of an exemplary computing device supporting one embodiment of the present disclosure. [Figure 6] FIG. 1 is a diagram of one embodiment of a standalone computing system supporting one embodiment of the present disclosure. [Figure 7] FIG. 1 is a diagram of an exemplary distributed computing network supporting an exemplary embodiment of the present disclosure. [Figure 8] FIG. 1 is an exemplary external view of a computer system supporting an exemplary embodiment of the present disclosure. [Figure 9] 1 is a flow chart for remote and / or automated medical diagnosis and care delivery, according to an example embodiment of the present invention. [Figure 10] FIG. 1 is an exemplary schematic diagram of a process for calculating blood pressure based on an image of an eye, according to an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0029] The systems and methods of the invention described herein (hereinafter sometimes more conveniently referred to as "systems" or "methods") facilitate remote and / or automated medical diagnosis. Specifically, a medical / consumer electronic device with multiple sensors can receive information from at least two sensors. The medical / consumer electronic device can transmit the information received from the at least two sensors over a network to a user device, which can further transmit the data to a remote server hosting a remote analysis platform. The remote server can use artificial intelligence (AI) and the information received from the at least two sensors to determine a diagnosis, urgency, and / or recommendation, and transmit a signal indicative of the determined diagnosis, urgency, and / or recommendation over the network to the medical / consumer electronic device. The medical / consumer electronic device can display the determined diagnosis, urgency, and / or recommendation. The invention reduces the time and costs associated with receiving a medical diagnosis. The invention facilitates medical diagnosis without exposing health care workers to potentially infectious patients and without exposing users of the invention to potentially infectious patients or environments in public health care settings.

[0030] One or more different embodiments may be described in this application. Furthermore, numerous alternative arrangements may be described for one or more of the embodiments described herein. It should be appreciated that these are presented for illustrative purposes only and do not limit the scope of the embodiments contained herein or the claims presented herein. As may be readily apparent from this disclosure, one or more of the arrangements may be broadly applied to numerous embodiments. In general, the arrangements have been described in sufficient detail to enable one skilled in the art to practice one or more of the embodiments, and it should be appreciated that other arrangements may be utilized and that structural, logical, software, electrical, and other changes may be made without departing from the scope of the embodiments. One or more specific features of the embodiments described herein may be described with reference to one or more specific embodiments or drawings that form a part of this disclosure, in which one or more specific arrangements of the aspects are shown for illustrative purposes. It should be appreciated, however, that such features are not limited to use in one or more specific embodiments or drawings referenced in the description. The present disclosure is not a literal description of every configuration of one or more of the embodiments, nor is it a list of one or more features of the embodiments that must be present in all configurations.

[0031] The section headings contained herein and the title of this patent application are for convenience only and are not to be construed as limiting the disclosure in any way.

[0032] Devices that are in communication with each other need not be in continuous communication with each other unless explicitly specified otherwise. In addition, devices that are in communication with each other may communicate directly or indirectly through one or more communication means or intermediaries, whether logical or physical.

[0033] The description of an embodiment involving several components in communication with each other does not imply that all such components are required. To the contrary, various optional components may be described to illustrate a variety of possible embodiments and to more completely describe one or more embodiments. Similarly, although process steps, method steps, algorithms, and the like may be described in a sequential order, such processes, methods, and algorithms may generally be configured to operate in alternative orders unless specifically stated otherwise. In other words, the sequence or order of steps that may be described in this patent application does not, in itself, indicate a requirement that the steps be performed in that order. Steps of a described process may be performed in any order that is practical. Furthermore, some steps may be performed simultaneously, even though they are described or implied as occurring non-concurrently (e.g., because one step is described after another). Moreover, the description of a process by depiction in a drawing does not imply that the illustrated process is exclusive of other variations and modifications thereto, does not imply that the illustrated process or any of its steps are required for one or more of the embodiments, and does not imply that the illustrated process is preferred. Also, although a step may generally be described once per embodiment, this does not imply that it must occur once or that it may occur only once each time the process, method, or algorithm is performed or executed. Some steps may be omitted in some embodiments or occurrences, or some steps may be performed multiple times in a given embodiment or occurrence.

[0034] Where a single device or article is described herein, it will be readily apparent that two or more devices or articles may be used in place of the single device or article. Similarly, where two or more devices or articles are described herein, it will be readily apparent that the single device or article may be used in place of the two or more devices or articles.

[0035] The functionality or features of a device may alternatively be embodied by one or more other devices not explicitly described as having such functionality or features, and thus other embodiments need not include the device itself.

[0036] Techniques and mechanisms described or referenced herein may be described in the singular for clarity. However, it should be appreciated that a particular embodiment may include multiple iterations of a technique or multiple instantiations of a mechanism unless otherwise noted. The process illustrations or blocks in the figures should be understood to represent modules, segments, or portions of code that include one or more executable instructions for implementing a particular logical function or step in the process. Alternative implementations are included within the scope of various embodiments, and as will be appreciated by those skilled in the art, such embodiments may perform functions in an order different from that shown or discussed, including substantially simultaneously or in reverse order, depending, for example, on the functionality involved.

[0037] Conceptual Architecture 1 illustrates an environment for remote and / or automated medical diagnosis according to an exemplary embodiment of the present invention. The environment includes medical / consumer electronic devices 140, an analysis system 120, a data repository 130, a network 150, a user device 160, a telemedicine platform 170, and a third-party stakeholder platform 180. The various computing devices described herein are exemplary and for purposes of illustration only. As will be appreciated by those skilled in the art, the system may be rearranged or integrated to perform the same tasks on one or more other servers or computing devices without departing from the scope of the present invention.

[0038] The medical / household electronic device 140 may include a plurality of sensors. The medical / household electronic device 140 may include a browser for accessing web applications hosted on the analysis system 120. The medical / household electronic device 140 may include an application for interacting with web applications hosted on the analysis system 120. The medical / household electronic device 140 may include an application obtained from the analysis system 120. The medical / household electronic device 140 may transmit data collected from a plurality of sensors to the analysis system 120 via the network 150. The medical / household electronic device 140 may receive a diagnosis, urgency, and / or recommendation from the analysis system 120 via the network 150. The medical / household electronic device 140 may include a device used by a consumer at home. The medical / household electronic device 140 may include sensors as an attachment to a user device such as a laptop or a smartphone. The medical / household electronic device 140 may include a device used in a medical institution. For example, the medical / consumer electronic device 140 may suggest to a physician which medications to prescribe to the patient, which specialists to refer the patient to, which imaging or other diagnostic procedures to suggest to the patient, admitting the patient to a hospital or emergency room, etc. The medical / consumer electronic device 140 may include a device for collecting biometrics to determine whether an individual should be given access to an area. The medical / consumer electronic device 140 may forward data to a remote security server (not shown) for biometric matching. The medical / consumer electronic device 140 may retrieve data from the remote security server for biometric matching. The medical / consumer electronic device 140 may be used in advertising to determine that a user is located in front of a digital billboard (not shown). The medical / consumer electronic device 140 may identify the user and / or characteristics of the user and facilitate targeted advertising to the user based on the identified user and / or the identified characteristics. The medical / consumer electronic device 140 may forward data to a remote advertising server (not shown) for targeted advertising.The medical / consumer electronic device 140 can retrieve data from a remote ad server for targeted advertising. The medical / consumer electronic device 140 can cause a digital billboard to display targeted advertising for the user. The medical / consumer electronic device 140 is described in more detail with reference to FIG. 2. The medical / consumer electronic device 140 can include sensors as wired or wireless attachments to a user device such as a laptop, tablet, or smartphone. The medical / consumer electronic device / advertising / security device can include devices for collecting biometrics to determine whether an individual should be given access to an area. The medical / consumer electronic device / advertising / security device can be used in advertising to determine that a user is located in front of a digital billboard.

[0039] The analysis system 120 may include a remote server. The analysis system 120 may include one or more computing devices. The analysis system 120 may include a cloud computing environment. The analysis system 120 may host a website. The analysis system 120 may host a web application. The analysis system 120 may provide an application, such as a mobile application, for download by the medical / consumer electronic device 140 via the network 150. The analysis system 120 may receive information from sensors from the medical / consumer electronic device 140 via the network 150. The analysis system 120 may determine a diagnosis, urgency, and / or recommendation based on the information received from the sensors. The analysis system 120 may transmit the determined diagnosis, urgency, and / or recommendation to the medical / consumer electronic device 140 via the network 150. The analysis system 120 may receive data, such as artificial intelligence (AI) training data, from the data repository 130 via the network 150. Analysis system 120 can store data received from a particular medical / consumer electronic device, as well as associated determined diagnoses, urgency, and / or recommendations, in data repository 130 via network 150. Analysis system 120 is described in more detail with reference to FIG. 3. The analysis system can transmit or receive data, such as artificial intelligence training data or analysis data or results (e.g., without limitation, diagnoses, urgency, and / or recommendations), to or from the data repository via the network. The analysis system can transmit (wired or wirelessly) the determined diagnoses, urgency, and / or recommendations to the medical device or devices connected thereto via the network.

[0040] The data repository 130 may include data storage. The data repository 130 may include AI training data. The data repository 130 may include a data set. The data set may associate information received from a particular medical / consumer electronic device 140, such as information from a sensor. The data set may include a diagnosis, urgency, and / or recommendation associated with the information received from a particular medical / consumer electronic device 140. The data repository 130 may include biometric data. The data repository 130 may include information for targeting advertisements. The data repository may include one or more AI engines and / or algorithms.

[0041] The network 150 can facilitate communication between the medical / consumer electronic devices 140, the analysis system 120, the data repository 130, the user devices 160, the telemedicine platform 170, and the third party stakeholder platform 180, as well as other devices, as would be understood by one skilled in the art.

[0042] Network 150 generally represents a network or collection of networks (such as the Internet or a corporate intranet, or a combination of both) on which reside the various components depicted in FIG. 1 (including other components that may be necessary to implement the systems described herein, as would be readily understood by one of ordinary skill in the art). In certain embodiments, network 150 is an intranet, an extranet, a Virtual Private Network (VPN), a Local Area Network (LAN), a Wireless LAN (WLAN), a Wide Area Network (WAN), a Metropolitan Area Network (MAN), a portion of the Internet, or another network 150 or a combination of two or more such networks 150. One or more links connect the systems and databases described herein to network 150. In certain embodiments, each of the one or more links includes one or more wired, wireless, or optical links. In certain embodiments, each of the one or more links may include an intranet, an extranet, a VPN, a LAN, a WLAN, a WAN, a MAN, a portion of the Internet, or another link or combination of two or more such links. This disclosure contemplates any suitable network 150 and any suitable links for connecting the various systems and databases described herein.

[0043] Network 150 connects the various systems and computing devices described or referenced herein. In certain embodiments, network 150 is an intranet, an extranet, a virtual private network (VPN), a local area network (LAN), a wireless LAN (WLAN), a wide area network (WAN), a metropolitan area network (MAN), a portion of the Internet, or another network or combination of two or more such networks 150. This disclosure contemplates any suitable network 150.

[0044] One or more links couple one or more systems, engines, or devices to network 150. In certain embodiments, each of the one or more links includes one or more wired, wireless, or optical links. In certain embodiments, each of the one or more links includes an intranet, an extranet, a VPN, a LAN, a WLAN, a WAN, a MAN, a portion of the Internet, or another link or a combination of two or more such links. This disclosure contemplates any suitable link coupling one or more systems, engines, or devices to network 150.

[0045] In certain embodiments, each system or engine may be a monolithic server or may be a distributed server across multiple computers or multiple data centers. The systems, engines, or modules may be of various types, such as, but not limited to, web servers, news servers, mail servers, message servers, ad servers, file servers, application servers, exchange servers, database servers, or proxy servers. In certain embodiments, each system, engine, or module may include hardware, software, or embedded logic components or a combination of two or more such components to perform the appropriate functionality implemented or supported by the respective server. For example, a web server may generally host websites that include web pages or particular elements of web pages. More specifically, a web server may host Hyper Text Markup Language (HTML) files or other file types, or may dynamically create or compose files upon request and communicate them to one or more computing devices or other devices in response to Hyper Text Transfer Protocol (HTTP) or other requests from one or more computing devices or other devices. A mail server may generally provide electronic mail services to one or more various computing devices or other devices. A database server may generally provide an interface for managing data stored in one or more data stores.

[0046] In certain embodiments, one or more data storages may be communicatively linked to one or more servers via one or more links. In certain embodiments, the data storages may be used to store various types of information. In certain embodiments, the information stored in the data storages may be organized according to a particular data structure. In certain embodiments, each data storage may be a relational database. Certain embodiments may provide an interface that allows a server or a client to manage, e.g., retrieve, modify, add, or delete, the information stored in the data storages.

[0047] The system may also include other subsystems and databases not shown in Figure 1, but which would be readily understood by one of ordinary skill in the art. For example, the system may include databases for storing data, storing features, storing outcomes (training sets), and storing models. As would be readily understood by one of ordinary skill in the art, other databases and systems may be added or subtracted without departing from the scope of the invention.

[0048] The first user may feel warm, have shortness of breath, a cough, and chest pain. The first user may use a thermometer on the first medical / consumer electronic device to measure a body temperature. The first user may use a pulse oximeter on the first medical / consumer electronic device to measure a heart rate and oxygen saturation. The first user may use a stethoscope on the first medical / consumer electronic device to listen to chest sounds. The body temperature may be 39 degrees Celsius (102 degrees Fahrenheit). The heart rate may be 92 beats per minute. The oxygen saturation may be 89%. The chest sounds may include congestion sounds. The first medical / consumer electronic device may transmit the body temperature, heart rate, oxygen saturation, and chest sounds over a network to a telehealth platform. The telehealth platform may use artificial intelligence (AI) to determine a likely infection associated with the respiratory system. The telehealth platform may determine that the likely causes include bacterial pneumonia, viral pneumonia, COVID-19, a COPD exacerbation, and the like. The telehealth platform may assign an urgency of 4 on a scale of 1 to 5, with 5 being the most urgent. The telehealth platform may recommend contacting a physician immediately, and proceeding to an urgent medical care facility if that is not possible. In one embodiment, the data repository may include an AI server / system / platform that analyzes the data and transmits the results to the telehealth platform and / or medical / consumer electronics / security / advertising equipment and / or any devices (wired or wireless) connected thereto.

[0049] The second user may experience chest pain along with numbness in the left arm, sweating, extreme weakness, and shortness of breath. The second user may use a thermometer on the second medical / consumer electronic device to measure body temperature. The second user may use a pulse oximeter on the second medical / consumer electronic device to measure pulse and oxygen saturation. The second user may use a high magnification camera on the second medical / consumer electronic device to determine blood pressure. The second user may obtain cardiac measurements using an electrocardiogram (EKG). The body temperature may be 27 degrees Celsius (80 degrees Fahrenheit). The pulse may be 106 beats per minute. The oxygen saturation may be 90%. The blood pressure may be 90 / 60. Cardiac measurements from an EKG may reveal a heart rate of 92 beats per minute, irregular heartbeats, and abnormal EKG patterns. The second medical / consumer electronic device may transmit heart measurements via temperature, pulse, oxygen saturation, blood pressure, and EKG over the network to the telehealth platform. The telehealth platform may use artificial intelligence (AI) to determine possible conditions include heart attack, blood clot in the lungs, pericarditis, etc. The telehealth platform may assign an urgency of 5 on a scale of 1 to 5, with 5 being the most urgent. The telehealth platform may recommend contacting a physician immediately, proceeding immediately to an emergency care facility, or calling 911. In one embodiment, the data repository may include an AI server / system / platform that analyzes the data and transmits the results to the telehealth platform and / or the medical / consumer electronic device / security / advertising device and / or any device (wired or wireless) connected thereto.

[0050] The third user may have nausea, vomiting, abdominal pain, very dark urine, and yellowing of the skin. The third user may use a thermometer on the third medical / consumer electronic device to measure a body temperature. The third user may use a pulse oximeter on the third medical / consumer electronic device to measure a pulse and oxygen saturation. The third user may use a high magnification camera on the third medical / consumer electronic device to take an image of one or more eyes. The body temperature may be 37 degrees Celsius (99 degrees Fahrenheit). The pulse may be 92 beats per minute. The oxygen saturation may be 96%. The blood pressure may be 90 / 60. The image of one or more eyes may reveal yellowing of the sclera. The telehealth platform may use artificial intelligence (AI) to determine that possible conditions include cholecystitis, gallstones, a pancreatic tumor causing a blockage of bile, and the like. The telehealth platform may assign an urgency of 3 on a scale of 1 to 5, with 5 being the most urgent. The telemedicine platform can recommend contacting a telemedicine doctor. The telemedicine platform can initiate a call with the telemedicine doctor. In one embodiment, the data repository can include an AI server / system / platform that analyzes the data and transmits the results to the telemedicine platform and / or medical / consumer electronics / security / advertising equipment and / or any devices (wired or wireless) connected thereto.

[0051] The fourth user may work in a secure area of ​​an office building or may seek access to a virtual secure area. To access the secure area, the fourth medical / consumer electronic device may be on an adjustable stand or may be part of a networked wearable headset such that the height of the high magnification camera of the fourth medical / consumer electronic device is adjusted to the eye level of the fourth user. The high magnification camera may photograph or video the unique individualized pattern of conjunctival vessels and / or the white space between conjunctival vessels within a predefined area of ​​at least 0.25 mm square and up to 10 cm (4 in) square of the fourth user to identify the fourth user as a user with access credentials to the physical or virtual secure area and allow the fourth user to access the secure area.

[0052] A fifth user may stand in front of the digital billboard. The fifth medical / consumer electronic device associated with the digital billboard may include various sensors for the user, including one or more microphones to detect noises made by the user, a thermometer to detect the user's body temperature, a barometer to detect changes in air pressure due to the user's presence, a Bluetooth receiver to detect and / or identify user devices that the user may be carrying, and / or multiple cameras to identify the user and / or user characteristics and / or user object characteristics, such as the license plate of a vehicle if the user is driving towards the digital billboard. In one embodiment, the conjunctival vasculature and / or the white space between the conjunctival vasculature may be measured. Additionally, data from the digital billboard sensors may be combined with data on in-house and / or affiliated and / or non-affiliated servers of other providers associated with the applicant and / or user (providers with whom the user has previously shared data) and which the user has previously provided permission for third parties to access for targeted advertising purposes. The various sensors may be used to identify the fifth user and / or the characteristics of the fifth user. Based on the identity of the fifth user and / or the identified characteristics of the fifth user, targeted advertisements for the fifth user may be selected and displayed on the digital billboard. A speaker associated with the digital billboard may address the fifth user by name to attract the fifth user's attention.

[0053] User device 160 refers to a computing device that may be used to display user interface elements associated with those that may be generated by relational database engine 102. User device 160 (also referred to herein as a user input device, user device, or client device) may generally include a computer or computing device that includes functionality for communicating (e.g., remotely) over network 150. User device 160 may be a server, a desktop computer, a laptop computer, a personal digital assistant (PDA), an in-vehicle or off-vehicle navigation system, a smart phone or other cellular or mobile phone, or a handheld gaming device, among other suitable computing devices. User device 160 may run one or more client applications, such as a web browser (e.g., Microsoft Windows® Internet Explorer, Mozilla Firefox, Apple Safari, Google Chrome, and Opera, etc.), or a dedicated application for submitting user data to create prediction queries over network 150.

[0054] In certain embodiments, each user device 160 may be an electronic device that includes hardware, software, or embedded logic components, or a combination of two or more such components, capable of performing the appropriate functions implemented or supported by the user device 160. For example, without limitation, the user device 160 may be a desktop computer system, a notebook computer system, a netbook computer system, a handheld electronic device, or a mobile phone. This disclosure contemplates any user device as a user device 160. The user device 160 may enable a network user of the user device 160 to access the network 150. The user device 160 may enable its user to communicate with other users of other client devices.

[0055] The user device 160 may have a web browser, such as MICROSOFT INTERNET EXPLORER, GOOGLE CHROME, or MOZILLA FIREFOX, and may have one or more add-ons, plug-ins, or other extensions, such as TOOLBAR or YAHOO TOOLBAR. The user device 160 may enable a user to enter a Uniform Resource Locator (URL) or other address that directs the web browser to a server, and the web browser may generate an HTTP request and communicate the HTTP request to the server. The server may accept the HTTP request and, in response to the HTTP request, communicate one or more HTML files to the user device 160. The user device 160 may render a web page based on the HTML files from the server for presentation to the user. This disclosure contemplates any suitable web page file. By way of illustration and not limitation, the web page may be rendered from an HTML file, an Extensible Hyper Text Markup Language (XHTML) file, or an Extensible Markup Language (XML) file, depending on particular needs. Such pages may also execute scripts written in a combination of markup languages ​​and scripts, such as, for example, but not limited to, JAVASCRIPT®, JAVA®, MICROSOFT SILVERLIGHT, Asynchronous JAVASCRIPT® and XML (AJAX), etc. As used herein, references to a web page include one or more corresponding web page files (that may be used by a web browser to render the web page), and vice versa, where appropriate.

[0056] The user device 160 may also include applications that are loaded onto the user device 160. The applications retrieve data from the network 150 and display it to the user within an application interface.

[0057] The present disclosure contemplates any suitable number of user devices 160, including computing systems taking any suitable physical form. By way of illustration and not limitation, a computing system may be an embedded computer system, a System-On-Chip (SOC), a Single-board Computer System (SBC) (e.g., a Computer-On-Module (COM) or System-On-Module (SOM)), a desktop computer system, a laptop or notebook computer system, a kiosk, a mainframe, a mesh of computer systems, a mobile phone, a personal digital assistant (PDA), a server, or a combination of two or more of these. Where appropriate, a computing system may include one or more computer systems and may be monolithic or distributed, may span multiple locations, may span multiple machines, or may reside in a cloud, which may include one or more cloud components in one or more networks. Where appropriate, one or more computing systems may perform one or more steps of one or more methods described or illustrated herein without substantial spatial or temporal limitations. By way of example and not limitation, one or more computing systems may perform one or more steps of one or more methods described or illustrated herein in real time or batch mode. One or more computing systems may perform one or more steps of one or more methods described or illustrated herein at different times or in different locations, where appropriate. In one embodiment, user device 150 may communicate and / or interface with medical / consumer electronic devices to obtain data from the devices via communication channels as disclosed herein and transmit the data to analysis system 120 via communication channels as disclosed herein.

[0058] Telemedicine platform 170 enables communication between patients and providers such as doctors, nurses, etc. Telemedicine platform 160 may enable communication between any users associated with any of the devices described herein.

[0059] All devices described herein may include a communication module for communicating with other devices described herein as well as devices not disclosed herein. The communication module may communicate via a wired connection (e.g., including a physical connection such as a cable with an appropriate connection interface such as Universal Serial Bus (USB), mini-USB, etc.) and / or a wireless network (e.g., via Near Field Communication (NFC), Bluetooth, Wireless Fidelity (WiFi), RFID, or any type of digital network not connected by a cable). For example, devices may communicate with each other directly in a pair-wise connection (1:1 relationship), or a hub-spoke or broadcast connection ("one-to-many", i.e., 1:m relationship). As another example, devices may communicate with each other via a mesh networking connection (e.g., "many-to-many", i.e., m:m relationship), such as Bluetooth mesh networking.Wireless communication includes Global System for Mobile Communications (GSM), Enhanced Data GSM Environment (EDGE), High-Speed ​​Downlink Packet Access (HSDPA), High-Speed ​​Uplink Packet Access (HSUPA), Evolution Data-Only (EV-DO), HSPA, HSPA+, Dual-Cell HSPA (DC-HSPDA), Long Term Evolution (LTE), Near Field Communications (NFC), Wideband Code Division Multiple Access (W-CDMA), Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), and LTE. Wireless network deployments may use any of a number of communication standards, protocols, and technologies, including, but not limited to, IEEE 802.11a, IEEE 802.11b, IEEE 802.11g, IEEE 802.11n, etc., or other suitable communication protocols. Some wireless network deployments may combine networks from multiple cellular networks (e.g., 3G, 4G, 5G) and / or may use a mix of cellular, WiFi, and satellite communications.

[0060] The third party stakeholder platform 180 may be comprised of third parties such as, but not limited to, insurance companies, hospitals, pharmacies, etc. In one embodiment, the third party stakeholder platform 180 may receive information from the medical / consumer electronics devices 140, the telemedicine platform 170, the analytics system 120, and / or the data repository 130 for preventive health purposes, overall medical cost savings, deductibles / premiums, etc.

[0061] Medical / Consumer Electronics 2 is a diagram of an example medical / consumer electronic device 200, according to an example embodiment of the present invention. The example medical / consumer electronic device 200 may receive information about a user from a sensor. The example medical / consumer electronic device 200 may receive information about a user from a sensor for medical diagnostic purposes. The example medical / consumer electronic device 200 may receive information about a user from a sensor for identification purposes. The example medical / consumer electronic device 200 may receive information about a user from a sensor for targeted advertising purposes. The example medical / consumer electronic device 200 may be the same as or similar to the medical / consumer electronic device 140 of FIG. 1. The illustrative medical / consumer electronic device 200 includes a high magnification camera 202, a motorized camera 204, a stethoscope 206, an infrared (IR) thermometer 208, an electrocardiogram (EKG) 210, a pulse oximeter 212, a body fat and / or muscle tone sensor 214, a transceiver 216, a display 218, input devices 220, a processor 222, a memory 224, a glucometer 226, and a hematology analyzer 228. Although not shown, the illustrative medical / consumer electronic device may also include one or more microphones for detecting noises made by a user, one or more speakers for providing audio to the user, a barometer for detecting changes in air pressure due to the user's presence, a tongue depressor for opening the user's mouth for an image and / or video of the throat, a urine analyzer for detecting and / or quantifying a number of analytes including bilirubin, protein, glucose, and red blood cells in the user's urine, and / or a Bluetooth receiver for detecting and / or identifying user devices that may be carried by the user. Although only one type of each sensor is shown in the example medical / consumer electronic device 200, the example medical / consumer electronic device may include more than one type of a particular sensor. For example, the example medical / consumer electronic device may include multiple high magnification cameras. As one skilled in the art would readily appreciate, other systems and databases may be used without departing from the scope of the present invention. The example medical device may receive information about the user from the sensors for informational and / or entertainment purposes.In one embodiment, the sensors may include one or more speakers and / or one or more microphones in addition to those described herein.

[0062] The high magnification camera 202 may include one or more Light-Emitting Diodes (LEDs). The high magnification camera 202 can capture high quality images and / or video of the patient's eye and / or skin. The high magnification camera 202 can zoom in on the blood vessels (veins / venules and / or arteries / arterioles) of the patient's eye and / or capture video of blood flowing through the blood vessels (veins / venules and / or arteries / arterioles) of the eye and / or the spaces between such vessels, which can be used to calculate the user's blood pressure. The high magnification camera 202 can capture the user's conjunctival vasculature, which can be used for user identification and / or advertising. The high magnification camera 202 can capture objects associated with the user, such as a vehicle license plate, which can be used for user identification.

[0063] The motorized camera 204 may include one or more LEDs. The motorized camera 204 can extend or retract as needed to obtain a clear position for capturing images and / or video. The motorized camera 204 can capture high quality images and / or video of the patient's ears, nose, and / or throat. The motorized camera can also be used as a tongue depressor.

[0064] The stethoscope 206 may be used to capture sound from the patient's heart. The stethoscope 206 may be used to capture sound from the patient's lungs. The stethoscope 206 may be used to capture sound from the patient's arteries. The stethoscope 206 may be used to capture sound from the patient's blood vessels (veins / venules and / or arteries / arterioles) and / or spaces between such blood vessels. The IR temperature 208 may capture the body temperature of the user. The EKG 210 may include a diode. The EKG 210 may capture electrical signals from the patient's heart. The pulse oximeter 212 may capture the patient's pulse. The pulse oximeter 212 may capture the patient's oxygen saturation. The body fat and / or muscle tone sensor 214 may capture the patient's body fat. The body fat and / or muscle tone sensor 214 may capture the patient's muscle tone.

[0065] The transceiver 216 can transmit and receive data to and from a remote server over a network. For example, the transceiver 216 can transmit signals indicative of information received from sensors of the medical / consumer electronic device 200 of FIG. 2 to the analysis system 120 of FIG. 1 over the network 150 of FIG. 1. For example, the transceiver 216 can receive signals indicative of a diagnosis, urgency, and / or recommendation from the analysis system 120 over the network 150. The transceiver 216 can preprocess the information received from sensors of the medical / consumer electronic device 200 into a format that can be transmitted over a network, such as the network 150, and ultimately consumed by a remote server, such as the analysis system 120. The transceiver 216 can receive information from a network, such as the network 150, and preprocess the information into a format that can be consumed by the processor 222.

[0066] The display 218 can display information captured from sensors of the medical / consumer electronic device 200. The display 218 can display information entered via the input 220, such as patient information (e.g., name, height, weight, etc.). The display 218 can display information received from a remote server, such as the analysis system 120 of FIG. 1. The input 220 can include a keyboard (e.g., a physical keyboard, a virtual keyboard, a projected keyboard, etc.), a mouse, a joystick, etc. The input 220 can be used to enter characters, make selections, etc. The processor 222 can execute instructions stored in the memory 224, such as an application for accessing the analysis system 120.

[0067] The glucometer 226 can receive a strip containing a blood sample. The glucometer 226 can measure the glucose concentration in the blood sample. The hematology analyzer 228 can receive the blood sample. The hematology analyzer 228 can analyze cells in the blood sample.

[0068] Analysis System FIG. 3 illustrates an example analysis system 300 according to an exemplary embodiment of the present invention. The example analysis system 300 can receive a signal from a medical / consumer electronic device, such as medical / consumer electronic device 140 of FIG. 1, via a network, such as network 150 of FIG. 1, the signal including information from a sensor of the medical / consumer electronic device. The example analysis system 300 can use the information from the sensor of the medical / consumer electronic device to determine a diagnosis, urgency, and / or recommendation. The example analysis system 300 can transmit a signal including the determined diagnosis, urgency, and / or recommendation to the medical / consumer electronic device, such as medical / consumer electronic device 140, via a network, such as network 150. The example analysis system 300 can be the same as or similar to analysis system 120 of FIG. 1. The example analysis system 300 includes one or more artificial intelligence (AI) modules. The AI ​​module can use training data to create a model for determining an output (e.g., a diagnosis, urgency, recommendation, etc.) for a particular input. The AI ​​module can update the model based on feedback from the responses, creating a feedback loop that continually improves future results based on past results. An example analysis system 300 includes a medical / consumer electronics interface 302, an AI eye analysis engine 304, an AI skin analysis engine 306, an AI ear analysis engine 308, an AI nose analysis engine 310, an AI throat analysis engine 312, an AI heart analysis engine 314, an AI lung analysis engine 316, an AI small artery analysis engine 318, an AI vein analysis engine 320, an AI body temperature analysis engine 322, an AI electrocardiogram (EKG) analysis engine 324, an AI pulse oximeter analysis engine 326, an AI body fat and / or muscle tone sensor analysis engine 328, an AI glucose analysis engine 330, an AI blood analysis engine 332, and an AI feedback engine 334. As one skilled in the art would readily appreciate, other systems and databases may be used without departing from the scope of the present invention.

[0069] The medical / consumer electronics interface 302 can pre-process information intended for a medical / consumer electronic device, such as medical / consumer electronic device 140 of Figure 1, into a format that can be transmitted over a network, such as network 150 of Figure 1, and ultimately consumable by the medical / consumer electronic device. The medical / consumer electronics interface 302 can receive information from a network, such as network 150, and pre-process the information into a format that can be consumable by the example analysis system 300.

[0070] The AI ​​eye analysis engine 304 may use computer vision to interpret eye images and / or videos captured by a camera, such as the high magnification camera 202 of FIG. 2. The AI ​​eye analysis engine 304 may identify one or more possible conditions, such as conjunctivitis, iritis, subconjunctival hemorrhage, scleral lesions, jaundice, liver drainage obstruction, bile drainage obstruction, hepatitis, gallbladder attack, bile blockage due to gallstones, pancreatic cancer, bile duct cancer, styes (chalazions), allergies, etc. The AI ​​eye analysis engine 304 may determine conjunctival vascular velocity by examining multiple time-stamped eye images or eye videos. The AI ​​eye analysis engine 304 may determine blood pressure based on the determined conjunctival vascular velocity. The AI ​​eye analysis engine 304 may determine identity by matching conjunctival vascular patterns (e.g., including conjunctival vascular blood flow velocity), which may include spaces between conjunctival vessels. The AI ​​eye analysis engine 304 may use AI to determine a state or condition. The AI ​​eye analysis engine 304 can use AI to determine a range of possible states or conditions. The AI ​​eye analysis engine 304 can associate a probability with each determined state or condition. An exemplary process employed by the AI ​​eye analysis engine 304 can be seen below in connection with FIG. 10 and the corresponding description.

[0071] The AI ​​skin analysis engine 306 may use computer vision to interpret images and / or video of the skin captured by a camera, such as the high magnification camera 202 of FIG. 2. The AI ​​skin analysis engine 306 may identify one or more possible rashes, hives, sumac, shingles, tumors, melanomas, bruises, petechiae, leg or other swelling, ulcers, bed sores, etc. The AI ​​skin analysis engine 306 may use AI to determine a condition or condition. The AI ​​skin analysis engine 306 can use AI to determine a range of possible conditions or conditions. The AI ​​skin analysis engine 306 can associate a probability with each determined condition or condition.

[0072] The AI ​​ear analysis engine 308 may use computer vision to interpret images and / or video of the ear captured by a camera, such as the motorized camera 204 of FIG. 2. The AI ​​ear analysis engine 308 may identify one or more possibilities of neoplasia, perforation, tympanosclerosis, middle ear effusion, retracted tympanic membrane, bloody tympanic membrane, etc. The AI ​​ear analysis engine 308 may use AI to determine a condition or condition. The AI ​​ear analysis engine 308 may use AI to determine a range of possible conditions or conditions. The AI ​​ear analysis engine 308 may associate a probability with each determined condition or condition.

[0073] The AI ​​nasal analysis engine 310 may use computer vision to interpret images and / or video of the nose captured by a camera, such as the motorized camera 204 of FIG. 2. The AI ​​nasal analysis engine 310 may identify one or more possibilities of nasal obstruction, sinusitis, rhinitis, polyps, adenoids, cancer, etc. The AI ​​nasal analysis engine 310 may use AI to determine a condition or condition. The AI ​​nasal analysis engine 310 may use AI to determine a range of possible conditions or conditions. The AI ​​nasal analysis engine 310 may associate a probability with each determined condition or condition.

[0074] The AI ​​throat analysis engine 312 may use computer vision to interpret images and / or video of the throat captured by a camera, such as the motorized camera 204 of FIG. 2. The AI ​​throat analysis engine 312 may identify one or more possibilities of viral infection, bacterial infection, Gastroesophageal Reflux Disease (GERD), tonsillitis, cancer, etc. The AI ​​throat analysis engine 312 may use AI to determine a condition or condition. The AI ​​throat analysis engine 312 may use AI to determine a range of possible conditions or conditions. The AI ​​throat analysis engine 312 may associate a probability with each determined condition or condition.

[0075] The AI ​​cardiac analysis engine 314 can interpret cardiac sounds obtained by a sensor, such as the stethoscope 206 of FIG. 2. The AI ​​cardiac analysis engine 314 may identify one or more possibilities for a heart murmur, an irregular heart rhythm, etc. The AI ​​cardiac analysis engine 314 may use AI to determine a state or condition. The AI ​​cardiac analysis engine 314 can use AI to determine a range of possible states or conditions. The AI ​​cardiac analysis engine 314 can associate a probability with each determined state or condition.

[0076] The AI ​​pulmonary analysis engine 316 can interpret pulmonary sounds obtained by a sensor, such as the stethoscope 206 of FIG. 2. The AI ​​pulmonary analysis engine 316 may identify one or more possibilities for abnormal breath sounds, wheezing, congestion sounds, etc. The AI ​​pulmonary analysis engine 316 may use AI to determine a condition or state. The AI ​​pulmonary analysis engine 316 can use AI to determine a range of possible conditions or states. The AI ​​pulmonary analysis engine 316 can associate a probability with each determined condition or state.

[0077] The AI ​​small artery analysis engine 318 can interpret audio of one or more arteries (and / or the spaces between) obtained by a sensor, such as stethoscope 206 of FIG. 2. The AI ​​small artery analysis engine 318 may identify one or more possibilities of arterial insufficiency, stenosis, occlusion, etc. The AI ​​small artery analysis engine 318 may use AI to determine a state or condition. The AI ​​small artery analysis engine 318 can use AI to determine a range of possible states or conditions. The AI ​​small artery analysis engine 318 can associate a probability with each determined state or condition.

[0078] The AI ​​vein analysis engine 320 may interpret audio of one or more blood vessels (veins / venules and / or spaces between such vessels) obtained by a sensor, such as the stethoscope 206 of FIG. 2. The AI ​​vein analysis engine 320 may identify one or more possibilities of venous insufficiency, stenosis, blockage, etc. The AI ​​vein analysis engine 320 may use AI to determine a condition or state. The AI ​​vein analysis engine 320 may use AI to determine a range of possible conditions or states. The AI ​​vein analysis engine 320 may associate a probability with each determined condition or state.

[0079] The AI ​​body temperature analysis engine 322 may interpret a temperature obtained by a sensor such as the IR thermometer 208 of FIG. 2. The AI ​​vein analysis engine 320 may identify either a normal temperature, a low-grade fever, a severe fever, hypothermia, etc. The AI ​​body temperature analysis engine 322 may use AI to determine a state or condition. The AI ​​body temperature analysis engine 322 can use AI to determine a range of possible states or conditions. The AI ​​body temperature analysis engine 322 can associate a probability with each determined state or condition.

[0080] The AI ​​EKG analysis engine 324 may interpret EKG measurements obtained by a sensor such as EKG 210 of FIG. 2. The AI ​​EKG analysis engine 324 may identify one or more possibilities of normal, arrhythmia, tachycardia, bradycardia, atrial fibrillation, dropped beats, premature atrial contractions (PACs), premature ventricular contractions (PVCs), ST elevation, ST depression, premature ventricular beats (VPBs), accidental VPBs, VPB trigeminas, VPB bigeminy, VPB doublets, VPB trigeminas, VPB quadruplets, VPB RonT, etc. The AI ​​EKG analysis engine 324 may use AI to determine a state or condition. The AI ​​EKG analysis engine 324 may use AI to determine a range of possible states or conditions. The AI ​​EKG analysis engine 324 may associate a probability with each determined state or condition.

[0081] The AI ​​pulse oximeter analysis engine 326 may interpret pulse measurements and / or oxygen saturation obtained by a sensor, such as pulse oximeter 212 of FIG. 2. The AI ​​pulse oximeter analysis engine 326 may identify one or more possibilities, such as a fast pulse rate (e.g., cardiac), a slow pulse rate, a normal pulse rate, normal oxygen saturation, low oxygen saturation, etc. The AI ​​pulse oximeter analysis engine 326 may use AI to determine a state or condition. The AI ​​pulse oximeter analysis engine 326 can use AI to determine a range of possible states or conditions. The AI ​​pulse oximeter analysis engine 326 can associate a probability with each determined state or condition.

[0082] The AI ​​body fat and / or muscle tone sensor analysis engine 328 may interpret body fat and / or muscle tone measurements obtained by a sensor, such as the body fat and / or muscle tone sensor 214 of FIG. 2. The AI ​​body fat and / or muscle tone sensor analysis engine 328 may identify one or more possibilities of obesity, muscle atrophy, and the like. The AI ​​body fat and / or muscle tone sensor analysis engine 328 may additionally use user input data, such as height and weight, to determine a fitness level. The AI ​​body fat and / or muscle tone sensor analysis engine 328 may use AI to determine a state or condition. The AI ​​body fat and / or muscle tone sensor analysis engine 328 may use AI to determine a range of possible states or conditions. The AI ​​body fat and / or muscle tone sensor analysis engine 328 may associate a probability with each determined state or condition.

[0083] The AI ​​glucose analysis engine 330 can interpret a concentration of glucose in a blood sample obtained by a sensor, such as glucometer 226 of FIG. 2. The AI ​​glucose analysis engine 330 may identify one or more possibilities, such as normal, pre-diabetes, diabetes, hypoglycemia, etc. The AI ​​glucose analysis engine 330 may use AI to determine a state or condition. The AI ​​glucose analysis engine 330 can use AI to determine a range of possible states or conditions. The AI ​​glucose analysis engine 330 can associate a probability with each determined state or condition.

[0084] The AI ​​blood analysis engine 332 can interpret cells in a blood sample obtained by a sensor, such as hematology analyzer 228 of FIG. 2. The AI ​​blood analysis engine 332 may identify one or more possibilities, such as sickle cell, anemia, human immunodeficiency virus (HIV), etc. The AI ​​blood analysis engine 332 may use AI to determine a state or condition. The AI ​​blood analysis engine 332 can use AI to determine a range of possible states or conditions. The AI ​​blood analysis engine 332 can associate a probability with each determined state or condition.

[0085] The AI ​​feedback engine 334 may take as input the health state and / or condition determined by the other modules. The AI ​​feedback engine 334 may use the overall picture of the health state and / or condition to determine a diagnosis. The AI ​​feedback engine 334 may use the overall picture of the health state and / or condition, or trends of such over time, to determine a range of diagnoses. Each entry within the range of diagnoses may include an associated degree of certainty. The AI ​​feedback engine 334 may assign an urgency to each diagnosis. The AI ​​feedback engine 334 may assign an urgency associated with the most urgent entry in the range of diagnoses to the range of diagnoses. The AI ​​feedback engine 334 may assign an urgency associated with the most likely entry in the range of diagnoses to the range of diagnoses. The AI ​​feedback engine 334 may assign a recommendation to each diagnosis. The AI ​​feedback engine 334 may assign a recommendation associated with the most urgent entry in the range of diagnoses to the range of diagnoses. The AI ​​feedback engine 334 may assign a recommendation associated with the most likely entry in the range of diagnoses to the range of diagnoses.

[0086] Processes for remote and / or automated medical diagnosis FIG. 4 illustrates a method 400 of remote and / or automated medical diagnosis in an example embodiment. In an embodiment, the method steps or techniques depicted and described herein may be performed in a processor of the medical / consumer electronic device 140 of FIG. 1, with the method steps encoded as processor-executable instructions in a non-transitory memory of the medical / consumer electronic device 140. In an embodiment, the method steps or techniques depicted and described herein may be performed in a processor, the analysis system 120 of FIG. 1, with the method steps encoded as processor-executable instructions in a non-transitory memory of the telemedicine platform 170. The technique of FIG. 4 may be implemented in an operating system kernel, a separate user process, a library package bound to a network application, a specially constructed machine, an Application-Specific Integrated Circuit (ASIC), or a Field Programmable Gate Array (FPGA).

[0087] At step 402, information may be received from at least two sensors, one of which may include and / or be associated with a high magnification camera, a motorized camera, a stethoscope, an infrared (IR) thermometer, an electrocardiogram (EKG), a pulse oximeter, a body fat and / or muscle tone sensor, a glucometer, or a hematology analyzer. The information from the at least two sensors may include one or more of an eye image, a skin image, an ear image, a nose image, a throat image, a chest sound, a body temperature, an EKG measurement, a pulse measurement, an oxygen saturation, a body fat measurement, a muscle tone measurement, a glucose measurement, and a hematology measurement.

[0088] In step 404, a signal indicative of the information received from the at least two sensors may be transmitted over a network to a remote server. The remote server may include a telemedicine platform. The remote server may include a cloud computing environment. The remote server may include an artificial intelligence (AI) system. The AI ​​system may be trained with data from the data repository.

[0089] In step 406, a signal may be received over the network from a remote server. The signal received from the remote server may include data indicative of a diagnosis, urgency, and / or recommendation. The diagnosis, urgency level, and / or recommendation may be based on the transmitted signal indicative of information received from at least two sensors. The diagnosis, urgency, and / or recommendation may be based on an AI system. The diagnosis may include a list of possible conditions. The list of possible conditions may be ordered by likelihood. The list of possible conditions may be ordered by severity. The urgency may include a number within a range. The range may include, for example, 1-5. A low number may be the least urgency. A high number may be the least urgency. The recommendation may include initiating a call with a telemedicine physician. If multiple possible conditions are determined, an urgency associated with the most urgent of the possible conditions may be selected. If multiple possible conditions are determined, an urgency associated with the most likely of the possible conditions may be selected.

[0090] In step 408, a diagnosis, urgency, and / or recommendation may be displayed. In the list of possible conditions, items associated with a threshold urgency and / or severity level may be displayed with a particular color and / or property. For example, a condition that may be associated with an elevated or higher severity may be displayed in bold red. The urgency may be displayed with a particular color and / or property if it is associated with a threshold urgency. For example, if urgency is measured from 1 to 5, with 5 being the most urgency, then an urgency of 4 or above may be displayed in bold red. Recommendations associated with a threshold urgency and / or severity level may be displayed with a particular color and / or property. For example, recommendations associated with an elevated or higher severity may be displayed in bold red.

[0091] Hardware Architecture In general, the techniques disclosed herein may be implemented in hardware or a combination of software and hardware. For example, such techniques may be implemented in an operating system kernel, a separate user process, a library package bound to a network application, a specially constructed machine, an application specific integrated circuit (ASIC), or in a network interface card.

[0092] At least some software / hardware hybrid implementations of the embodiments disclosed herein may be implemented on programmable network-resident machines (which should be understood to include intermittently connected network-enabled machines) that are selectively activated or reconfigured by a computer program stored in memory. Such network devices may have multiple network interfaces that may be configured or designed to utilize different types of network communication protocols. Some general architectures of these machines may be described herein to illustrate one or more exemplary means by which the functionality of a given unit may be implemented. According to specific embodiments, at least some of the features or functionality of the various embodiments disclosed herein may be implemented on one or more general-purpose computers associated with one or more networks, such as, for example, end-user computer systems, client computers, network servers or other server systems, mobile computing devices (e.g., tablet computing devices, mobile phones, smartphones, laptops, or other suitable computing devices), home electronic devices, music players, or other suitable electronic devices, routers, switches, or other suitable devices, or any combination thereof. In at least some embodiments, at least a portion of the features or functionality of various embodiments disclosed herein may be implemented in one or more virtualized computing environments (e.g., a network computing cloud, virtual machines hosted on one or more physical computing machines, or other suitable virtualization environments).

[0093] Referring now to FIG. 5, a block diagram illustrating an exemplary computing device 10 suitable for implementing at least some of the features or functionality disclosed herein is shown. The computing device 10 may be, for example, any one of the computing machines listed in the previous paragraph, or indeed any other electronic device capable of executing software or hardware-based instructions according to one or more programs stored in memory. The computing device 10 may be configured to communicate with a number of other computing devices, such as clients or servers, over a communications network, such as a wide area network, a metropolitan area network, a local area network, a wireless network, the Internet, or any other network, whether wireless or wired, using known protocols for such communications.

[0094] In one aspect, computing device 10 includes one or more Central Processing Units (CPUs) 12, one or more interfaces 15, and one or more buses 14 (such as a Peripheral Component Interconnect (PCI) bus). When operating under the control of appropriate software or firmware, CPU 12 may be responsible for implementing specific functions related to the functionality of a specifically configured computing device or machine. For example, in at least one aspect, computing device 10 may be configured or designed to function as a server system utilizing CPU 12, local memory 11 and / or remote memory 16, and interfaces 15. In at least one aspect, CPU 12 may be adapted to perform one or more of different types of functionality and / or operations under the control of software modules or components, which may include, for example, an operating system and any suitable application software, drivers, etc.

[0095] CPU 12 may include one or more processors 13, such as, for example, a processor from one of the Intel, ARM, Qualcomm, and AMD families of microprocessors. In some embodiments, processor 13 may include specially designed hardware, such as an application specific integrated circuit (ASIC), an electrically erasable programmable read-only memory (EEPROM), a field programmable gate array (FPGA), etc., for controlling the operation of computing device 10. In certain aspects, local memory 11 (e.g., non-volatile random access memory (RAM) and / or read-only memory (ROM), including one or more levels of cached memory) may also form part of CPU 12. However, there are many different ways in which memory may be coupled to system 10. Memory 11 may be used for a variety of purposes, such as, for example, caching and / or storing data, programming instructions, etc. It should be further appreciated that CPU 12 may be one of a variety of system-on-a-chip (SOC) type hardware that may include additional hardware such as memory or graphics processing chips, such as the QUALCOMM SNAPDRAGON™ or SAMSUNG EXYNOS™ CPUs that are becoming increasingly common in the art, for use in mobile or all-in-one devices.

[0096] As used herein, the term "processor" is not limited to integrated circuits referred to in the art as processors, mobile processors, or microprocessors, but refers broadly to microcontrollers, microcomputers, programmable logic controllers, application specific integrated circuits, and any other programmable circuit.

[0097] In one embodiment, interface 15 is provided as a network interface card (NIC). Typically, a NIC controls the transmission and reception of data packets over a computer network. Other types of interfaces 15 may support, for example, other peripherals used with computing device 10. Interfaces that may be provided include an Ethernet interface, a frame relay interface, a cable interface, a DSL interface, a token ring interface, a graphics interface, among others. In addition, various types of interfaces may be provided, such as, for example, Universal Serial Bus (USB), Serial, Ethernet, FIREWIRE®, THUNDERBOLT®, PCI, parallel, Radio Frequency (RF), BLUETOOTH®, near field communications (e.g., using a near magnetic field), 802.11 (WiFi), Frame Relay, TCP / IP, ISDN, a high speed Ethernet interface, a Gigabit Ethernet interface, a Serial ATA (SATA) or an external SATA (ESATA) interface, a High-Definition Multimedia Interface (HDMI®), a Digital Visual Interface (DVI), an analog or digital audio interface, an Asynchronous Transfer Mode (ATM) interface, a High-speed Serial Interface (HSSI) interface, a Point of Sale (POS) interface, a Fiber Data Distributed Interface (FDDI), and the like. In general, such an interface 15 may include any physical port suitable for communication with a suitable medium.In some cases, these may include a separate processor (such as a dedicated audio or video processor as is common in the art for hi-fi A / V hardware interfaces) and, in some cases, volatile and / or non-volatile memory (e.g., RAM).

[0098] The system shown in FIG. 5 illustrates one specific architecture for a computing device 10 for implementing one or more of the embodiments described herein, but is by no means the only device architecture in which at least some of the features and techniques described herein may be implemented. For example, an architecture having one or any number of processors 13 may be used, and such processors 13 may be present in a single device or distributed among any number of devices. In one embodiment, a single processor 13 handles communication as well as route computation, while in other embodiments, a separate dedicated communication processor may be provided. In various embodiments, different types of features or functionality may be implemented in a system according to an embodiment that includes a client device (such as a tablet device or smartphone running client software) and a server system (such as the server system described in more detail below).

[0099] Regardless of the configuration of the network devices, the system of an embodiment may employ one or more memories or memory modules (e.g., remote memory block 16 and local memory 11, etc.) configured to store data, program instructions for general network operation, or other information related to the functionality of the embodiments described herein (or any combination of the above). The program instructions may, for example, control or include the execution of an operating system and / or one or more applications. Memory 16 or memory 11, 16 may also be configured to store data structures, configuration data, encryption data, past system operation information, or any other specific or general non-program information described herein.

[0100] The medical / consumer electronic device 140 and / or the telemedicine platform 170 in FIG. 1 may be and / or may include a computing device 10 .

[0101] Because such information and program instructions may be employed to implement one or more of the systems or methods described herein, at least some network device embodiments may include a non-transitory machine-readable storage medium that may be configured or designed to store, for example, program instructions, state information, etc. for performing various operations described herein. Examples of such non-transitory machine-readable storage media include, but are not limited to, magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROM disks, magneto-optical media such as optical disks, and hardware devices specifically configured to store and execute program instructions, such as read-only memory devices (ROM), flash memory (common in mobile devices and all-in-one systems), solid state drives (SSDs) and "hybrid SSD" storage drives that may combine the physical components of a solid state drive and a hard disk drive into a single hardware device (as is becoming increasingly common in the art for personal computers), memristor memory, random access memory (RAM), and the like. It is to be appreciated that such storage means may be integral and non-removable (such as a RAM hardware module that may be soldered to a motherboard or otherwise integrated into an electronic device) or may be removable, such as a swappable flash memory module (such as a "thumb drive" or other removable media designed for rapid replacement of physical storage devices), a "hot-swappable" hard disk drive or solid state drive, a removable optical storage disk, or other such removable media, and that such integral and removable storage media may be used interchangeably.Examples of program instructions include both object code, such as may be produced by a compiler, machine code, such as may be produced by an assembler or linker, such as byte code, such as may be produced by a JAVA compiler and executed using the Java Virtual Machine or equivalent, or files containing higher-level code (e.g., scripts written in Python, Perl, Ruby, Groovy, or other scripting languages) that may be executed by the computer using an interpreter.

[0102] In some embodiments, the system may be implemented on a stand-alone computing system. Referring now to FIG. 6 above, a block diagram illustrating an exemplary illustrative architecture of one or more embodiments or components thereof on a stand-alone computing system is shown. The computing device 20 includes a processor 21 that may execute software to perform one or more functions or applications of the embodiments, such as, for example, a client application 24. The processor 21 may execute computing instructions under the control of an operating system 22, such as, for example, a version of the MICROSOFT WINDOW® operating system, the APPLE macOS® or iOS™ operating system, several various versions of the Linux® operating system, the ANDROID® operating system, etc. Often, one or more shared services 23 may be operational on the system 20 and useful for providing common services to the client applications 24. The services 23 may be, for example, WINDOWS® services, user space common services in a Linux® environment, or other types of common service architectures used with the operating system 22. The input device 28 may be of any type suitable for receiving user input, including, for example, a keyboard, a touch screen, a microphone (e.g., for voice input), a mouse, a touch pad, a trackball, or any combination thereof. The output device 27 may be of any type suitable for providing output to one or more users, whether remote or local to the system 20, and may include, for example, one or more screens for visual output, speakers, a printer, or any combination thereof. The memory 25 may be a random access memory having any structure and architecture known in the art, for example, for use by the processor 21 to execute software.Storage device 26 may be any magnetic, optical, mechanical, memristor, or electrical storage device for storing data in digital form (such as that described above with reference to FIG. 6). Examples of storage device 26 include flash memory, magnetic hard drives, CD-ROMs, etc.

[0103] The medical / consumer electronic device 140 and / or the telemedicine platform 170 in FIG.

[0104] In some embodiments, the system may be implemented on a distributed computing network, such as a network having any number of clients and / or servers. Referring now to FIG. 7, a block diagram is shown illustrating an exemplary architecture 30 for implementing at least a portion of a system according to one embodiment on a distributed computing network. According to the embodiment, any number of clients 33 may be provided. Each client 33 may execute software for implementing a client-side portion of the system, and the clients may include a system 20, such as the system shown in FIG. 6. In addition, any number of servers 32 may be provided to process requests received from one or more clients 33. The clients 33 and servers 32 may communicate with each other via one or more electronic networks 31, which in various embodiments may be any of the Internet, a wide area network, a mobile phone network (such as a CDMA or GSM cellular network), a wireless network (such as WiFi, WiMAX, LTE, etc.), or a local area network (or indeed any network topology known in the art, and the embodiment does not prefer any one network topology over another). Network 31 may be implemented using any known network protocol, including, for example, wired and / or wireless protocols.

[0105] Additionally, in some embodiments, the server 32 may call an external service 37 as necessary to obtain additional information or to reference additional data regarding a particular call. Communication with the external service 37 may occur, for example, over one or more networks 31. In various embodiments, the external service 37 may include web-enabled services or functionality associated with or installed on the hardware device itself. For example, in one aspect where the client application 24 is implemented on a smartphone or other electronic device, the client application 24 may retrieve information stored in an external service 37 deployed on one or more of the server systems 32 in the cloud or on the premises of a particular enterprise or user.

[0106] In some embodiments, the client 33 or server 32 (or both) may utilize one or more dedicated services or appliances that may be deployed locally or remotely across one or more networks 31. For example, one or more databases 34 may be used or referenced by one or more embodiments. Those skilled in the art will appreciate that the databases 34 may be arranged in a variety of architectures and using a variety of data access and manipulation means. For example, in various embodiments, the one or more databases 34 may include a relational database system that uses Structured Query Language (SQL), while other databases may include alternative data storage technologies such as those referred to in the art as "NoSQL" (e.g., HADOOP CASSANDRA™, GOOGLE BIGTABLE™, etc.). In some embodiments, variant database architectures such as columnar databases, in-memory databases, clustered databases, distributed databases, or flat file data repositories may be used in accordance with the aspects. Those skilled in the art will appreciate that unless a particular database technology or a particular arrangement of components is specified for a particular embodiment described herein, any combination of known or future database technologies may be used as appropriate. Moreover, it will be appreciated that the term "database" as used herein may refer to a physical database machine, a cluster of machines functioning as a single database system, or a logical database within an overall database management system. Unless a specific meaning is specified for a given use of the term "database," the term should be construed to mean any of these meanings, all of which would be understood by one of skill in the art as the plain meaning of the term "database."

[0107] Similarly, some embodiments may utilize one or more security systems 36 and configuration systems 35. Security and configuration management is a common information technology (IT) and web function, and generally any IT or web system will have some degree of security and configuration management associated with it. Those skilled in the art will appreciate that, unless a particular security 36 or configuration system 35 or technique is specifically required by the description of a particular embodiment, any configuration or security subsystem known in the art, now or in the future, may be used in conjunction with the embodiments without limitation.

[0108] The medical / consumer electronics devices 140 and / or telemedicine platform 170 and / or data repository 130 and / or third party stakeholder platform 180 and / or analysis system 120 of FIG. 1 may be and / or include one or more of the servers 32.

[0109] FIG. 8 shows an exemplary overview of a computer system 40, as may be used in any of a variety of locations throughout the system. This diagram is illustrative of any computer capable of executing code to process data. Various modifications and variations may be made to the computer system 40 without departing from the broad scope of the systems and methods disclosed herein. A central processing unit (CPU) 41 is connected to a bus 42, which also connects to a memory 43, a non-volatile memory 44, a display 47, an input / output (I / O) unit 48, and a network interface card (NIC) 53. The I / O unit 48 may typically be connected to a keyboard 49, a pointing device 50, a hard disk 52, and a real-time clock 51. The NIC 53 connects to a network 54, which may be the Internet or a local network, which may or may not have a connection to the Internet. Also shown as part of the system 40 is a power supply unit 45, which in this example is connected to a mains alternating current (AC) power source 46. A battery is not shown but may be present, and there are many other devices and modifications that are well known but do not apply to the specifically novel features of the present systems and methods disclosed herein. It should be appreciated that some or all of the components shown may be combined in various integrated applications, such as, for example, Qualcomm or Samsung system-on-chip (SOC) devices, or whenever it is appropriate to combine multiple capabilities or functions into a single hardware device (e.g., in mobile devices such as smartphones, video game consoles, in-vehicle computer systems such as automobile navigation or multimedia systems, or other integrated hardware devices).

[0110] The medical / consumer electronics device 140 and / or telemedicine platform 170 and / or data repository 130 and / or third party stakeholder platform 180 and / or analysis system 120 of FIG. 1 may be and / or may include a computer system 40.

[0111] In various embodiments, functionality for implementing the systems or methods of various embodiments may be distributed among any number of client and / or server components. For example, various software modules may be implemented to perform various functions in conjunction with any particular embodiment of the system, and such modules may be variously implemented to execute on the server and / or client components.

[0112] Those skilled in the art will recognize the numerous possible modifications of the various embodiments described above. Therefore, the present invention is defined by the claims and their equivalents.

[0113] Metaverse / Virtual World Telemedicine Integration 9 illustrates a method 900 of remote and / or automated medical diagnosis and care delivery in a metaverse / virtual world in an example embodiment. In an embodiment, the method steps or techniques depicted and described herein may be performed in a virtual world / metaverse provider environment, with possible assistance from / integration with one or more of the other systems, methods, and devices described in the preceding sections and paragraphs of this disclosure.

[0114] In step 902, a virtual healthcare telemedicine store (virtual clinic) is generated based on the virtual world provider's plug-and-play elements or API-based imported elements.

[0115] At step 904, using the virtual world provider's plug-and-play or API-based imported elements, one or more objects are selected for inclusion in the virtual store based on information associated with the user.

[0116] In step 906, the store layout is customized based on information related to the user imported from external healthcare data repositories (130 in FIG. 1) and the telemedicine platform (170 in FIG. 1) and combined with data about the user from the virtual world provider's plug-and-play or API-based elements.

[0117] In step 908, other users (healthcare professionals including doctors, nurses, therapists, pharmacists, etc.) are selected for display in the store based on information associated with the user imported from external healthcare data repositories (130 in FIG. 1) and telemedicine portals (170 in FIG. 1) and combined with data about the user from plug-and-play or API-based elements of the virtual world provider.

[0118] In step 910, the virtual healthcare telemedicine store is provided to the user within the virtual world provider's environment, either via an existing external telemedicine portal (170 in FIG. 1 ) (via an API) or via a direct user connection to the virtual world provider's environment.

[0119] The user can then interact with the virtual healthcare telemedicine store (within the virtual world provider's environment) in the same manner as the user interacts with an existing telemedicine application or website described in this disclosure, interacting with the devices described in this disclosure and sending data to (and receiving data from) the virtual healthcare telemedicine store, in conjunction with the existing telemedicine application or website, data repository, and all other network nodes and systems described in this disclosure.

[0120] 10 illustrates a process for calculating an individual's blood pressure based on an image of the individual's eye in an example embodiment. The example process includes receiving an image of the individual's external eye 1001, calculating vascular measurements of the external eye blood vessels 1003, calculating a blood pressure measurement for the individual 1005, and providing a feedback result 1007.

[0121] In step 1001, the process may include receiving an image of the individual's external eye. The image may be received from an end-user handheld electronic device (e.g., medical / consumer electronic device 140 or 200). The end-user handheld electronic device may include and / or interface with a high magnification camera operable to capture digital images and / or video of the individual's external eye. The high magnification camera may be integral to the end-user handheld electronic device. The high magnification camera may be operable to capture digital images and / or video of blood flowing through the vascular system of the external eye. The end-user handheld electronic device may be operable to capture a series of digital images and provide them to a remote server for further processing. The series of images may include a series of images with a time stamp. The images may be received in real time as the images are acquired by the end-user handheld electronic device.

[0122] In step 1003, the process may include calculating vascular measurements for at least one blood vessel present in the received image. Calculating the vascular measurements may include applying a computer vision algorithm via the processor to analyze the received image. The computer vision algorithm may extract pixel value information associated with at least one pixel for each image of the received image (e.g., each image of the sequence of images). The computer vision algorithm may analyze the extracted pixel information to determine at least one image characteristic including, but not limited to, blood vessel diameter, blood vessel wall thickness, blood vessel length, distance between blood vessels or blood vessel segments, area between blood vessels or blood vessel segments, and blood flow velocity measurements associated with the at least one blood vessel. The calculated vascular measurement may include one of the image characteristics. The calculated vascular measurement may include measurements calculated from a combination of the image characteristics. The calculated vascular measurement may include measurements calculated using at least one of the image characteristics. For example, the image characteristic may be used to calculate at least one characteristic including, but not limited to, blood pressure, volume, resistance, volumetric flow rate, flow velocity, and cross-sectional area associated with the blood vessel that is associated with blood flow through the vessel. The computer vision algorithm may determine at least one measure for the series of images, including, but not limited to, at least one of a minimum vessel diameter, a maximum vessel diameter, an average vessel diameter, a minimum vessel wall thickness, a maximum vessel wall thickness, an average vessel wall thickness, an area between vessels (e.g., an area formed by surrounding vessels), a distance between vessels (e.g., a distance between an outer wall of a first vessel and an outer wall of a second vessel), a minimum blood flow velocity, a maximum blood flow velocity, and an average blood flow velocity. The computer vision algorithm may calculate a vascular measurement for at least one vessel based on the at least one measure for the series of images. In one aspect, a blood flow characteristic may be calculated based on at least one image characteristic. In one aspect, a change in at least one of a vessel diameter, a vessel wall thickness, a distance between vessels or vessel segments, and an area between vessels or vessel segments may indicate a change in a blood flow characteristic.For example, an increase in vascular diameter may be associated with a change in blood flow characteristics (e.g., a decrease in blood pressure, an increase in volume, and / or a decrease in velocity), while a decrease in vascular diameter may be associated with an opposite change in blood flow characteristics (e.g., an increase in blood pressure, a decrease in volume, and / or an increase in velocity). Similarly, a decrease in the space (e.g., distance, area, etc.) between blood vessels may indicate an increase in the vascular diameter of the blood vessel associated with that space due to increased flow (e.g., a decrease in blood pressure, an increase in volume, and / or a decrease in velocity). In other words, a decrease in the space between blood vessels indicates an increase in flow characteristics, and an increase in the space between blood vessels indicates a decrease in flow characteristics.

[0123] The computer vision algorithm may include pre-processing steps or algorithms operable to prepare the image for further analysis. Pre-processing may include, but is not limited to, noise removal, filtering, smoothing, contrast enhancement, artifact removal, scaling, dilation, erosion, and the like. The computer vision algorithm may include at least one of object detection, edge detection, video tracking, object recognition, 3D pose estimation, and motion estimation (e.g., tracking and / or optical flow). The computer vision algorithm may include windowing, where selective processing is performed on pixels that meet certain value criteria (e.g., minimum value threshold, maximum value threshold). The computer vision algorithm may include area of ​​interest analysis, where selective processing is performed on pixels within a specified location in the image (e.g., where blood vessels are identified). The computer vision algorithm may determine velocity by tracking at least one pixel within a specified region of interest. The region of interest may be of a specified size. The region of interest may be of a fixed size. The region of interest may be smaller than the total size of the image being analyzed. The region of interest may be a fixed location across a series of images.

[0124] The computer vision algorithm may be trained using machine learning techniques such as neural networks and / or deep learning. The computer vision algorithm may be trained using at least one of supervised learning, unsupervised learning, semi-supervised learning, and reinforcement learning. The computer vision algorithm may be trained using at least one of linear regression, logistic regression, decision trees, random forest algorithms, support vector machines, Naive Bayes algorithms, random walk algorithms, k-nearest neighbor algorithms, k-means clustering, and Markov models. These training methods are merely exemplary, and other training methods / techniques may be used without departing from the scope of the present invention, as would be apparent to one of ordinary skill in the art. The computer vision may be trained using labeled images. The labeled images may include labels (e.g., labeled pixels) for at least one of blood vessels, blood vessel walls, blood flow area, inner vessel diameter, outer vessel diameter, distance between blood vessels or blood vessel segments, area between blood vessels or blood vessel segments, and blood flow velocity.

[0125] In step 1005, the process may include calculating a blood pressure measurement of the individual based on the acquired image of the individual's external eye. Calculating the blood pressure measurement may include calculating, by the processor, the blood pressure measurement using an artificial intelligence (AI) analysis algorithm. The AI ​​analysis algorithm may relate at least one image characteristic and / or the calculated vascular measurement (e.g., determined by a computer vision algorithm) to a blood pressure value. The AI ​​analysis algorithm may be trained using machine learning techniques such as neural networks and / or deep learning. The AI ​​analysis algorithm may be trained using at least one of supervised learning, unsupervised learning, semi-supervised learning, and reinforcement learning. The AI ​​analysis algorithm may be trained using at least one of linear regression, logistic regression, decision trees, random forest algorithms, support vector machines, Naive Bayes algorithms, random walk algorithms, k-nearest neighbor algorithms, k-means clustering, and Markov models. These training approaches are merely exemplary and it will be apparent to one of ordinary skill in the art that other training approaches / techniques may be used without departing from the scope of the present invention.

[0126] The AI ​​analytic algorithm may be trained using previously acquired images of the individual's extraocular vasculature and / or previously acquired blood pressure measurements associated with the individual. The previously acquired images may have been processed using computer vision algorithms as described above. The AI ​​analytic algorithm may be trained using previously acquired and processed images of the individual's extraocular eye taken in combination (e.g., simultaneously) with previously acquired blood pressure measurements associated with the individual. The previously acquired images and previously acquired blood pressure measurements may be taken in combination (e.g., simultaneously) with one another at multiple different time points (e.g., multiple time points over a day and / or over several days or weeks). For example, for each user or individual who is examined / imaged using an end-user handheld electronic device, corresponding extraocular vasculature imaging and blood pressure measurements can be repeated over time for use in training the AI ​​analytic algorithm. In one aspect, the AI ​​analytic algorithm may rely on a generalized model trained using broader population data (e.g., a combination of images and blood pressure measurements for multiple different individuals) in calculating blood pressure measurements. In one aspect, the corresponding imaging and blood pressure measurements taken for each user or individual are used to train an AI analytic algorithm unique to that user or individual. Because each individual may have different extraocular vasculature characteristics that may correspond to different blood pressure measurements compared to other individuals, an AI analysis algorithm unique to each individual may allow for more accurate blood pressure measurements than a generalized model that attempts to calculate blood pressure using a model trained with broader population data.

[0127] The AI ​​analytics algorithm may calculate an amount of change in at least one image characteristic (e.g., as determined by a computer vision algorithm). The AI ​​analytics algorithm may calculate the amount of change by comparing at least one current image characteristic associated with the individual to at least one previously calculated image characteristic (e.g., from a baseline image and / or an image used in training, from a previously acquired image on a different day and / or time). The AI ​​analytics may calculate a rate of change in the at least one image characteristic. The AI ​​analytics algorithm may calculate a blood pressure measurement based on the amount of change. The AI ​​analytics algorithm may calculate a blood pressure based on a magnitude of the change, such that the calculated blood pressure increases or decreases by an amount relative to the magnitude of the change. The AI ​​analytics algorithm may calculate a blood pressure based on a direction of the change, such that the calculated blood pressure increases or decreases in relation to whether the calculated change is an increase or decrease. The AI ​​analytics algorithm may calculate an amount of change in the space between blood vessels between the current image and the previously acquired image. The change in the space between blood vessels may serve as an indicator of a change in blood vessel diameter, which may be an indicator of a change in blood flow (e.g., blood pressure, volume, velocity, etc.). For example, a decrease in the space between blood vessels between a previous image and a current image may indicate a lower blood pressure at the current time compared to the previous time due to the decrease in the space between blood vessels calculated due to the increase in blood vessel diameter. Although described herein as separate algorithms, the computer vision algorithms and the AI ​​analytics algorithms may be combined into a single algorithm that performs the composite functions of each individual algorithm without departing from the scope of the present invention. Similarly, each algorithm may be further decomposed into smaller algorithms or sub-algorithms that collectively perform the same basic function without departing from the scope of the present invention.

[0128] At step 1007, the process may include providing a feedback result to the handheld end-user electronic device. The feedback result may be provided via network communication. The feedback result may include at least the calculated blood pressure. The feedback result may include at least one of the above calculated measures including, but not limited to, the calculated blood pressure, the vessel diameter, the vessel wall thickness, the vessel length, the distance between the vessels or vessel segments, the area between the vessels or vessel segments, a blood flow velocity measurement associated with at least one vessel, and a calculated change measure. Providing the feedback result may include providing the feedback result in near real-time such that upon receiving the image from the handheld end-user device, the image is processed as described above, and upon completion of the processing, the feedback result is promptly provided to the end-user electronic device for display. The feedback result may be at least one of prepared in a format and converted to a format transmittable over a network. The feedback result may be prepared in a format suitable for display on the handheld end-user electronic device and / or communicated in such format.

[0129] Further considerations As used herein, the phrase "one embodiment" or "one embodiment" means that a particular element, feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment. The appearances of the phrase "in one embodiment" in various places in the specification are not necessarily all referring to the same embodiment.

[0130] Some embodiments may be described using the terms "coupled" and "connected," along with their derivatives. For example, some embodiments may be described using the term "coupled" to indicate that two or more elements are in direct physical or electrical contact with each other. However, the term "coupled" may also mean that two or more elements are not in direct contact with each other, but still co-operate or interact with each other. The embodiments are not limited in this context.

[0131] As used herein, the terms "comprises," "comprising," "includes," "including," "has," "having," or other variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, article, or apparatus that includes a list of elements is not necessarily limited to only those elements, but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Furthermore, unless expressly stated otherwise, "or" refers to an inclusive "or," not an exclusive "or." For example, a condition A or B is satisfied by any one of the following: A is true (or present) and B is false (or absent), A is false (or absent) and B is true (or present), or both A and B are true (or present).

[0132] In addition, the use of "a" or "an" is employed herein to describe elements and components of the embodiments. This is merely for convenience and to give a general sense of the invention. This description should be read to include one or at least one, unless it is clear that something else is meant, and the singular also includes the plural.

[0133] Upon reading this disclosure, those skilled in the art will appreciate further additional and alternative structural and functional designs for systems and processes for remote and / or automated medical diagnostics through the principles disclosed herein. Thus, while specific embodiments and applications have been illustrated and described, it is to be understood that the disclosed embodiments are not limited to the precise structures and components disclosed herein. Various obvious modifications, changes and variations may be made in the arrangement, operation and details of the methods and apparatus disclosed herein without departing from the spirit and scope as defined in the appended claims.

Claims

1. 1. A computer-implemented method for providing feedback regarding a blood pressure of an individual using analysis of sensor data and artificial intelligence (AI) analysis of an image of the individual's external eye, comprising: receiving, at a remote server via network communication with a handheld end-user electronic device, a series of digital images of at least a portion of the individual's external eye, the handheld end-user electronic device including a high magnification camera operable to capture at least one of images and videos of blood flowing through the vascular system of the external eye; a processor calculating, using at least one computer vision algorithm, a vascular measurement of at least one blood vessel within the portion of the individual's external eye, wherein the computer vision algorithm extracts pixel value information from at least one pixel for each image of the series of digital images and processes the pixel value information to determine at least one image characteristic associated with the series of images, the at least one image characteristic comprising at least one of a blood vessel diameter, a blood vessel wall thickness, a blood vessel length, a distance between blood vessels or vessel segments, an area between blood vessels or vessel segments, and a blood flow velocity measurement associated with at least one blood vessel or vessel segment, the blood vessel measurement being calculated based on the at least one image characteristic; a processor calculating a blood pressure measurement for the individual using an AI analysis algorithm that relates the vascular measurements to blood pressure values, the AI ​​analysis algorithm being trained using previously acquired and processed images of the individual's extraocular vasculature and previously obtained blood pressure measurements associated with the individual; providing, via network communication, feedback results to the handheld end-user electronic device and / or a provider via a telehealth connection for display on the end-user electronic device and / or a provider portal via a telehealth connection, the feedback results including at least the calculated blood pressure reading; 4. A computer-implemented method comprising:

2. 2. The computer-implemented method of claim 1, wherein the computer vision algorithm is trained using labeled images including labeled pixels associated with at least one of blood vessels, blood vessel walls, blood flow area, inner vessel diameter, outer vessel diameter, distance between blood vessels or vessel segments, area between blood vessels or vessel segments, and blood flow velocity through blood vessels or vessel segments.

3. 2. The computer-implemented method of claim 1, wherein the computer vision algorithm is operable to determine at least one metric for the series of images, the at least one metric including minimum medial or outer vessel diameter, maximum medial or outer vessel diameter, average medial or outer vessel diameter, minimum vessel wall thickness, maximum vessel wall thickness, average vessel wall thickness, minimum blood flow velocity, maximum blood flow velocity, average blood flow velocity, and distance or area between adjacent or nearby vessels or vessel segments.

4. 2. The computer-implemented method of claim 1, wherein the computer vision algorithm is operable to perform at least one of object detection, edge detection, video tracking, object recognition, 3D pose estimation, and motion estimation.

5. 2. The computer-implemented method of claim 1, wherein the AI ​​analysis algorithm calculates an amount of change in at least one image characteristic by comparing at least one current image characteristic with at least one previously calculated image characteristic associated with the individual, and calculates the blood pressure measurement based on the amount of change.

6. 2. The computer-implemented method of claim 1, wherein the AI ​​analysis algorithm is trained using previously acquired and processed images of the external eye of the individual taken in combination with the previously acquired blood pressure measurements associated with the individual.

7. The computer-implemented method of claim 6 , wherein the previously-acquired image and the previously-acquired blood pressure measurement are acquired simultaneously.

8. The computer-implemented method of claim 6 , wherein the previously acquired images and previously acquired blood pressure measurements are obtained in combination with one another at a number of different time points.

9. The computer-implemented method of claim 1 , wherein blood flow velocity is determined by tracking at least one pixel within a specified region of interest, the region of interest being a region of a specified size.

10. 10. The computer-implemented method of claim 9, wherein the regions of interest are of a fixed size and are located at specified locations within the images, and the size of the regions of interest is less than a combined size of each image.

11. The computer-implemented method of claim 1 , wherein the sequence of images comprises a sequence of time-stamped images.

12. 2. The computer-implemented method of claim 1, wherein the images are received in real-time as they are acquired by the handheld end-user electronic device, and the feedback results are provided to the handheld end-user electronic device and / or a provider portal via a telehealth connection in real-time as the blood pressure measurements are calculated.

13. 2. The computer-implemented method of claim 1, wherein the feedback results are converted and / or communicated in a format suitable for display on the handheld end-user electronic device and / or a provider portal via a telehealth connection.

14. 10. The computer-implemented method of claim 1, wherein the high magnification camera is operable to be connected to or integral to the handheld end-user electronic device.

15. 2. The computer-implemented method of claim 1, wherein at least one of the AI ​​analytics algorithms and computer vision algorithms is trained using at least one of a linear regression, a logistic regression, a decision tree, a random forest algorithm, a support vector machine, a Naive Bayes algorithm, a random walk algorithm, a k-nearest neighbor algorithm, k-means clustering, and a Markov model.

16. 2. The computer-implemented method of claim 1 , wherein at least one of the AI ​​analytics algorithm and the computer vision algorithm is trained using at least one of supervised learning, unsupervised learning, semi-supervised learning, and reinforcement learning.

17. The computer-implemented method of claim 1 , wherein the vascular measurement comprises one or is calculated using a combination of the image characteristics.

18. 1. A computing system for artificial intelligence (AI) analysis of an image of an individual's external eye to calculate a blood pressure of the individual, comprising: at least one computing processor; and a memory containing instructions that, when executed by the at least one computing processor, cause the computing system to: receiving, at a remote server via network communication with a handheld end-user electronic device, a series of digital images of at least a portion of the individual's external eye, the handheld end-user electronic device including a high magnification camera operable to capture at least one of images and videos of blood flowing through the vascular system of the external eye; calculating, by a processor, using at least one computer vision algorithm, a vascular measurement of at least one blood vessel within the portion of the individual's external eye, wherein the computer vision algorithm extracts pixel value information from at least one pixel for each image of the series of digital images and processes the pixel value information to determine at least one image characteristic associated with the series of images, the at least one image characteristic including at least one of a blood vessel diameter, a blood vessel wall thickness, a blood vessel length, a distance between blood vessels or vessel segments, an area between blood vessels or vessel segments, and a blood flow velocity measurement associated with at least one blood vessel or vessel segment, the blood vessel measurement being calculated based on the at least one image characteristic; calculating, by a processor, a blood pressure measurement for the individual using an AI analysis algorithm that relates the vascular measurements to blood pressure values, the AI ​​analysis algorithm being trained using previously acquired and processed images of the individual's extraocular vasculature and previously obtained blood pressure measurements associated with the individual; providing, via network communication, feedback results to the handheld end-user electronic device for display on the end-user electronic device and / or on a provider portal via a telehealth connection, the feedback results including at least the calculated blood pressure reading; A computing system that enables

19. 1. A non-transitory computer readable medium comprising instructions that, when executed by a processor, enable the processor to perform a method for calculating blood pressure based on an AI analysis of an image of an individual's external eye, the method comprising: receiving, at a remote server via network communication with a handheld end-user electronic device, a series of digital images of at least a portion of the individual's external eye, the handheld end-user electronic device including a high magnification camera operable to capture at least one of images and videos of blood flowing through the vascular system of the external eye; calculating, by a processor, using at least one computer vision algorithm, a vascular measurement of at least one blood vessel within the portion of the individual's external eye, wherein the computer vision algorithm extracts pixel value information from at least one pixel for each image of the series of digital images and processes the pixel value information to determine at least one image characteristic associated with the series of images, the at least one image characteristic including at least one of a blood vessel diameter, a blood vessel wall thickness, a blood vessel length, a distance between blood vessels or vessel segments, an area between blood vessels or vessel segments, and a blood flow velocity measurement associated with at least one blood vessel or vessel segment, the blood vessel measurement being calculated based on the at least one image characteristic; calculating, by a processor, a blood pressure measurement for the individual using an AI analysis algorithm that relates the vascular measurements to blood pressure values, the AI ​​analysis algorithm being trained using previously acquired and processed images of the individual's extraocular vasculature and previously obtained blood pressure measurements associated with the individual; providing, via network communication, feedback results displayed on the end-user electronic device to the handheld end-user electronic device and / or to a provider portal via a telehealth connection, the feedback results including at least the calculated blood pressure reading; A non-transitory computer readable medium comprising:

Images