A system and method for determining sleep analysis based on body images.

Abstract

A system and method for determining a sleep disorder in a patient is disclosed. A storage device stores a digital image including the patient's face and neck. A database stores previously identified phenotypes and dimensions of facial and neck features. A sleep disorder analysis engine is coupled to the storage device and the database. The sleep disorder analysis engine is operable to identify facial and neck features on the image by determining landmarks on the image. The sleep disorder analysis engine classifies at least one phenotype on the image based on a comparison with the database. The sleep disorder analysis engine correlates the at least one phenotype and at least one feature with a sleep disorder. The sleep disorder analysis engine determines a risk score for the sleep disorder based on the correlation between the phenotype and the feature.

Description

Technical Field

[0001] "Cross - reference to Related Applications" This application claims the benefit and priority of U.S. Provisional Patent Application No. 63 / 124,577, filed on December 11, 2020, which is hereby incorporated by reference in its entirety.

[0002] The present disclosure generally relates to sleep disorder detection systems, and more specifically, to image analysis systems for determining sleep disorders.

Background Art

[0003] A range of respiratory diseases exist. Certain diseases can be characterized by specific events, such as apnea, hypopnea, and hyperventilation. Obstructive sleep apnea (OSA) is one form of sleep - disordered breathing (SDB) and is characterized by events such as closure or obstruction of the upper airway during sleep. This is caused by an abnormally small upper airway and a normal absence of muscle tone in the tongue area during sleep, in combination with the soft palate and the posterior oropharyngeal wall. When a patient suffers from the disease, the patient's breathing typically stops for periods ranging from 30 seconds to 120 seconds, and sometimes 200 to 300 times a night. As a result, excessive daytime sleepiness often occurs, which can cause cardiovascular disease and brain damage. This syndrome is a common disease, especially common in middle - aged overweight men, but patients have no awareness of the symptoms.

[0004] Other sleep - related diseases include Cheyne - Stokes respiration (CSR), obesity - hypoventilation syndrome (OHS), chronic obstructive pulmonary disease (COPD), etc. COPD encompasses any of a group of lower - airway diseases that have certain common characteristics. This includes an increase in resistance to air movement, an extended expiratory phase of breathing, and a decrease in normal elasticity in the lungs. Examples of COPD are emphysema and chronic bronchitis. The causes of COPD include chronic smoking (the primary risk factor), occupational exposure, air pollution, and genetic factors.

[0005] Continuous positive airway pressure (CPAP) therapy is used in the treatment of obstructive sleep apnea (OSA). By pressing the soft palate and tongue forward or backward against the posterior oropharyngeal wall, the application of CPAP therapy functions as an air splint, thereby preventing upper airway obstruction.

[0006] Non-invasive ventilation (NIV) provides ventilatory support to a patient through the upper airway, assisting deep breathing and / or maintaining adequate oxygen levels throughout the body by performing some or all of the respiratory function. Ventilation support is provided through a patient interface. NIV is used to treat conditions such as CSR, OHS, COPD, and chest wall disorders. In some forms, it can improve the comfort and effectiveness of these treatments. Invasive ventilation (IV) provides ventilatory support to patients who are no longer able to breathe effectively on their own and may be provided using a tracheostomy tube.

[0007] The treatment system may include a respiratory pressure therapy device (RPT device), an air circuit, a humidifier, a patient interface, and data management. The patient interface may be used to provide the wearer with an interface to the respiratory appliance, for example, by providing airflow to the airway inlet. Airflow may be provided via a mask to the nose and / or mouth, a tube to the mouth, or a tracheostomy tube to the patient's trachea.

[0008] One of the problems is determining whether a potential patient has a sleep disorder. While patients may self-report, many do not report or even recognize that they have a sleep disorder. Among patients, one current method is questionnaires that gather information from subjective sensations in an attempt to diagnose a sleep disorder. Because such questionnaires include subjective responses, the information can be inaccurate. Furthermore, there is no guarantee that patients will fill out such questionnaires or fill them out accurately. Another mechanism for diagnosing sleep disorders is a sleep laboratory, where patients are observed while sleeping. Sleep laboratories are effective but relatively labor-intensive and resource-intensive. Moreover, since being observed makes many patients uncomfortable, it is difficult to persuade patients to undergo observation in a sleep laboratory.

[0009] A system is needed that can accurately and individually determine sleep disorders. A system is needed that incorporates image data from users to analyze potential sleep disorders. A system is also needed that incorporates user images and other sensor data to determine sleep disorders. [Overview of the Initiative]

[0010] One disclosed example is a method for determining a patient's sleep disorder. A digital image, including the patient's face and neck, is provided from a storage device. Facial and neck features on the image are measured by determining landmarks on the image. At least one phenotype on the image is classified from previously identified phenotypes stored in a database. Measurements of at least one phenotype and at least one feature correlate with sleep disorder. A risk score for sleep disorder is determined based on the correlation between the phenotype and the measurement of at least one feature.

[0011] In other implementations of the disclosed exemplary method, images are provided by a mobile device camera. In another implementation, the exemplary method includes providing multiple images, including the patient's face and neck. In another implementation, the exemplary method includes measuring the patient's physiological measurements. The sleep disorder risk score is determined in part on the physiological measurements. In another implementation, correlation is performed using a machine learning model trained with images from a patient population and the respective sleep disorder scores of the patient population. In another implementation, the exemplary method includes storing the images, classified phenotypes, feature dimensions, and sleep disorder scores. The method also includes updating a database of patient populations using the stored classified phenotypes, feature dimensions, and patient sleep disorder scores. In another implementation, the sleep disorder is one of the following: obstructive sleep apnea (OSA), Cheyne-Stokes respiration (CSR), obesity hyperventilation syndrome (OHS), or chronic obstructive pulmonary disease (COPD), which are forms of sleep-disordered breathing (SDB). In another embodiment, the exemplary method includes determining a comorbidity risk score based on at least one phenotype. In another implementation, the exemplary method includes providing a video of the patient and determining the dynamic movement of one of the features. The risk score for sleep disorder is determined by the dynamic movement. In another implementation, the phenotype is encoded by a color on the image, and the image and color code are displayed on the screen. In another implementation, the color code of the phenotype represents the degree of correlation with sleep disorder. In another implementation, at least one phenotype is one of obesity / neck circumference, jutting jaw / mandible, and crowded / narrow upper airway. In another implementation, the exemplary method includes tailoring treatment for sleep disorder based on the determined phenotype. In another implementation, the exemplary method includes determining the severity of sleep disorder based on the determined sleep disorder score and determining therapy based on the severity of sleep disorder. In another implementation, the feature is neck dimensions. Neck dimensions correlate with tissue mass and stiffness parameters. The correlation with sleep disorder is related to tissue mass and stiffness parameters.

[0012] Another example of disclosure is a computer program product that, when executed by a computer, contains instructions that cause the computer to perform the above method. In further implementations, the computer program product is a non-temporary computer-readable medium.

[0013] Another example of disclosure is a system for determining a patient's sleep disorder. The system includes a storage device that stores digital images, including the patient's face and neck. The database stores previously identified phenotypes and dimensions of facial and neck features. The system includes a sleep disorder analysis engine coupled to the storage device and the database. The sleep disorder analysis engine identifies facial and neck features in an image by determining landmarks in the image. The engine classifies at least one phenotype in the image based on comparison with the database. The engine correlates at least one phenotype and at least one feature with a sleep disorder. The engine determines a risk score for the sleep disorder based on the correlation between phenotypes and features.

[0014] In other implementations of the disclosed system, images are provided by a mobile device's camera. In another implementation, a storage device stores multiple images, including the patient's face and neck. In yet another implementation, the exemplary system includes a sensor interface coupled to a sleep disorder analysis machine and sensors that measure the patient's physiological measurements. The sleep disorder risk score is determined in part based on the physiological measurements. In yet another implementation, correlation is performed using a machine learning model trained with images from a patient population and the respective sleep disorder scores of the patient population. In yet another implementation, a sleep disorder analysis engine stores images, classified phenotypes, feature dimensions, and sleep disorder scores. The engine updates a database using the stored classified phenotypes, feature dimensions, and patient sleep disorder scores. In yet another implementation, the sleep disorder is one of the following: obstructive sleep apnea (OSA), Cheyne-Stokes respiration (CSR), obesity hyperventilation syndrome (OHS), or chronic obstructive pulmonary disease (COPD), which are forms of sleep-disordered breathing (SDB). In yet another implementation, the sleep disorder analysis engine determines a comorbidity risk score based on the phenotype. In another implementation, the storage device contains a video of the patient. The sleep disorder analysis engine determines a dynamic movement of one of the features from the video, and the risk score for the sleep disorder is determined by that dynamic movement. In another implementation, the phenotype is encoded by a color on the image, and the image and color code are displayed on the screen. In another implementation, the color code of the phenotype represents the degree of correlation with the sleep disorder. In another implementation, the phenotype is selected from obesity / neck circumference, jutting jaw / mandible, and crowded / narrow upper airway. In another implementation, the sleep disorder analysis engine adapts the treatment for the sleep disorder based on the determined phenotype. In another implementation, the sleep disorder analysis engine includes determining the severity of the sleep disorder based on the determined sleep disorder score and determining the therapy based on the severity of the sleep disorder. In another implementation, the feature is neck dimensions. Neck dimensions correlate with tissue mass and stiffness parameters, and the correlation of the sleep disorder is related to the tissue mass and stiffness parameters.

[0015] The above summary is not intended to illustrate any or all embodiments of the present disclosure. Rather, the above summary merely provides some examples of novel embodiments and features described herein. The above features and advantages of the present disclosure, as well as other features and advantages, will be readily apparent from the following detailed description of representative embodiments and modes for carrying out the invention, in connection with the accompanying drawings and claims. [Brief explanation of the drawing]

[0016] [Figure 1] This describes a system for collecting data on sleep disorders. [Figure 2] This is a diagram of the components of a computing device used to capture facial data. [Figure 3A] This is a screenshot of the application interface used to capture patient images. [Figure 3B] This is a screen image of an overlay on a patient's image showing the facial phenotypic region. [Figure 4] This is a colored overlay screen image obtained on patient images, showing the correlation between facial and neck phenotypic regions and sleep disorders. [Figure 5] This is a flowchart illustrating the process of collecting patient image data based on patient input scans and analysis for the diagnosis of sleep disorders. [Modes for carrying out the invention]

[0017] This disclosure is susceptible to various modifications and alternative forms. Several representative embodiments are shown as examples in the drawings and will be described in detail herein. However, it should be understood that the invention is not intended to be limited to any particular form disclosed. Rather, this disclosure covers all modifications, equivalents, and alternatives that fall within the spirit and scope of the invention as defined by the appended claims.

[0018] The present invention can be embodied in many different forms. Representative embodiments are shown in the drawings and described in detail herein. This disclosure is an example or illustration of the principles relating to this disclosure and is not intended to limit the broader aspects of this disclosure to the embodiments shown herein. To that extent, elements and limitations described in, for example, the abstract, summary of the invention, and detailed description of the invention that are not expressly expressed in the claims should not be incorporated into the claims alone, comprehensively, implicitly, by inference, or otherwise. For the purpose of providing a detailed explanation herein, unless otherwise specified, singular forms include plural forms and vice versa. Also, the term "includes" means "includes but not limited to." Furthermore, approximate words such as "about," "almost," "substantially," and "approximately" can be used here to mean, for example, "at," "near," "near," "3-5%," "within manufacturing tolerances," or logical combinations thereof.

[0019] This disclosure enables patients to obtain a diagnosis of sleep disorders more quickly and conveniently by correlating data sources that can be acquired with minimal effort. Examples of sleep disorders include obstructive sleep apnea (OSA), a form of sleep-disordered breathing (SDB), Cheyne-Stokes respiration (CSR), obesity hyperventilation syndrome (OHS), and chronic obstructive pulmonary disease (COPD). Images of different physical characteristics can be determined by analyzing a patient population with the characteristics of individual patients determined by the scanning process. The scanning process allows patients to quickly measure their anatomical structures in the comfort of their homes using computing devices such as desktop computers, tablets, smartphones, and other mobile devices. Additional features related to sleep disorders can be analyzed from the scanned anatomical images.

[0020] In a favorable implementation, the technology may use an application downloadable from the manufacturer or a third-party server to a smartphone or tablet equipped with an integrated camera. Once launched, the application may provide visual and / or audio instructions. Following these instructions, the user (i.e., the patient) stands in front of a mirror and presses the camera button on the user interface. The activated process can then take a series of photographs of the patient, and subsequently, within, for example, a few seconds, it may acquire the face, neck, or other dimensions (based on a processor analyzing the photographs) for selecting the interface.

[0021] The user / patient may upload images or a series of images of their anatomical structures. Instructions provided by an image analysis application, stored on a computer-readable medium (when executed by a processor), detect various landmarks in the images, measure and scale the distances between such landmarks, and record these distances and other metrics in a data record. Measuring features may be relevant to the analysis of sleep disorders. Landmarks and features can also be used to classify phenotypes that may be associated with sleep disorders. The application may also collect other sleep disorder-related data from the patient, such as physiological and demographic data, which can be used by the image analysis engine to generate a sleep disorder score for the patient.

[0022] FIG. 1 shows an exemplary data collection system 100 that can be implemented to automatically measure body features from a patient's image and provide an analysis of the patient's sleep disorder. The data collection system 100 can generally include one or more of a server 110, a communication network 120, and a computing device 130. The server 110 and the computing device 130 can communicate via the communication network 120, which can be a wired network with a wired network 122, a wireless network 124, or a wireless link 126. In some versions, the server 110 can communicate unidirectionally with the computing device 130 by providing information to the computing device 130, or vice versa. In other embodiments, the server 110 and the computing device 130 can share information and / or processing tasks. The system can be implemented, for example, to enable the analysis of facial or other body images to determine sleep disorders, which will be described in more detail herein.

[0023] A database 160 is provided to collect data regarding a population of patients represented by patients 162. An external database 170 can include additional relevant collected data regarding patients that can be accessed by the server 110 for the purpose of sleep disorder analysis. The patient 162 can access the computing device 130, or other mobile computing devices such as a mobile phone 134 or a tablet 136. In this example, the patient 162 can wear a physiological sensor 150 that collects physiological data. The sensor 150 can include a heart rate sensor, an oxygen level sensor, an ECG sensor, a pulse rate sensor, and the like. The sensor 150 can communicate with a computing device such as the mobile phone 134 to record physiological data, particularly during the sleep period. As will be described later, the system 100 enables the collection of data for diagnosing whether the patient 162 may be suffering from a sleep disorder.

[0024] A second patient population, represented by patient 164, is previously diagnosed with a sleep disorder. Such patients 164 may be receiving treatment for the sleep disorder in the form of a respiratory pressure therapy device (RPT) 152. Examples of RPTs include continuous positive airway pressure (CPAP) devices that include both an operation sensor that monitors the operation of the device and a patient-based sensor that records the patient's physiological response to the device. In this example, data is collected from the second patient population 164, correlations are determined, and analysis of sleep disorders in other patients is enabled. The data may include physiological data, facial and neck image data, and demographic data. Treatment data, such as data collected by RPT 152, is correlated with the severity and type of sleep disorder in patients of the second group. Such a correlation with the sleep disorder may be performed via an exemplary machine learning server 112. Patients 164 of the second patient population may also access a computing device 130, or other mobile computing devices such as a mobile phone 134 or a tablet 136. Image data, physiological data, and treatment data may be collected via such devices of patient 164.

[0025] The machine learning server 112 and / or the computing device 130 can also communicate with a respiratory pressure therapy device such as RPT 152. The RPT device 152 in this example collects operational data regarding patient use, mask leak, and other relevant data and provides feedback regarding mask use and thus treatment of sleep disorders such as OSA. Data from the RPT device 152 is collected and correlated with the individual patient data of patient 164 using the RPT device 152 within the patient database 160.

[0026] The sleep disorder analysis engine, run by server 110, is used to determine potential sleep disorders by correlating image data and additional data from a first group of patients, represented by patient 162, with potential sleep disorders. The machine learning server 112 may access database 160 to provide machine learning models for better data correlation with sleep disorders. As described in detail, the sleep disorder analysis engine receives image data to be stored and accesses a database of data, such as phenotypes, from a patient population correlated with sleep disorders. The sleep disorder analysis engine identifies anatomical features of the patient's images by determining landmarks on the images. Such anatomical features may include facial and neck anatomical features on the images. The analysis engine classifies at least one phenotype on the images based on comparison with the database. The phenotype and at least one feature correlate with sleep disorders. A risk score for sleep disorders is determined based on the correlation between the phenotype and the feature.

[0027] Therefore, the relevant data can be correlated with new patient anatomical dimension data derived from points identified from the patient's images. As described later, the server 110 runs a sleep disorder analysis engine by collecting data from multiple patients stored in the database 160 and outputting the likelihood of sleep disorders for each patient.

[0028] The computing device 130 may be a desktop or laptop computer 132, or a mobile device such as a smartphone 134 or tablet 136. Figure 2 shows a typical architecture 200 of the computing device 130. The computing device 130 may include one or more processors 210. The computing device 130 may also include a display interface 220, a user control / input interface 231, sensors 240 and / or sensor interfaces for one or more sensors, an inertial measurement unit (IMU) 242, and non-volatile memory / data storage 250.

[0029] Sensor 240 may be one or more cameras (e.g., CCD charge-coupled elements or active pixel sensors) integrated into the computing device 130, such as those found in a smartphone or laptop. Alternatively, the computing device 130 may be a desktop computer, but may include a sensor interface for coupling with an external camera, such as the webcam 133 shown in Figure 1. Other exemplary sensors that can be used to support the method described herein, and which may be integrated with or located outside the computing device, include stereoscopic cameras for capturing three-dimensional images, or photodetectors capable of detecting reflected light from a laser or strobe / structured light source.

[0030] The user control / input interface 231 allows the user to enter commands or respond to prompts or instructions provided to the user. These may include, for example, a touch panel, keyboard, mouse, microphone, and speaker.

[0031] The display interface 220 may include a monitor, LCD panel, etc., for displaying prompts, output information (such as facial measurements or interface size recommendations), and other information such as an capture display, as will be described in more detail below.

[0032] The memory / data storage 250 may be the internal memory of a computing device, such as RAM, flash memory, or ROM. In some embodiments, the memory / data storage 250 may be external memory linked to the computing device 130, such as an SD card, server, USB flash drive, or optical disc. In other embodiments, the memory / data storage 250 may be a combination of external and internal memory. The memory / data storage 250 includes stored data 254 and processor control instructions 252 that instruct the processor 210 to perform a specific task. The stored data 254 may include data received by a sensor 240, such as captured images, and other data provided as a component part of an application. The processor control instructions 252 may also be provided as a component part of an application.

[0033] As described above, body images, such as images of the face or neck, can be captured by a mobile computing device such as a smartphone 134. A suitable application running on the computing device 130 or server 110 can provide relevant three-dimensional facial and other anatomical data to aid in the determination of sleep disorders. The application may use a suitable method of facial scanning. Such applications may include capture from StandardCyborg (https: / / www.standardcyborg.com / ), an application from Scandy Pro (https: / / www.scandy.co / products / scandy-pro), a Beauty3D application from Qianxun3d (http: / / www.qianxun3d.com / scanpage), an Unre 3D FaceApp (http: / / www.unre.ai / index.php?route=ios / detail), and an application from Bellus3D (https: / / www.bellus3d.com / ). The detailed process of facial scanning includes the techniques disclosed in WO2017000031, which is incorporated herein by reference in its entirety.

[0034] Imaging routines can also be combined with 3D facial models to estimate dimensions. For example, 3D images can be captured using a 3D scanner or depth sensor from a mobile device, such as the TrueDepth camera in an Apple phone. Multiple 3D images may be captured to create an accurate 3D model. For example, one way to capture a series of images is to rotate the head from side to side while keeping the rest of the body (from the neck down) fixed. This is problematic because it can cause distortion or errors in the neck when reconstructing the 3D model. To mitigate this, 3D images can be captured with the entire camera unit rotating along a vertical axis while keeping the head, neck, and body fixed (i.e., not rotating relative to each other). One example of this method is to have the user sit in a swivel chair. A device with a camera can be mounted on a stand and a series of 3D images can be taken while the chair slowly rotates. This allows the 3D camera to properly capture facial and head features and accurately reconstruct the 3D model.

[0035] One such application is application 260 for measuring facial features and / or sizing patient interfaces, which may be an application downloadable to mobile devices such as smartphones 134 and / or tablets 136. Application 260, which may be stored in a computer-readable medium such as memory / data storage 250, includes programmed instructions for processor 210 to perform specific tasks related to measuring anatomical features and / or patient sizing. The application also includes data that may be processed by algorithms of an automated methodology. Such data may include data records, reference features, and correction factors, as will be described in more detail below.

[0036] Application 260 is executed by processor 210 and measures anatomical details such as the facial and neck features of a patient using two-dimensional or three-dimensional images. This method can generally be characterized as including three or four distinct stages, namely, a pre-acquisition stage, an acquisition stage, a post-acquisition image processing stage, and a comparison and output stage.

[0037] In some cases, an application for measuring facial features and sizing a patient interface may control the processor 210 to output a visual display containing reference features on the display interface 220. The user may position the features adjacent to their own facial features by moving the camera, for example. The processor can then capture and store one or more images of facial features in relation to the reference features, provided certain conditions, such as alignment conditions, are met. This can be done with the assistance of a mirror, which displays the reference features and the user's face to the camera. The application then controls the processor 210 to identify specific facial features in the images and measure the distance between them. Image analysis processing may convert facial feature measurements, such as the number of pixels, into standard mask measurements based on the reference features using scaling factors. Such values ​​may be expressed in standardized units of measurement, such as meters or inches, and in such units that are suitable for mask sizing.

[0038] Additional correction factors may be applied to the measurements. Facial feature measurements can be compared to data records that include measurement ranges corresponding to different patient interface sizes for specific patient interface configurations, such as nasal masks and FFMs. A recommended size is then selected and output to the user / patient as a recommendation based on the comparison. Such a process can be conveniently influenced within any user's comfortable location. An application can perform this method within seconds. In one example, the application performs this method in real time.

[0039] In the pre-capture stage, the processor 210 assists the user, among other things, in establishing appropriate conditions for capturing one or more images for sizing. Some of these conditions include, for example, appropriate lighting and camera orientation, as well as motion blur caused by an unstable hand holding the computing device 230.

[0040] A user can conveniently download an application that performs automatic measurement and sizing on the computing device 130 from a server such as a third-party application store server to their computing device 130. Once downloaded, such an application can be stored in the computing device's internal non-volatile memory, such as RAM or flash memory. The computing device 230 is preferably a mobile device such as a smartphone 134 or a tablet 136.

[0041] When a user launches the application, the processor 210 may prompt the user via the display interface 220 to provide patient-specific information such as age, gender, weight, height, and lifestyle factors related to the diagnosis of sleep disorders (e.g., diet, alcohol, opioid and other drug use, smoking history, physical and mental activity patterns, occupation, exposure to pollution, and geographical location). However, the processor 210 may prompt the user to input this information at any time, such as after the user's facial features have been measured. The processor 210 may also present tutorials, which may be presented audibly and / or visually, as provided by the application, to help the user understand their role in the process. Prompts may also require information about the type of patient interface, such as the nose or the entire face, and the type of device on which the patient interface will be used. In addition, in the pre-intake stage, the application may estimate patient-specific information through artificial intelligence based on information already collected by the user and machine learning techniques, such as after receiving an captured image of the user's face.

[0042] When the user is ready to proceed (which may be indicated by user input or a response to a prompt via the user control / input interface 231), the processor 210 activates the sensor 240 according to the instructions of the application's processor control instruction 252. The sensor 240 is preferably a front-facing camera of the mobile device, located on the same side as the mobile device's display interface 220. Cameras are typically configured to capture two-dimensional images. Mobile device cameras that capture two-dimensional images are ubiquitous. This technology takes advantage of this ubiquity to avoid the burden on the user that would otherwise be placed on obtaining specialized equipment.

[0043] Almost simultaneously with the activation of the sensor / camera 240, the processor 210 presents an acquisition display on the display interface 220, in accordance with the instructions of the application 260. The acquisition display may include a live action preview of the camera, a reference feature, a target box, and one or more status indicators, or any combination thereof. In this example, the reference feature is displayed in the center of the display interface and has a width corresponding to the width of the display interface 320. The vertical position of the reference feature may be such that the top edge of the reference feature touches the top edge of the display interface 220, or the bottom edge of the reference feature touches the bottom edge of the display interface 220. Part of the display interface 220 displays a live action preview of the camera, typically showing face and neck features captured in real time by the sensor / camera 240 when the user is in the correct position and orientation.

[0044] A reference feature is a (predetermined) feature known to the computing device 130, providing the processor 210 with a frame of reference that enables the processor 210 to scale the captured image. Preferably, the reference feature is a feature other than the user's face or anatomical features. Therefore, during the image processing stage, the reference feature helps the processor 210 determine when certain alignment conditions are met, such as during the pre-capture stage. The reference feature may be a Quick Response (QR) code, a known sample, or a marker, which can provide the processor 210 with specific information, such as scaling information, orientation, and / or any other desired information that can be arbitrarily determined from the structure of the QR code®. The shape of the QR code® is square or rectangular. When displayed on the display interface 220, the reference feature has predetermined dimensions, such as in millimeters or centimeters, and its value can be encoded in the application and communicated to the processor 210 at the appropriate time. The actual dimensions of the reference feature may vary across different computing devices. In some versions, the application may be configured so that the dimensions of the reference feature when displayed on a particular model are specific to the computing device model for which they are already known. However, in other embodiments, the application may instruct the processor 210 to obtain certain information from the device 130, such as display size and / or zoom characteristics, so that the processor 210 can calculate the real-world / actual dimensions of the reference features displayed on the display interface 220 via scaling. Nevertheless, the actual dimensions of the reference features displayed on the display interface 220 of such a computing device are generally known before post-capture image processing.

[0045] Along with reference features, a target box may be displayed on the display interface 220. The target box allows the user to align a specific component within the capture display 222 within the target box, which is desirable for successful image capture.

[0046] The status indicator provides the user with information about the process status. This eliminates the need for the user to make significant adjustments to the sensor / camera alignment before image acquisition is complete.

[0047] Therefore, when the user holds the display interface 220 parallel to the anatomical feature being measured and presents the user display interface 220 to a mirror or other reflective surface, the reference feature is prominently displayed and overlaid with a real-time image seen by the camera / sensor 240 and reflected by the mirror. This reference feature can be fixed near the top of the display interface 220. The reference feature is displayed prominently in this manner, at least partially, so that the sensor 240 can clearly see the reference feature and the processor 210 can easily identify the feature. Furthermore, the reference feature can be overlaid with a live view of the user's face and neck to avoid user confusion.

[0048] The user may also be instructed by the processor 210 via the display interface 220, by audible instructions via the speaker of the computing device 130, or by a tutorial to position the display interface 220 within the plane of the anatomical feature to be measured. Since the final captured image is two-dimensional, plane alignment allows the scale of the reference feature to be equally applicable to the facial feature measurement. In this regard, the distances between the mirror and both the user's facial feature and the display are approximately the same.

[0049] Once the user is positioned in front of the mirror and the display interface 220, which includes the reference feature, is roughly positioned and planely aligned with the anatomical feature to be measured, the processor 210 checks for specific conditions that help ensure sufficient alignment. As previously mentioned, one exemplary condition that can be established by the application is that the process cannot proceed unless the entire reference feature is detected within the target box 228. If the processor 210 detects that the reference feature is not fully positioned within the target box, it may prohibit or delay image acquisition. The user can then move their face or neck along with the display interface 220 to maintain planarity until the reference feature displayed in the live action preview is positioned within the target box. This helps optimize the alignment of the feature and the display interface 220 with respect to the mirror for image acquisition.

[0050] Once the processor 210 has detected the entire reference feature within the target box, it can read the IMU 242 of the computing device to detect the tilt angle of the device. The IMU 242 may include, for example, an accelerometer or a gyroscope. Thus, the processor 210 can evaluate the tilt of the device, for example by comparing it with one or more thresholds, and confirm that it is within an appropriate range. For example, if it is determined that the computing device 130, and by extension the display interface 220 and the desired feature, is tilted in any direction within approximately ±5 degrees, the process may proceed to the capture stage. In other embodiments, the tilt angle for continuation may be within approximately ±10 degrees, ±7 degrees, ±3 degrees, or ±1 degree. If excessive tilt is detected, a warning message may be displayed or sounded to correct the undesirable tilt. This is particularly useful in assisting the user in preventing or mitigating excessive tilt, especially in the front-to-back direction. Failure to correct this can lead to measurement errors, as the captured reference image will not have an appropriate aspect ratio.

[0051] Once the alignment is determined by the processor 210 according to application control, the processor 210 proceeds to the acquisition phase. The acquisition phase is preferably performed automatically once the alignment parameters and other preceding conditions are met. However, in some embodiments, the user may initiate the acquisition in response to a prompt.

[0052] When image acquisition begins, the processor 210 acquires n images, preferably multiple images, via the sensor 240. For example, the processor 210 may acquire approximately 5 to 20 images, 10 to 20 images, or 10 to 15 images via the sensor 240. The amount of images acquired may be time-based. In other words, the number of images acquired may be based on the number of images of a predetermined resolution that the sensor 240 can acquire within a predetermined time interval. For example, if the number of images that the sensor 240 can acquire at a predetermined resolution within 1 second is 40, and the predetermined time interval for acquisition is 1 second, the sensor 240 will acquire 40 images for processing by the processor 210. The amount of images may be defined by the user, or it may be detected by the server 110 based on artificial intelligence or machine learning of the detected environmental conditions, or based on an intended accuracy target. For example, if high accuracy is required, more acquired images may be needed. While it is preferable to acquire multiple images for processing, one image may be considered and successfully used to obtain accurate measurements. However, using multiple images allows for obtaining average measurements. This can reduce errors / inconsistencies and potentially improve accuracy. The images may be placed in the memory / data storage 250's stored data 254 by the processor 210 for post-acquisition processing.

[0053] Once an image is acquired, it is processed by the processor 210 to detect or identify anatomical features / landmarks and measure the distances between them. The resulting measurements can be analyzed for correlation with sleep disorders. Other analyses, such as phenotypic classification, can be performed by analyzing the image. Alternatively, this processing may be performed by the server 110 that receives the transmitted acquired image, and / or on a computing device (e.g., a smartphone) operated by the user. The processing may be performed by a combination of the processor 210 and the server 110.

[0054] The processor 210, controlled by the application, retrieves one or more captured images from the stored data 254. The images are then extracted by the processor 210, and each pixel constituting the two-dimensional captured image is identified. Next, the processor 210 detects specific, pre-specified facial features within the pixel configuration.

[0055] Detection may be performed by the processor 210 using edge detection, such as Canny, Prewitt, Sobel, or Robert edge detection. For example, these edge detection techniques / algorithms help identify the location of specific face and neck features within the pixel configuration that correspond to the actual face and neck features presented for image acquisition. For example, the edge detection technique can first identify the face in the image and then identify the pixel locations in the image that correspond to specific face and neck features, such as each eye and its boundary, the mouth and its corners, the left and right nostrils, the selion, the splamenton, the glabella, and the left and right nasolabial folds. The processor 210 can then mark, tag, or store each of these specific pixel locations of features. Alternatively, or if such detection by the processor 210 / server 110 fails, pre-specified face and neck features may be manually detected, marked, tagged, or stored by a human operator accessing the acquired image via the user interface of the processor 210 / server 110.

[0056] Once the pixel coordinates of these facial features are identified, application 260 controls processor 210 to measure the pixel distance between specific identified features. For example, the distance is generally determined by the number of pixels of each feature and may include scaling. For instance, the pixel width of the nose can be determined by measuring between the left and right alae, and / or the pixel height of the face can be determined by measuring between the serion and supramenton. Other examples include measuring tongue fat, neck circumference, and skeletal parameters. Other examples include pixel distances between each eye, between the corners of the mouth, and between the left and right nasolabial folds to obtain additional measurement data for specific structures such as the mouth. Distances between facial and neck features can also be measured further. In this example, specific facial and neck dimensions are used to analyze the likelihood of sleep disorders. Skeletal parameters can be predicted from the dimensions, and the results of head standard photo analysis may correlate with obstruction caused by obstructive sleep apnea. Measurements and corresponding anatomical features may correlate with sleep disorders through machine learning analysis.

[0057] Once pixel measurements of pre-specified facial and neck features are obtained, an anthropometric correction factor may be applied to the measurements. It should be understood that this correction factor can be applied before or after applying a scaling factor, as described below. The anthropometric correction factor can compensate for errors that may occur in the automated process. These errors may be observed to occur consistently from patient to patient. The correction factor, which can be empirically extracted from mass examinations, helps to bring the results closer to the actual measurements and to mitigate or eliminate size errors. This correction factor can be improved or enhanced over time because the measurement and sizing data for each patient is communicated from their respective computing devices to a server 110 where such data can be further processed to improve the correction factor.

[0058] The measurements can be scaled from pixel units to other values ​​that accurately reflect the distance between the patient's face or neck features presented for image acquisition. A reference feature can be used to obtain one or more scaling values. Thus, the processor 210 also determines the dimensions of the reference feature, which may include pixel width and / or pixel height (x and y) measurements (e.g., number of pixels) of the entire reference feature. The pixel dimensions of the many squares / dots that make up the QR code® reference feature, and / or more detailed measurements of the pixel area occupied by the reference feature and its components can also be determined. Thus, each square or dot of the QR code® reference feature can be measured in pixel units to determine a scaling factor based on the pixel measurement of each dot, and then the average can be calculated across all the measured squares or dots. This improves the accuracy of the scaling factor compared to measuring the full size of the QR code® reference feature once. However, it should be understood that whatever measurements of the reference feature are taken, those measurements can be used to scale the pixel measurements of the reference feature to the corresponding known dimensions of the reference feature.

[0059] Once the reference feature measurements are obtained by the processor 210, a scaling factor is calculated by the processor 210 according to the application's control. The pixel measurements of the reference feature are associated with the known corresponding dimensions of the reference feature displayed by the display interface 220 for image acquisition in order to obtain a transformation or scaling factor. Such a scaling factor may be in the form of length / pixel or area / pixel A2. In other words, the known dimensions may be divided by the corresponding pixel measurements (e.g., number).

[0060] Next, processor 210 applies a scaling factor to the facial feature measurements (number of pixels) to convert the measurements from pixels to other units to reflect the actual distances between features. This may typically involve multiplying the number of pixels of the distance between facial and neck features related to possible sleep disorders by the scaling factor.

[0061] These measurement and calculation steps for both facial and neck features and reference features are repeated for each captured image until scaled and / or corrected feature measurements are obtained for each image in the set.

[0062] Next, the corrected and scaled measurements of the image set can be arbitrarily averaged by the processor 210 to obtain final measurements of the anatomical structures of the patient's face and neck. Such measurements may reflect the distances between the features of the patient's face and neck.

[0063] During the comparison and output phase, the results from the post-import image processing stage can be directly output (displayed) to stakeholders, or the possibility of sleep disorders can be obtained by comparing them with data records. This data can also be used to determine the optimal therapy or treatment for sleep disorders.

[0064] Once all measurements have been determined, the result (e.g., the mean) can be displayed to the user via the display interface 220 by the processor 210. In one embodiment, this may terminate the automated process. The user / patient can record the measurements for further use.

[0065] Alternatively, the final measurements may be automatically or by user command transferred from the computing device 130 to the server 110 via the communication network 120. The server 110 or a server-side individual can perform further processing and analysis to determine an appropriate patient interface and patient interface size.

[0066] In a further embodiment, final face and neck feature measurements, reflecting the distances between the patient's actual face and neck features, are compared by the processor 210 to features indicating sleep disorders, such as data records. These data records may be part of the application. For example, these data records may include a lookup table accessible by the processor 210, which may contain distance / values ​​of the patient's face features correlated with sleep disorders. The data records may include multiple tables, many of which may correspond to ranges relevant to the classification of sleep disorders. The analysis engine performs symptom matching of the patient against existing image data from patients diagnosed with sleep disorders such as OSA. Various relevant anatomical regions can be highlighted to indicate different symptoms of sleep disorders. Areas may be measured to determine whether abnormal dimensions indicate a sleep disorder. For example, neck measurements might reveal stiffness in the neck tissue, which could indicate a sleep disorder. These regions may be further analyzed by determining visual features such as color or texture, which are further signs of sleep disorders. For example, the eyes can be identified and their color analyzed to determine whether the patient has red eyes.

[0067] This application can also record video of a patient's movements and extract images from the video. The video provides better image data for determining three-dimensional measurements. For example, the video provides data on the dynamic features of the face or neck. Still images from the motion video are taken at various angles / head and neck positions to obtain the patient's "difference" features. The video file can be stored in the same way as the captured images.

[0068] Figure 3A shows an exemplary interface 300 for capturing anatomical features such as facial and neck features. Interface 300 is generated by application 260. After images such as image 302 are captured, or after a series of images or video are captured, a grid 304 can be applied to determine different landmarks on the patient's face and neck in this example.

[0069] Additional point markers can be derived from image 302 to provide geodesic or specular measurements. Therefore, by collecting as many landmarks as possible (thousands, not just a few), the surface of the face can be constructed. A surface is necessary so that geodesic measurements can be performed. Typically, geodesic measurements are performed between the same landmarks as point-to-point measurements. The collected landmarks are fitted into a statistical shape model (SSM) (also known as a 3D morphable model (3DMM)). This provides a large amount of information that can be linked to other data. This may include other pre-trained models such as whole-head models (for calculating conduit size), deformable mask models, comfort prediction models, and custom mask models. Color and image texture can be collected from the image to facilitate fitting the shape model to the 2D image. Image depth may be determined by predicting image depth (using small changes in perception between consecutive images) or by actual depth images collected from a device with a depth sensor, such as a smartphone. Similar to texture, depth data aids in fitting and allows for more accurate scaling.

[0070] Figure 3A shows the upload status interface 310 displaying the survey input selection 312. While associating facial data to determine anatomical features and performing sleep disorder analysis based on image-related data, the application may collect subjective patient input data through additional display interfaces.

[0071] Figure 3B shows a series of output interfaces 320 that display different highlighted regions that may indicate signs of sleep disorder. Interface 320 displays the captured image 330. The eye region 332 is highlighted, the region below the eyes 334 is highlighted, and the neck region 336 is highlighted. The identified regions 332, 334, and 336 are indicators of measurements that correlate with sleep disorder. Various data may be obtained from the image in relation to the identified measurements or phenotypes. In this example, regions may be colored to highlight a high correlation with sleep disorder.

[0072] Additional regions of interest can be identified from the captured images. Figure 4 shows a series of interfaces 410, 420, and 430 generated using the initial patient image 400 captured by the patient. Different regions of the images in each of interfaces 410, 420, and 430 are shaded in different colors to identify regions of interest relevant to sleep disorder diagnosis. For example, the images in interfaces 410, 420, and 430 have a red shaded region 432 around the eyes, which shows a high correlation with sleep disorders. Other regions, such as the region below the eyes 434, are shaded in blue, which shows a low correlation with sleep disorders. Other regions in the images of interfaces 420 and 430, such as the neck region 436, are also shaded in red to show a high correlation with sleep disorders.

[0073] The images of interfaces 420 and 430 also show the cheek region 438 and the forehead region 440 identified and classified from the image data. The shading colors can represent different classifications of phenotypic measurements for sleep disorders. In this example, the classifications are color-coded based on their correlation with sleep disorders to aid in sleep sequence analysis. Thus, a color scale from green (low correlation) to red (high correlation) can be used in the final image to identify phenotypes from the anatomical regions shown in the image.

[0074] Sleep disorder diagnosis can be performed by combining data. This data combination may include: a) anatomical measurement data obtained from self-images acquired from a mobile device or webcam; b) phenotypic classification based on self-images; c) dynamic data from the patient's 3D movement; d) physiological data collected during sleep by patient sensors such as Sensor 150; and e) subjective survey data collected from the patient after sleep.

[0075] Automated measurements of various anatomical features can be analyzed by determining the distances between anatomical landmarks or other types of markers. For example, for the neck, a model can be developed that correlates neck dimensions with tissue mass and stiffness parameters. This model can then be applied to estimate airway collapse tissue mass and stiffness parameters used as features in OSA classification. For example, if a patient is determined to have high tissue mass and low neck stiffness, it indicates a high probability of sleep apnea syndrome. Thus, the soft tissue of the neck can be classified as normal, obese, or small in the maxilla and mandible. The bony enclosure can also be measured. These two measurements can correlate with airway size; normal soft tissue and a large bony enclosure indicate a normal airway size, while obese or small tissue and bone structure indicate a narrowed airway size that correlates with sleep disorders. This distribution of airway tissue based on neck features and facial bone features can predict an increased likelihood of postural orthostatic sleep apnea syndrome, for example, a significantly higher incidence of events in the supine position compared to other positions. Predictions of sleep disorders and their severity based on various characteristic measurements can be used to output recommended therapies or treatments. For example, identifying postural orthostatic apnea syndrome based on the distribution of airway tissue may lead to outputting patient treatments, including positional therapy. As another example, if an anatomical model predicts a large tongue or other soft tissue, or excessively elastic tissue anterior to the airway, the patient may be considered more susceptible to postural orthostatic apnea syndrome.

[0076] Regarding phenotypes, analysis of collected images may allow for the detection of phenotypes influenced by sleep disorders. Phenotypes associated with sleep disorders can be classified from the patient's anatomical images. Such phenotypes can be classified as phenotypes of interest based on studies and observed phenotypes in patients with sleep disorders. Such data can be loaded into database 160 for the purpose of phenotype classification. Examples of phenotypes associated with sleep disorders include obesity / neck circumference (related to the severity of sleep apnea), jaundice / mandible linked to greater airway collapse during sleep, and a crowded / narrow upper airway. Another set of factors that may be related to sleep order and its severity may include high AHI vs. low AHI, apnea-to-hypnea ratio, sleep fragmentation, snoring severity / frequency, prevalence of respiratory effort-related awakenings, degree of daytime sleepiness, cognitive impairment, morning headache, nocturia, hypertension, type of respiratory instability (high loop gain), arousal threshold (e.g., does the patient wake up during mild blood flow restriction or after prolonged apnea?), sympathetic / parasympathetic tension, stress, anxiety, and depressive symptoms. These factors may also be used to suggest one or more optimal therapies to the user (e.g., CPAP, positional therapy, mandibular appliances). They may also be used to better optimize one or more parameters of the therapy, such as CPAP settings.

[0077] Patients may be classified based on a phenotypic classification of their sleep disorder. This phenotypic classification can correlate with various types of therapy as part of the recommendations. For example, certain phenotypes may indicate suitability for neurostimulation or postural therapy. Other phenotypes can also be used to select accessories for devices or therapeutic equipment, such as the type of mask for RPT devices.

[0078] Phenotypic data may also correlate with the risk of comorbidities in patients and recommendations for the treatment of those comorbidities. For example, a diagnosis of a comorbidity such as diabetes may lead to recommendations for dietary changes and / or exercise as complementary therapies. Another example is a diagnosis of insomnia in which cognitive therapy may be recommended.

[0079] The diagnosis and treatment of OSA may incorporate a process of detecting fatigue as a symptom of OSA from image analysis. Thus, image analysis can determine dark circles around the eyes by detecting the area under the eyes that has discoloration / dark skin color compared to the rest of the person's face / normal skin color. A second symptom that may be detected is bloodshot and red eyes. This may be determined by segmenting the white area of ​​the eye in a facial image. The number of pixels that are red may be determined. The routine then determines whether the ratio of red pixels to white pixels in the eye area exceeds a predetermined threshold for “normal” eyes versus “bloodshot” eyes. Another symptom may be droopy eyes, which may be detected by determining the aspect ratio of open eyes in an image. Preferably, benchmarks are used to avoid anatomical biases such as ethnically related anatomical features. Such benchmarks may be obtained by taking an image of the person when instructed to open their eyes as wide as possible and measuring it compared to the shape of the eyes at “rest.” Another symptom may be reaction time, which may be determined by a test performed by an application, such as displaying markers for the user to follow. The eye's response to a marker can be measured based on how accurately the eye tracks the marker, or the delay between displaying the marker and the eye tracking its movement. Another symptom may be the patient's skin tone. For example, since patients may experience increased flushing after treatment for OSA, a comparison of the color of facial images before and after treatment may be used.

[0080] Image analysis can also provide a measure of “fatigue.” Changes in fatigue throughout the day may indicate OSA or its severity. For example, a patient with severe OSA might be assigned a fatigue score of 8 / 10 upon waking. The patient with OSA may remain tired throughout the day, peaking at 10 / 10 after lunch or in the evening. A normal person without OSA will wake up refreshed and have a lower fatigue score (e.g., 1 or 2 out of 10). As the day progresses, a normal person may experience progressive fatigue to a “normal” level, perhaps peaking at 4 or 5 at specific times of the day / night. The flux or rate of change in fatigue throughout the day can be a predictor of OSA. For example, a person without OSA may wake up refreshed and feel a “normal” level of fatigue throughout the day. In contrast, a patient with OSA will be tired upon waking and remain tired throughout the day.

[0081] 2D image analysis may also be incorporated. For example, 2D images can be used to predict the rest of the head / neck. This can be achieved, for example, by using a pair of PCA models. Preferably, multiple 2D images, such as various front and side views, and views from below and above, are incorporated into the prediction. Other dimensions determined from the images may be incorporated into the estimation of neck dimensions. For example, the width of the neck can be used to obtain the frontal circumference.

[0082] Furthermore, 2D images captured over time can be incorporated into additional analyses. For example, consecutive 2D images taken during and immediately after breathing can provide additional data. For instance, if a user takes a deep breath before speaking or during a break, this may be an indicator of OSA (Obstructive Surgical Aspiration). 2D images showing head movement or rotation, and jaw opening and closing can be used to determine the biomechanics of the jaw. Comparing 2D images of a seated patient with images of a reclining patient may reveal the effects of gravity on soft tissues. Such images can also be used to determine the possibility of airway compression during sleep. 2D images of blinking and eye tracking can determine indicators of fatigue. 2D images showing an open mouth may be an indicator of mouth breathing. 2D images of posture can be used to determine fatigue.

[0083] In this example, subjective patient data input may be collected via a user application running on the computing device 230. The user application may be part of user application 260 that instructs the user to acquire facial and neck landmark features, or it may be a separate application. This may also include subjective data obtained through a questionnaire containing questions for collecting data on sleep disorder symptoms.

[0084] For example, questions may relate to relevant user behaviors such as sleep characteristics. Subjective questions may include whether the user experiences a dry mouth upon waking, whether they breathe through their mouth, and what their comfort preferences are. Such sleep information may include sleep duration, the user's sleep patterns, and external influences such as temperature and stressors. Subjective data can be simple, such as a score for comfort, or more detailed responses. Such subjective data may also be collected through a graphical interface. Objective data, such as patient demographic data including age, sex, and location, may also be collected through the interface. The application may also include interfaces with specific questions to narrow down phenotypic classifications, or questions from standard sleep-related surveys. Such standard surveys may include STOP BANG, Berlin, Pittsburgh Sleep Quality, Insomnia Severity Index, and Chronic Fatigue Index. There may be multiple surveys. Furthermore, subjective data can be combined with anatomical images for a more accurate diagnosis. For example, certain anatomical features may predict a high apnea-hypopnea index (AHI), while anxiety predicts increased sympathetic activity, making the illness appear milder. Such questions can be asked while displaying images of patients with the identified specific phenotype, in order to support patient information sharing.

[0085] The collected patient input data can be assigned to the patient database 160 in Figure 1. The application may ask additional questions via the interface to assist in the selection of therapy or treatment. For example, the survey may record the patient's responses to questions such as whether they prefer therapy A to therapy B. Other subjective data related to the patient's psychological safety regarding potential treatments may be collected. For example, questions may be asked and input collected such as whether the patient becomes claustrophobic when wearing the mask, or how psychologically comfortable the patient feels when wearing the mask next to their bed partner. Other questions regarding preferred sleeping positions may also be taken into consideration, as well as questions such as whether the patient likes to move around frequently at night, or whether the patient prefers "freedom" in the form of a tube-up mask that allows for more movement. Alternatively, if the patient tends to lie still on their back or side, a conventional mask with a tube hanging from the mouth may also be acceptable. Subjective input data from patients can be used as input for both the diagnosis of sleep disorders and the effectiveness of treating sleep disorders.

[0086] The sleep disorder score can be used by healthcare professionals to recommend treatment or therapy. Healthcare professionals can also receive treatment effectiveness scores. The application can also facilitate pre-consultations with healthcare professionals regarding sleep, or provide patients with telemedicine consultations for greater convenience. Based on the consultation results, treatment may be recommended based on the data and the determined severity of the sleep disorder. For example, if the input data determines that the sleep disorder is mild, therapies such as mandibular repositioning devices (MRD), positional stimulation, nerve stimulation, cognitive behavioral therapy (CBT), or cognitive behavioral therapy for insomnia (CBTi) may be recommended. If the sleep disorder is more severe, personal consultation may be recommended, and treatment devices such as RPT, medication, or automated sleep coaching programs may be offered.

[0087] After the initial scoring, system 100 continues to collect patient-related data. For example, if a patient in the first group, represented by patient 162, is diagnosed with a sleep disorder and prescribed medication, that patient may be considered to belong to the second group of patients, represented by patient 164. Additional follow-up data regarding the effectiveness of the treatment and the accuracy of the diagnosis may be added to database 160. Feedback from new patients to the second group represented by patient 164 may be used to improve the determination of sleep disorders and treatment effectiveness. Furthermore, the patients in the second group represented by patient 164 may have their images and other data updated periodically. This allows for the prediction of future illnesses or symptoms based on periodic re-evaluation of the new data.

[0088] If therapy is recommended, treatment data can be used to confirm screening results. That is, data from the therapy or treatment device can be combined with data from the questionnaire / screener interface for a second group of patients represented by patient 164 to complement the prediction of OSA.

[0089] For example, in a specific market, healthcare professionals may recommend a treatment device such as RPT during a trial period (e.g., 4 weeks). During the trial period, System 100 can compare the results from the treatment device and continue to provide further recommendations. System 100 can also recommend therapies to treat sleep disorders and continue monitoring the patient. If the initial therapy fails, another therapy may be recommended.

[0090] Data correlation may occur based on an analysis of patient populations with known sleep disorders. Images may be collected from such patients in conjunction with applications on computing devices operated by those patients. Such applications may use processing similar to the image processing described above. Other physiological factors from patients may be collected from physiological sensors. Alternatively, additional data may be collected from therapeutic devices such as RPTs. Furthermore, survey data similar to those detailed above may be collected.

[0091] As explained above, operational data for each RPT can be collected for a large number of patients. This may include usage data based on when each patient operates the RPT. Therefore, compliance data, such as how long and how often a patient uses the RPT over a given period, can be determined from the collected operational data. Leak data can be determined from operational data, such as analysis of flow rate data or pressure data. Mask switching data can be derived using acoustic signal analysis to determine whether a patient is switching masks.

[0092] The data collection can be used to train a machine learning model to correlate images of phenotypic features with sleep disorders. Another application may involve using machine learning to correlate phenotypic classification with sleep disorders for the most effective treatment. Inputs to the machine learning model may include various data that can be evaluated in the model design process. In this example, inputs to the machine learning model include physical measurements of facial and neck features, phenotypes derived from facial and neck images, physiological data, and patient input data. The training set is derived from data collected from a second group of patients represented by patient 164 in Figure 1, and includes sleep disorder data and treatments received by the patients. For example, treatments may include the use of devices such as RPT, medication, cognitive behavioral therapy for insomnia (CPTi), mandibular advancement devices / surgery, hypoglossal nerve stimulation for insufficient muscle activity during sleep, and nasal congestion treatments such as nasal sprays.

[0093] In this example, the machine learning model outputs a sleep disorder likelihood score and a treatment effectiveness score. The machine learning model is trained, and its internal weights are adjusted based on the training dataset. After evaluation against the training set, the model may be deployed after reaching a predetermined level of accuracy for both the sleep disorder likelihood score and the treatment effectiveness score. The machine learning model may then be deployed on the server 110 in Figure 1 to evaluate a first group of patients represented by patient 162.

[0094] In this example, the machine learning model is a neural network. The neural network may be a multilayer perceptron (MLP) neural network model that uses one or more hidden layers and has no direct connections between nodes. The neural network MLP model adjusts internally derived and computed weights between established node connections by minimizing an error function against actual values ​​during the training process. Other examples of machine learning models may include decision tree ensembles, support vector machines, Bayesian networks, or gradient boosting machines. Such structures are configured to implement linear or nonlinear predictive models.

[0095] Unsupervised machine learning can also be used to discover further correlations between physical characteristics and phenotypes and sleep disorders. Machine learning may employ techniques such as neural networks, clustering, or traditional regression methods. Training data can be used to test various types of machine learning algorithms and determine which are most accurate in predicting sleep disorders or treatment effectiveness.

[0096] The machine learning model for determining sleep disorder scores and treatment effectiveness scores can be continuously updated with new input data from the system shown in Figure 1. Therefore, the model can become more accurate as its use by system 100 increases.

[0097] Figure 5 is a flowchart of the sleep disorder analysis process that can be performed by the server 110 in Figure 1. This process involves collecting images of the patient's face and neck. In this example, the patient's face and neck are scanned via a depth camera on a mobile device such as a smartphone 134 or tablet 136 in Figure 1 to generate a composite 3D image of the face and neck (500). Alternatively, the 3D face scan data may be obtained from a storage device containing already scanned 3D face images of the patient, or from recorded video. Landmark points are determined within the face and neck mesh from the 3D scan (502). A set of points representing key dimensions and features relevant to the sleep disorder analysis is measured from the images (504).

[0098] Next, the images are analyzed to classify phenotypes associated with sleep disorders (506). Physiological data are collected from sensors such as sensor 150 in Figure 1 (508). Then, feature dimensions, physiological data, and classified phenotypes are correlated with sleep disorders (510). Next, a sleep disorder score is determined and output from the input data (512). The machine learning model may also determine treatment based on the input data and the sleep disorder score (514).

[0099] The flowchart in Figure 5 represents exemplary machine-readable instructions for collecting and analyzing data to assess the potential for sleep disorders in a patient. In this example, the machine-readable instructions include an algorithm executed by (a) a processor, (b) a controller, and / or (c) one or more other suitable processing devices. This algorithm may be embodied in software stored on a tangible medium such as flash memory, CD-ROM, floppy disk, hard drive, digital video (multipurpose) disk (DVD), or other storage device. However, those skilled in the art will readily understand that the entire algorithm and / or parts thereof may alternatively be executed by devices other than processors and / or embodied in firmware or dedicated hardware in well-known ways (for example, the entire algorithm and / or parts thereof may be implemented by application-specific integrated circuits (ASICs), programmable logic devices (PLDs), field-programmable logic devices (FPLDs), field-programmable gate arrays (FPGAs), discrete logic, etc.). For example, any or all of the interface components may be implemented by software, hardware, and / or firmware. Furthermore, some or all of the machine-readable instructions represented by the flowchart can be implemented manually. Moreover, although the exemplary algorithm has been described with reference to the flowchart shown in Figure 5, those skilled in the art will readily understand that many other methods can be used as alternatives to implement the exemplary machine-readable instructions. For example, the execution order of the blocks may be changed, and / or some of the described blocks may be modified, deleted, or combined.

[0100] As used in this application, terms such as “location element,” “module,” and “system” generally refer to any of the following: computer-related entities, hardware (e.g., circuits), combinations of hardware and software, software, or entities related to an operating machine having one or more specific functions. For example, a location element may, but is not limited to, a process, processor, object, executable module, execution thread, program, and / or computer running on a processor (e.g., a digital signal processor). Exemplarily, both an application running on a controller and the controller itself may be location elements. One or more location elements may reside within a process and / or execution thread, and location elements may be localized to one computer and / or distributed across two or more computers. Furthermore, “device” may take the form of specially designed hardware, generalized hardware in which the hardware is specialized by the execution of software capable of performing a specific function, software stored on a computer-readable medium, or a combination thereof.

[0101] The terms used herein are for the sole purpose of describing specific embodiments and are not intended to limit the invention. Where used herein, the singular forms "a," "an," and "the" are intended to also include the plural forms unless otherwise explicitly indicated in the context. Furthermore, to the extent that the terms "including," "includes," "having," "has," "with," or variations thereof are used in the detailed description and / or claims, such terms are inclusive in a similar manner to the term "comprising."

[0102] Unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as those generally understood by those skilled in the art. Furthermore, terms as defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art, and not as an idealized or overly formal meaning unless explicitly defined.

[0103] One or more further implementations and / or claims of the present disclosure can be formed by combining one or more elements, aspects, steps, or parts thereof from any one or more of the following claims 1 to 32 with one or more elements, aspects, steps, or parts thereof from any one or more of the other claims 1 to 32 or any combination thereof.

[0104] While various embodiments of the present invention have been described above, it should be understood that these are illustrative and not limiting. Although the present invention is illustrated and described in relation to one or more implementations, those skilled in the art will be able to conceive of equivalent changes and modifications by reading and understanding this specification and the accompanying drawings. Certain features of the present invention may be disclosed in relation to only one of several implementations, but such features may be combined with one or more other features in other implementations that are desirable and advantageous for any given or particular application. Therefore, the breadth and scope of the present invention should not be limited by any of the embodiments described above. Rather, the scope of the present invention should be defined according to the claims and their equivalents.

Claims

1. A method for determining a patient's sleep disorder, The processor acquires a digital image, including the patient's face and neck, from a storage device. The processor measures the facial and neck features in the image by determining landmarks in the image. The processor classifies at least one phenotype on the image from phenotypes identified before being stored in the database, The processor correlates the measurements of the at least one phenotype and at least one feature with sleep disorders. A method comprising the processor determining a risk score for sleep disorder based on the correlation between the phenotype and the measured value of at least one feature.

2. The method according to claim 1, wherein the aforementioned image is provided by the camera of a mobile device.

3. The method according to any one of claims 1 to 2, further comprising the processor acquiring a plurality of images including the patient's face and neck from the storage device.

4. The method according to any one of claims 1 to 3, further comprising the processor measuring the physiological measurements of the patient, wherein the risk score for the sleep disorder is determined on part basis of the physiological measurements.

5. The method according to any one of claims 1 to 4, wherein the correlation is performed using a machine learning model trained with multiple images from a patient population and the respective sleep disorder scores of the patient population.

6. The processor stores the image, the classified phenotype, the dimensions of the features, and the sleep disorder score in the memory device, The method according to claim 5, further comprising the processor updating a database of the patient population using the stored classified phenotypes, the dimensions of the features, and the patient's sleep disorder score.

7. The method according to any one of claims 1 to 6, wherein the sleep disorder is one of the following forms of sleep-disordered breathing (SDB): obstructive sleep apnea (OSA), Cheyne-Stokes respiration (CSR), obesity hyperventilation syndrome (OHS), and chronic obstructive pulmonary disease (COPD).

8. The method according to any one of claims 1 to 7, further comprising the processor determining a risk score for comorbidities based on the at least one phenotype.

9. The processor acquires video of the patient from the camera, The method according to claim 2, comprising the processor determining a dynamic movement of one of the features from the video, wherein the risk score for the sleep disorder is determined by the dynamic movement.

10. The method according to any one of claims 1 to 9, wherein the phenotype is encoded by the colors on the image, and the image and color code are displayed on a display.

11. The method according to claim 10, wherein the color code of the phenotype represents the degree of correlation with the sleep disorder.

12. The method according to any one of claims 1 to 11, wherein the at least one phenotype is selected from the group consisting of obesity / neck size, jutting jaw / mandible, and crowded / narrow upper airway.

13. The method according to any one of claims 1 to 12, further comprising the processor adapting the treatment of the sleep disorder based on the determined phenotype.

14. The processor determines the severity of the sleep disorder based on the determined sleep disorder score, The method according to any one of claims 1 to 13, further comprising the processor determining a therapy based on the severity of the sleep disorder.

15. The method according to any one of claims 1 to 14, wherein the aforementioned feature is the dimension of the neck, the dimension of the neck correlates with tissue mass and stiffness parameters, and the correlation of sleep disorders is related to the tissue mass and stiffness parameters.

16. A computer program product that, when executed by a computer, includes instructions causing the computer to perform the method according to any one of claims 1 to 15.

17. The computer program product according to claim 16, wherein the computer program product is a non-temporary computer-readable medium.

18. A system for determining a patient's sleep disorder, A memory device that stores a digital image including the face and neck of the aforementioned patient, A database that stores previously identified phenotypes and dimensions of facial and neck features, The sleep disorder analysis engine is coupled to the storage device and the database, and the sleep disorder analysis engine is The facial and neck features in the aforementioned image are identified by determining landmarks in the aforementioned image. Based on comparison with the aforementioned database, at least one phenotype on the image is classified, The aforementioned at least one phenotype and at least one feature are correlated with sleep disorders. A system capable of determining a risk score for the sleep disorder based on the correlation between the phenotype and the features.

19. The system according to claim 18, wherein the digital image is provided by the camera of a mobile device.

20. The system according to any one of claims 18 to 19, wherein the storage device stores a plurality of images including the patient's face and neck.

21. The system according to any one of claims 18 to 20, further comprising a sensor interface coupled to the sleep disorder analysis engine and a sensor for measuring the patient's physiological measurements, wherein the risk score for the sleep disorder is determined in part on the physiological measurements.

22. The system according to any one of claims 18 to 21, wherein the correlation is performed using a machine learning model trained with multiple images from a patient population and the respective sleep disorder scores of the patient population.

23. The aforementioned sleep disorder analysis engine further, The image, the classified phenotype, the dimensions of the features, and the sleep disorder score are stored. The system according to any one of claims 18 to 22, operable to update the database using the stored classified phenotypes, the dimensions of the features, and the patient's sleep disorder score.

24. The system according to any one of claims 18 to 23, wherein the sleep disorder is one of the forms of sleep-disordered breathing (SDB), which are obstructive sleep apnea (OSA), Cheyne-Stokes respiration (CSR), obesity hyperventilation syndrome (OHS), and chronic obstructive pulmonary disease (COPD).

25. The system according to any one of claims 18 to 24, wherein the sleep disorder analysis engine is operable to determine a risk score for comorbidities based on the at least one phenotype.

26. The system according to any one of claims 18 to 25, wherein the storage device includes a video of the patient, the sleep disorder analysis engine is operable to determine a dynamic movement of one of the features from the video, and the risk score of the sleep disorder is determined by the dynamic movement.

27. The system according to any one of claims 18 to 26, wherein the phenotype is encoded by the colors on the image, and the image and color codes are displayed on a display.

28. The system according to claim 27, wherein the color code of the phenotype represents the degree of correlation with the sleep disorder.

29. The method according to any one of claims 18 to 28, wherein the at least one phenotype is selected from the group consisting of obesity / neck size, jutting jaw / mandible, and crowded / narrow upper airway.

30. The system according to any one of claims 18 to 29, wherein the sleep disorder analysis engine is further operable to adapt the treatment of the sleep disorder based on the determined phenotype.

31. The system according to any one of claims 18 to 30, wherein the sleep disorder analysis engine is further operable to determine the severity of the sleep disorder based on the determined sleep disorder score and to determine a therapy based on the severity of the sleep disorder.

32. The system according to any one of claims 18 to 31, wherein the aforementioned feature is the dimensions of the neck, the dimensions of the neck correlate with tissue mass and stiffness parameters, and the correlation of sleep disorders is related to the tissue mass and stiffness parameters.

Images