Systems and methods for continually adjusting personalized mask shape - Patents.com

Abstract

A system and method for fitting an interface to a user's face for respiratory therapy. A facial image of a user is stored. Facial features are determined based on the facial image. A database stores a facial contour based on the facial features and a plurality of corresponding interfaces. The database stores operational data of a respiratory therapy device having a plurality of corresponding interfaces. A selection engine is coupled to the database. The selection engine operates to select an interface for the user from among the plurality of corresponding interfaces based on a desired outcome based on the stored operational data and the facial features. The collected data may also be used to determine whether the selected interface is an appropriate fit to the user's face.

Description

[Technical field]

[0001] [CROSS REFERENCE TO RELATED APPLICATIONS] This disclosure claims priority to and the benefit of U.S. Provisional Patent Application No. 63 / 154,223, filed February 26, 2021, and U.S. Provisional Patent Application No. 63 / 168,635, filed March 31, 2021, the contents of which are incorporated herein by reference in their entireties.

[0002] The present disclosure relates generally to interfaces for respiratory therapy devices and, more particularly, to a system for improved mask selection based on individual user data. [Background technology]

[0003] There are various respiratory diseases. Certain diseases may be characterized by specific episodes, such as apnea, hypopnea, and hyperpnea. Obstructive sleep apnea (OSA) is a form of sleep-disordered breathing (SDB) and is characterized by episodes such as closure or obstruction of the upper airway during sleep. It occurs when an abnormally narrow upper airway is combined with a decrease in normal muscle tone in the area of ​​the tongue, soft palate, and posterior oropharyngeal wall during sleep. This situation forces affected patients to stop breathing, usually for 30 to 120 seconds, and in some cases 200 to 300 times each night. It often causes excessive daytime sleepiness, which can lead to cardiovascular disease and brain damage. The syndrome is a common disorder, especially in middle-aged overweight men, although the affected person may not be aware of the problem.

[0004] Other sleep-related disorders include Cheyne-Stokes respiration (CSR), obesity hyperventilation syndrome (OHS), and chronic obstructive pulmonary disease (COPD). COPD encompasses any of a group of lower airway diseases that share certain common characteristics. These symptoms include increased resistance to airflow, a prolonged expiratory phase for breathing, and loss of normal elasticity of the lungs. Examples of COPD include emphysema and chronic bronchitis. COPD is caused by chronic smoking (the main risk factor), occupational exposure, air pollution, and genetic factors.

[0005] Continuous positive airway pressure (CPAP) therapy is used to treat obstructive sleep apnea (OSA). By pushing the soft palate and tongue forward or backward against the posterior oropharyngeal wall, application of CPAP therapy acts as a pneumatic splint, which may prevent closure of the upper airway.

[0006] Noninvasive ventilation (NIV) provides ventilatory support to a patient through the upper airway to assist the patient in breathing deeply and / or maintain adequate oxygen levels in the body by performing some or all of the respiratory functions. Ventilatory support is provided through a user interface. NIV is used to treat conditions such as CSR, OHS, COPD, and chest wall disorders. In some forms, it may improve the comfort and efficacy of these therapies. Invasive ventilation (IV) provides ventilatory support to patients who can no longer breathe effectively and may be provided using a tracheostomy tube.

[0007] The treatment system (also referred to herein as a respiratory therapy system) may include a respiratory pressure therapy device (RPT device), an air circuit, a humidifier, a user interface, and data management. The patient or user interface may be used to interface the respiratory device to its wearer, for example, by providing a flow of air to the entrance of the airway. The flow of air may be provided to the patient's nose and / or mouth via a mask, to the patient's mouth via a tube, or to the patient's trachea via a tracheotomy tube. Depending on the therapy being applied, the user interface may form a seal with, for example, an area of ​​the patient's face to facilitate delivery of gas at a pressure that varies sufficiently with ambient pressure to enable therapy, for example, about 10 cmH2O positive pressure relative to ambient pressure. For other forms of therapy, such as delivery of oxygen, the user interface may not include a seal sufficient to facilitate delivery of a gas supply to the airway at a positive pressure of about 10 cmH2O. Treatment of respiratory disorders with such therapies may be optional, and thus patients may choose not to comply with the therapy if they find the devices used to deliver such therapy uncomfortable, difficult to use, expensive, and / or unattractive.

[0008] There are many challenges in designing a user interface. The face has a complex three-dimensional shape. Nose size and shape vary greatly from person to person. The head contains bone, cartilage and soft tissue, so different regions of the face respond differently to mechanical forces. The jaw or mandible can move relative to other bones of the skull. The entire head can move during respiratory therapy.

[0009] As a result of these challenges, some masks are one or more of the following: unsightly, aesthetically undesirable, expensive, inappropriate, difficult to use, and uncomfortable, especially when worn for extended periods of time or when the user is unfamiliar with the system. For example, masks designed specifically for aviators, masks designed as part of personal protective equipment (e.g., filter masks), scuba masks, or masks for administering anesthetic drugs may be acceptable for their intended use, but such masks may nevertheless cause undesirable discomfort to the wearer when worn for extended periods of time (e.g., several hours). This discomfort may lead to reduced user compliance with therapy. This is especially true if the mask is required to be worn while sleeping.

[0010] CPAP therapy is highly effective in treating certain respiratory conditions as long as the user adheres to the therapy. The user interface allows the user to perform positive pressure therapy. It is common for the user to request a new user interface to replace the user's first or previous user interface to determine the recommended user interface size based on measurements of the user's facial anatomy, which are typically performed by the durable medical equipment provider. If the mask is uncomfortable or difficult to use, the user may not adhere to the therapy. Since it is often recommended to clean the mask regularly, if the mask is difficult to clean (e.g., difficult to assemble or disassemble), the mask may not be cleaned, which may affect user compliance. For air pressure therapy to be effective, the mask must not only provide the user with comfort when worn, but also form a tight seal between the face and the mask to minimize air leakage.

[0011] As mentioned above, the user interface can be provided to the user in various forms, such as a nasal mask, a full face mask / oral nasal mask (FFM), or a nasal pillow mask. Such user interfaces are manufactured in various sizes to fit the anatomical features of a particular user, for example, to facilitate having a comfortable interface that provides positive pressure therapy functionality. The size of the user interface can be customized to accommodate the particular facial anatomy of a particular user, or can be designed to fit a population of individuals whose anatomies fall within a predefined spatial boundary or range. However, in some cases, the masks are available in a variety of standard sizes from which the appropriate size must be selected.

[0012] In this regard, determining the size of a user interface for a user is typically performed by a trained individual, such as a durable medical equipment (DME) provider or physician. Typically, a user who requires a user interface to initiate or continue positive pressure therapy visits a trained individual at a receiving facility, who then takes a series of measurements in an effort to determine the appropriate user interface size from standard sizes. The appropriate size is intended to represent a particular combination of size of certain features, such as the seal-forming structure of the user interface, that provides sufficient comfort and seal to allow positive pressure therapy. Determining a size in this manner is not only a daunting task, but also an inconvenience. The inconvenience of having to take time out of a busy schedule and possibly travel significant distances is a barrier for many users to utilize new or alternative user interfaces and, ultimately, to receiving treatment. This inconvenience can prevent a user from receiving the desired user interface and being able to administer positive pressure therapy. Nonetheless, selecting the optimal size is important for quality of treatment and compliance.

[0013] A system is needed that allows for accurate individualization of a user interface based on selected facial dimension data. A system is needed that combines data about other similar users who use respiratory therapy devices to further select an interface that provides comfort in respiratory therapy use. There is also a need to select an interface that minimizes leakage between the interface and the user's face. Summary of the Invention

[0014] The disclosed system provides an adaptable system for determining the size of a mask to be used with a respiratory therapy device to improve therapy compliance for an individual user. The system collects facial data from a primary user and respiratory therapy device usage and other data from a larger population of users to assist in selecting the optimal mask for the primary user.

[0015] A disclosed example is a system for selecting an interface for respiratory therapy appropriate for a user's face. The system includes a storage device that stores a facial image of the user. A facial contour engine is operable to determine facial features based on the facial image. One or more databases store a plurality of facial features from a user population and a plurality of corresponding interfaces. The one or more databases store operational data of respiratory therapy devices used by the user population having a plurality of corresponding interfaces. A selection engine is coupled to the databases. The selection engine is operable to select an interface for the user from among the plurality of corresponding interfaces based on a desired outcome from the stored operational data and the determined facial features.

[0016] Another disclosed example is a method for selecting an appropriate interface for respiratory therapy for a user's face. A facial image of the user is stored in a storage device. Facial dimensions are determined based on landmarks. A plurality of facial features are stored from a user population, and a plurality of corresponding interfaces used by the patient population are stored in one or more databases. Operational data of respiratory therapy devices used by the user population having the plurality of corresponding interfaces is stored in the one or more databases. An interface for the user is selected from among the plurality of corresponding interfaces based on a desired outcome from the stored operational data and the determined facial features.

[0017] According to some embodiments, an exemplary method includes receiving sensor data related to a current fit of an interface to a user's face. The interface may be coupled to a breathing device. The sensor data is collected by one or more sensors of a mobile device separate from the breathing device. The method further includes generating a facial mapping using the sensor data. The facial mapping is indicative of one or more features of the user's face. The method further includes identifying a characteristic related to the current fit using the sensor data and the facial mapping. The characteristic is indicative of a quality of the current fit. The characteristic is associated with a characteristic location on the facial mapping. The method further includes generating output feedback based on the identified characteristic and the characteristic location. The output feedback is generated to evaluate or improve the current fit.

[0018] According to some embodiments, an exemplary system includes an electronic interface, a memory, and a control system. The electronic interface is configured to receive data related to a current fit of the interface. The memory stores machine-readable instructions. The control system includes one or more processors configured to execute the machine-readable instructions to generate a facial mapping using the received data and identify characteristics related to the current fit based on the received data and the facial mapping. The control system is further configured to generate output feedback based on the identified characteristics. The output feedback is generated to evaluate or improve the current fit.

[0019] The above summary is not intended to represent each embodiment or each aspect of the present disclosure. Rather, the foregoing summary merely provides examples of some of the novel aspects and features described herein. The above features and advantages of the present disclosure will become apparent from the following detailed description of exemplary embodiments and implementations of the present disclosure, when taken in conjunction with the accompanying drawings and claims, and other features and advantages of the present disclosure will become apparent.

[0020] The present disclosure will be better understood from the following description of exemplary embodiments with reference to the accompanying drawings. [Brief description of the drawings]

[0021] [Figure 1] 1 illustrates a system including a user wearing a user interface in the form of a full face mask to receive PAP therapy from an exemplary respiratory pressure therapy device. [Diagram 2] 1 shows a user interface in the form of a nasal mask with headgear in accordance with one form of the present technology. [Figure 3A] FIG. 1 is a front view of a face with some surface anatomical features. [Figure 3B] FIG. 1 is a side view of the head with some surface anatomical features identified. [Figure 3C]A basic drawing of a nose with some features identified. [Figure 4A] According to one aspect of the present technology, <2> 1 shows a respiratory pressure therapy device. [Figure 4B] FIG. 13 is a schematic diagram of air pressure paths of a respiratory pressure therapy device in accordance with one form of the present technology. [Figure 4C] FIG. 13 is a schematic diagram of electrical components of a respiratory pressure therapy device in accordance with one form of the present technology. [Figure 4D] FIG. 1 is a schematic diagram of the major data processing components of a respiratory pressure therapy system in accordance with one form of the present technology. [Diagram 5] FIG. 3 is a block diagram showing the placement of an acoustic sensor within a hose supplying air to the user interface of FIG. 2. [Figure 6] FIG. 2 is a diagram of parts of a computing device used to capture facial data. [Figure 7] FIG. 1 illustrates an example system for automatically selecting a patient interface that includes a computing device. [Figure 8A] 1 is an exemplary face scan showing different landmark points for identifying face dimensions for mask sizing. [Figure 8B] FIG. 8B is a diagram of the face scan of FIG. 8A showing different landmark points for identifying the first facial measurement. [Figure 8C] FIG. 8B is a diagram of the face scan of FIG. 8A showing different landmark points for identifying a second facial measurement. [Figure 8D] FIG. 8B is a diagram of the face scan of FIG. 8A showing different landmark points for identifying a third facial measurement. [Figure 9] 1 is a flow diagram illustrating a process for selecting a mask for a user based on scanning and analyzing user input taking into account big data collected from a user database. [Figure 10] 13 is a flow diagram of a process of follow-up evaluation from an initial selection of a mask to adjust correlation parameters based on data regarding a primary user. [Figure 11]FIG. 1 is a perspective view of a user controlling a user device to collect sensor data related to the current fitness of a user interface. [Figure 12] 11 is a user view of a user device for identifying thermal characteristics associated with a current fit of a user interface. [Figure 13] A user view of a user device for identifying contour-based characteristics associated with a current fitness of a user interface. [Figure 14] 1 is a flow chart illustrating a process for evaluating the compatibility of a user interface between user interface transition events. [Figure 15] 1 is a flow chart illustrating a process for evaluating the suitability of a user interface. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0022] The present disclosure is susceptible to various modifications and alternative forms. Several representative embodiments are shown by way of example in the drawings and will be described in detail herein. It should be understood, however, that the invention is not intended to be limited to the particular forms disclosed. Rather, the present disclosure is intended to cover all modifications, equivalents, and alternatives falling within the spirit and scope of the invention as defined in the appended claims.

[0023] The present invention can be embodied in many different forms. Representative embodiments are shown in the drawings and described in detail herein. The present disclosure is an example or illustration of the principles of the present disclosure, and is not intended to limit the broad aspects of the present disclosure to the embodiments shown. To that extent, elements and limitations disclosed, for example, in the Abstract, Summary, and Detailed Description, but not expressly recited in the Claims, should not be incorporated into the Claims, either singly or collectively, by implication, inference, or otherwise. For purposes of the Detailed Description, unless otherwise specified, the singular includes the plural, the plural includes the singular, and the term "including" means "including but not limited to." Additionally, approximation words such as "about," "almost," "substantially," "approximately," and the like may be used herein to mean "at," "near," or "nearly at," "within 3-5%," "within acceptable manufacturing tolerances," or logical combinations thereof.

[0024] The present invention relates to a system and method for selecting an optimal interface for a user using a respiratory therapy device. The system collects individual facial feature data of a user from a face scanning application on a mobile device and generates a 3D model of the face from the scanned image. The system accurately locates key landmarks on the scanned image to determine various facial dimensions. The system collects data from multiple users and learns successful correlations between various masks and respiratory treatments based on the user data and motion data from the respiratory therapy device using various masks. The system uses the facial dimension data and the data collected from the multiple users to select an appropriate mask size and type for the user, comparing the success of the match with past data collected from multiple users using the interface and the respiratory therapy device. This data can also be used to further learn new insights that can be provided to the user to improve compliance / adherence to the therapy. Thus, the method combines an appropriate face scanning method with landmark identification, mask selection, therapy monitoring, and feedback to provide an effective user interface selection.

[0025] Certain aspects and features of the present disclosure relate to assessing and improving the fit of a user interface (e.g., a filter mask or a respiratory therapy device user interface). Sensor data from one or more sensors of a user device (e.g., a portable user device, also referred to as a mobile phone, mobile computing device or mobile device) may be utilized to assist a user in ensuring that a wearable user interface (e.g., a user interface used with a respiratory device) fits the user's face properly. The sensors may collect data about the user's face i) before the user interface is worn, ii) while the user interface is worn, and / or iii) after the user interface is removed. A facial mapping may be generated to identify one or more features of the user's face. The sensor data and facial mapping may then be utilized to identify characteristics associated with the fit of the user interface, which may be used to generate output feedback to assess and / or improve the fit.

[0026] In other embodiments of the disclosed exemplary system, the interface is a mask. In another embodiment, the respiratory therapy device is configured to provide one or more of positive airway pressure (PAP) or non-invasive ventilation (NIV). In another embodiment, the at least one respiratory therapy device includes an audio sensor for collecting audio data during operation of the at least one respiratory therapy device. In another embodiment, the selection engine is operable to analyze the audio data to determine a type of corresponding interface of the at least one respiratory therapy device based on matching the audio data with an acoustic signature of a known interface. In another embodiment, the selection engine selects an interface based on demographic data of the user compared to demographic data of a user population stored in one or more databases. In another embodiment, the operational data includes one of a flow rate, a motor speed, and a treatment pressure. In another embodiment, the facial image is determined based on a scan from a mobile device including a camera. In another embodiment, the mobile device includes a depth sensor. The camera is a 3D camera. The facial features are three-dimensional features derived from a mesh surface derived from the facial image. In another embodiment, the facial image is a two-dimensional image including landmarks, and the facial features are three-dimensional features derived from the landmarks. In another embodiment, the facial image is one of a plurality of two-dimensional facial images. The facial features are three-dimensional features derived from a 3D deformable model adapted to match the facial image. In another embodiment, the facial image includes landmarks for at least one facial dimension. In another embodiment, the facial dimensions include face height, nose width, and nose depth. In another embodiment, the desired outcome is a seal between the interface and the face to prevent leakage. In another embodiment, the desired outcome is user compliance with the therapy. In another embodiment, the system includes a mobile computing device operable to collect subjective data input from a user, and the selection of the interface is based at least in part on the subjective data.In another embodiment, the system includes a machine learning module operable to determine a type of behavioral data that correlates to an interface that achieves a desired outcome. In another embodiment, the selected interface includes one of a plurality of types of interfaces and one of a plurality of sizes of interfaces. In another embodiment, the selection engine receives feedback from the user based on the behavior of the selected interface and based on undesirable outcomes and selects another interface based on the desired outcome. In another embodiment, the undesirable outcome is one of low user compliance with the therapy, high leakage, and unsatisfactory subjective outcome data.

[0027] In another embodiment of the above disclosed method of providing a suitable fitting interface, the interface is a mask. In another embodiment, the respiratory therapy device is configured to provide one or more of positive airway pressure (PAP) or non-invasive ventilation (NIV). In another embodiment, the at least one respiratory therapy device includes an audio sensor for collecting audio data during operation of the at least one respiratory therapy device. In another embodiment, the selection includes analyzing the audio data to determine a corresponding interface type of the at least one respiratory therapy device based on matching the audio data with an acoustic signature of a known interface. In another embodiment, the selection includes comparing demographic data of the user to demographic data of a user population stored in one or more databases. In another embodiment, the operational data includes one of a flow rate, a motor speed, and a treatment pressure. In another embodiment, the facial image is determined based on a scan from a mobile device including a camera. In another embodiment, the mobile device includes a depth sensor. The camera is a 3D camera, and the facial features are three-dimensional features derived from a mesh surface derived from the facial image. In another embodiment, the facial image is a two-dimensional image including landmarks, and the facial features are three-dimensional features derived from the landmarks. In another embodiment, the facial image is one of a plurality of two-dimensional facial images. The facial features are three-dimensional features derived from a 3D deformable model adapted to match the facial image. In another embodiment, the facial image includes landmarks for at least one facial dimension. In another embodiment, the facial dimensions include face height, nose width, and nose depth. In another embodiment, the desired outcome is a seal between the interface and the face to prevent leakage. In another embodiment, the desired outcome is user compliance with the therapy. In another embodiment, the method includes collecting subjective data input from the user by at least one mobile computing device, and the selection of the interface is based at least in part on the subjective data.In another embodiment, the method includes determining a type of behavioral data that correlates to an interface that achieves a desired outcome via a machine learning module. In another embodiment, the selected interface includes one of a plurality of types of interfaces and one of a plurality of sizes of interfaces. In another embodiment, the method includes receiving feedback from a user based on behavior of the selected interface and selecting another interface based on a desired outcome based on an undesirable outcome. In another embodiment, the undesirable outcome is one of a user having low compliance with the therapy, high leakage, or unsatisfactory subjective outcome data.

[0028] In other embodiments of the method of providing a user with a custom fit on an interface disclosed above, the interface may be fluidly coupled to a breathing device. In another embodiment, generating the output feedback includes determining a proposed action that, if performed, affects the characteristic to improve the current fit. The proposed action may also be presented using an electronic interface of the mobile device. In another embodiment, the sensor data includes infrared data from i) a passive thermal sensor, ii) an active thermal sensor, or iii) both i and ii. In another embodiment, the method further includes receiving sensor data related to a current fit of the selected interface on the user's face. The sensor data is collected by one or more sensors of the mobile device. The method further includes generating a facial mapping using the sensor data. The facial mapping is indicative of one or more features of the user's face. The method further includes identifying a characteristic related to the current fit using the sensor data and the facial mapping. The characteristic is indicative of a quality of the current fit, and the characteristic is associated with a characteristic location on the facial mapping. The method further includes generating the output feedback based on the identified characteristic and the characteristic location. In another embodiment, the sensor data includes distance data indicative of one or more distances between one or more sensors of the mobile device and the user's face. In another embodiment, the one or more sensors include i) a proximity sensor, ii) an infrared-based dot matrix sensor, iii) a LiDAR sensor, iv) a MEMS micromirror projector-based sensor, or v) any combination of i-iv. In another embodiment, the sensor data is collected i) before the user wears the interface, ii) while the user is wearing the interface with a current fit, iii) after the user removes the interface, or iii) any combination of i, ii, and iii. In another embodiment, the method further includes receiving initial sensor data. The initial sensor data is associated with the user's face before the interface is worn.Identifying the characteristic includes comparing the initial sensor data to the sensor data. In another embodiment, the characteristic includes i) a local temperature of the user's face, ii) a local temperature change of the user's face, iii) a local color of the user's face, iv) a local color change of the user's face, v) a local contour of the user's face, vi) a local contour change of the user's face, vii) a local contour of the interface, viii) a local change of the interface, ix) a local temperature of the interface, or x) any combination of i-ix. In another embodiment, the characteristic includes i) a vertical position of the interface relative to one or more features of the user's face, ii) a horizontal position of the interface relative to one or more features of the user's face, iii) a rotational orientation of the interface relative to one or more features of the user's face, iv) a distance between a distinguishing feature of the interface relative to one or more features of the user's face, or v) any combination of i-iv. In another embodiment, the one or more sensors include one or more directional sensors, and the sensor data includes directional sensor data from the one or more directional sensors. Receiving the sensor data includes scanning the user's face using the mobile device when the mobile device is oriented such that one or more directional sensors are pointed at the user's face, and tracking progress of the facial scan. Generating an invisible stimulus indicative of progress of the facial scan. In another embodiment, the method further includes receiving motion data related to movement of the mobile device, and applying the motion data to the sensor data to account for movement of the mobile device relative to the user's face. In another embodiment, the method further includes generating an interface mapping using the sensor data. The interface mapping indicates a relative position of one or more features of the interface relative to the facial mapping. Identifying the characteristic includes using the interface mapping. In another embodiment, the one or more sensors include a camera, and the sensor data includes camera data.The method further includes receiving sensor calibration data collected by the camera when the camera is pointed at the calibration surface, and calibrating the camera data of the sensor data based on the sensor calibration data. In another embodiment, the method further includes using the sensor data and the facial mapping to identify additional characteristics associated with a potential future failure of the interface. The method further includes generating output feedback based on the identified additional characteristics. The output feedback may be used to reduce the likelihood of a potential future failure occurring or to delay the occurrence of a potential future failure. In another embodiment, the method further includes accessing historical sensor data associated with one or more historical fits of the interface on the user's face prior to receiving the sensor data. The identifying the characteristics further includes using the historical sensor data. In another embodiment, the method further includes using the sensor data and the facial mapping to generate a current fit score. The output feedback is generated to improve a subsequent fit score. In another embodiment, the receiving the sensor data includes receiving audio data from one or more audio sensors. The identifying the characteristics includes using the audio data to identify an unintentional leak. In another embodiment, the one or more sensors include a camera, an audio sensor, and a thermal sensor. The sensor data includes camera data, audio data, and thermal data. Identifying the characteristic includes identifying the potential characteristic using at least one of the camera data, audio data, and thermal data, and confirming the potential characteristic using other data of at least one of the camera data, audio data, and thermal data. In another embodiment, the method further includes presenting a user command indicating an action to be performed by the user. The method further includes receiving a completion signal indicating that the user has performed the action. A first portion of the sensor data is collected prior to receiving the completion signal, and a second portion of the sensor data is collected after receiving the completion signal.Identifying the characteristic includes comparing a first portion of the sensor data to a second portion of the sensor data. In another embodiment, generating the facial mapping includes identifying a first individual and a second individual using the received sensor data, and using the received sensor data to identify the first individual as associated with the interface, and generating a facial mapping for the first individual. In another embodiment, receiving the sensor data includes determining adjustment data from the received sensor data. The adjustment data is related to i) a movement of the mobile device, ii) a noise inherent to the mobile device, iii) a breathing noise of the user, iv) a speech noise of the user, v) a change in ambient lighting, vi) a temporary shadow projected on the user's face, vii) a temporary colored light projected on the detected user's face, or viii) any combination of i-vii. Receiving the sensor data also includes applying an adjustment to at least a portion of the received sensor data based on the adjustment data. In another embodiment, receiving the sensor data includes receiving image data associated with a camera of one or more sensors operating in the visible light spectrum, receiving unstable data associated with an additional sensor of the one or more sensors that is an image sensor or a ranging sensor operating outside the visible light spectrum, determining image stabilization information associated with stabilization of the image data, and stabilizing the unstable data using the image stabilization information associated with stabilization of the image data. In another embodiment, the output feedback can be used to improve the current fit. In another embodiment, the method further includes using the sensor data to generate an initial score based on the current fit. The method further includes receiving subsequent sensor data associated with a subsequent fit of the interface to the user's face. The subsequent fit is based on the current fit after implementation of the output feedback. The method includes using the subsequent sensor data to generate a subsequent score based on the subsequent fit, and evaluating the subsequent score that indicates an improvement in quality over the initial score.In another embodiment, identifying a characteristic associated with the current fit includes determining a breathing pattern of the user based on the received sensor data, determining a thermal pattern associated with the user's face based on the received sensor data and the facial mapping, and using the breathing pattern and the thermal pattern to determine a leak characteristic. The leak characteristic indicates a balance between intentional vent leaks and unintentional seal leaks. In another embodiment, the one or more sensors include at least two sensors selected from the group consisting of: i) a passive thermal sensor, ii) an active thermal sensor, iii) a camera, iv) an accelerometer, v) a gyroscope, vi) an electronic compass, vii) a magnetometer, viii) a pressure sensor, ix) a microphone, x) a temperature sensor, xi) a proximity sensor, xii) an infrared-based dot matrix sensor, xiii) a LiDAR sensor, xiv) a MEMS micromirror projector-based sensor, xv) a radio frequency-based ranging sensor, and xvi) a wireless network interface. In another embodiment, the method includes receiving additional sensor data from one or more additional sensors of the respiratory device. Identifying the characteristic includes using the additional sensor data. In another embodiment, the method further includes transmitting a control signal that, when received by the respiratory device, causes the respiratory device to operate using the defined set of parameters. A first portion of the sensor data is collected while the respiratory device is operating using the defined set of parameters and a second portion of the sensor data is collected while the respiratory device is not operating using the defined set of parameters. Identifying the characteristic includes comparing the first portion of the sensor data to the second portion of the sensor data.

[0029] 1 shows a system including a user 10 wearing a user interface 100 in the form of a full face mask (FFM) receiving a supply of positive air pressure from a respiratory therapy device such as a positive airway pressure (PAP) device, and in particular a respiratory pressure therapy (RPT) device 40. Air from the RPT device 40 is humidified in a humidifier 60 and delivered to the user 10 through an air circuit 50.

[0030] In this example, the respiratory therapy device described herein may include any respiratory therapy device configured to provide one or more of positive airway pressure ventilation (PAP), non-invasive ventilation (NIV), or invasive ventilation. In this example, the PAP device may be a continuous positive airway pressure (CPAP) device, an automatic positive airway pressure device (APAP), a bi-level or variable positive airway pressure device (BPAP or VPAP), or any combination thereof. The CPAP device delivers a predetermined air pressure (e.g., determined by a sleep physician) to the user. The APAP device automatically varies the air pressure delivered to the user, for example, based on breathing data associated with the user. The BPAP or VPAP device is configured to deliver a first predetermined pressure (e.g., inspiratory positive airway pressure or IPAP) and a second predetermined pressure (e.g., expiratory positive airway pressure or EPAP) that is lower than the first predetermined pressure.

[0031] 2 shows a user interface 100 according to one aspect of the present technology, including a seal-forming structure 160, a plenum chamber 120, a positioning and stabilizing structure 130, a vent 140, a forehead support 150, and a form of connection port 170 for connecting to the air circuit 50 of FIG. 1. In some forms, the functional aspects may be provided by one or more physical components. In some forms, one physical component may provide one or more functional aspects. In use, the seal-forming structure 160 is positioned to surround the entrance of a user's airway and facilitates supplying air to the airway at positive pressure.

[0032] In one form of the present technology, the seal-forming structure 160 provides a seal-forming surface and may also provide a cushioning function. The seal-forming structure 160 of the present technology may be constructed of a soft, flexible, resilient material such as silicone. In one form, the seal-forming portion of the non-invasive user interface 100 includes a pair of nasal puffs or nasal pillows constructed and arranged to form a seal with the user's corresponding nares.

[0033] The nasal pillows of the present technology include a frustum cone, at least a portion of which forms a seal under the user's nose, a stem, and a flexible region below the frustum cone and connecting the frustum to the stem. Additionally, the structure to which the nasal pillows of the present technology are connected includes a flexible region adjacent the base of the stem. The flexible regions can act together to facilitate a universal joint structure that accommodates relative movement (both displacement and angle) between the frustum cone and the structure to which the nasal pillows are connected. For example, the frustum cone can be displaced axially toward the structure to which the stem is connected.

[0034] In one form, the non-invasive user interface 100 includes a seal-forming portion that, in use, forms a seal at the upper lip region (i.e., upper lip) of the user's face. In one form, the non-invasive user interface 100 includes a seal-forming portion that, in use, forms a seal at the chin region of the user's face.

[0035] Preferably, the plenum chamber 120 has a periphery that is formed complementary to the surface contours of a normal human face in the area where a seal is to be formed in use. In use, the periphery of the plenum chamber 120 is positioned proximate to the adjacent surface of the face. Actual contact with the face is provided by the seal-forming structure 160. The seal-forming structure 160 may extend around the entire periphery of the plenum chamber 120 in use.

[0036] Preferably, the seal-forming structure 160 of the user interface 100 of the present technology may be held in a sealing position by the positioning and stabilizing structure 130 during use.

[0037] In one form, the user interface 100 includes a vent 140 constructed and arranged to allow the exit of exhaled carbon dioxide. The vent 140 in accordance with one form of the present technology includes a plurality of holes, for example, about 20 to about 80 holes, or about 40 to about 60 holes, or about 45 to about 55 holes.

[0038] FIG. 3A shows a front view of a human face including the inner corners, alae of the nose, nasolabial fold, upper and lower lips, upper and lower lips, and cheekbones. Also shown are the width of the mouth, the sagittal plane dividing the head into left and right, and a directional indicator. The directional indicators indicate the radial inward / outward and upward / downward directions. FIG. 3B shows a side view of a human face including the brow, alae of the nose, bridge of the nose, anterior nose, subnasal, upper and lower lips, upper lip, apex of the alae of the nose, and upper and lower nasal bones. Also shown are directional indicators indicating the up-down and anterior-posterior directions. FIG. 3C shows a bottom view of the nose with several features identified including the nasolabial fold, lower lip, upper vermilion, nostrils, subnasal, columella, anterior nares, the long axis of the nostrils, and the sagittal plane.

[0039] The following describes in more detail the facial features shown in Figures 3A-3C.

[0040] Alar: the outer wall or "wing" of each nostril (plural: alar).

[0041] Alar point: the outermost point of the ala of the nose.

[0042] Alar curvature (or alar apex) point: The last point on the base line of curvature of each alar, located in the crease formed where the alar joins the cheek.

[0043] Pinna: the entire externally visible part of the ear.

[0044] Columella: The strip of skin that separates the nostrils and extends from the anterior nostril to the upper lip.

[0045] Columellar angle: the angle between a line passing through the midpoint of the nostril opening and a line perpendicular to the Frankfurt horizontal that intersects the bottom of the nose.

[0046] Glabellar space: Located on the soft tissue, it is the most prominent point in the midsagittal plane of the forehead.

[0047] Nostrils: Roughly oval openings that form the entrance to the nasal cavity. The singular form of nostril is naris (nostril). The nostrils are separated by the nasal septum.

[0048] Nasolabial fold or nasolabial fold: A fold or groove of skin that runs from either side of the nose to the corners of the mouth and separates the cheek from the upper lip.

[0049] Nasolabial angle: the angle between the bridge of the nose and the upper lip, intersecting with the subnasal point.

[0050] Superior ear base: uppermost point of attachment of the auricle to the skin of the face.

[0051] The uppermost point of attachment of the auricle to the superior facial skin.

[0052] Nasal tip: the most prominent point or tip of the nose that can be identified in a lateral view of the rest of the head.

[0053] Philtrum: the midline groove from the lower edge of the nasal septum to the top of the lip in the upper lip area.

[0054] Pogonion: Located in the soft tissue, at the anterior-most midpoint of the jaw.

[0055] Ridge (nose): The nasal ridge is a midline protrusion of the nose, extending from the serion to the tip of the nose.

[0056] Sagittal Plane: A vertical plane running from anterior (anterior) to posterior (posterior) that divides the body into right and left halves.

[0057] Therion: The most concave point located on the soft tissue, covering the area of ​​the frontonasal suture.

[0058] Nasal septum cartilage (nose): The nasal septum cartilage forms part of the nasal septum, separating the front of the nasal cavity.

[0059] Alar sinus: the point on the inferior edge of the base of the nose where it joins the skin of the upper lip.

[0060] Subnasal point: Located on the soft tissue, the point where the columella joins the upper lip in the midsagittal plane.

[0061] Upper lip: The most concave point on the midline of the lower lip between the lower lip and the corner of the soft tissue.

[0062] As explained below, there are several important dimensions from the face that can be used to select the size of a user interface such as the mask 100 of FIG. 1. In this example, there are three dimensions: face height, nose width, and nose depth. FIGS. 3A-3B show a line 3010 that represents face height. As shown in FIG. 3A, face height is the distance from the serion to the supramenton. Line 3020 in FIG. 3A represents nose width, which can be the distance between the alae of the nose (e.g., the left and right most outer points of the alae of the nose). Line 3030 in FIG. 3B represents nose depth, which can be the distance between the anterior nasal apex and the alae apex in a direction parallel to the sagittal plane.

[0063] FIG. 4A shows an exploded view of the components of an exemplary RPT device 40, including mechanical, pneumatic, and / or electrical components, configured to execute one or more algorithms, such as all or part of the methods described herein, according to one embodiment of the technology. FIG. 4B shows a block diagram of an exemplary RPT device 40. FIG. 4C shows a block diagram of an exemplary electrical control component of the RPT device 40. The upstream and downstream directions are indicated with reference to the blower and the user interface. The blower is defined as upstream of the user interface, and the user interface is defined as downstream of the blower, regardless of the actual flow direction at a particular moment. An item located in the pneumatic path between the blower and the user interface is located downstream of the blower and upstream of the user interface. The RPT device 40 can be configured to generate a flow of air for delivery to a user's airway, for example, to treat one or more respiratory disorders.

[0064] The RPT device 40 may have an exterior housing 4010 formed in two portions, an upper portion 4012 and a lower portion 4014. Additionally, the exterior housing 4010 may include one or more panels 4015. The RPT device 40 includes a chassis 4016 that supports one or more internal components of the RPT device 40. The RPT device 40 may include a handle 4018.

[0065] The pneumatic path of the RPT device 40 may include one or more pneumatic path items such as an inlet air filter 4112, an inlet muffler 4122, a pressure generator 4140 (e.g., a blower 4142) capable of supplying air at positive pressure, an outlet muffler 4124, and one or more transducers 4270, such as a pressure sensor 4272, a flow sensor 4274, and a motor speed sensor 4276.

[0066] One or more pneumatic path items may be disposed within a removable unitary structure referred to as a pneumatic block 4020. The pneumatic block 4020 may be disposed within the external housing 4010. In one form, the pneumatic block 4020 is supported by or formed as part of the chassis 4016.

[0067] The RPT device 40 may have a power supply 4210, one or more input devices 4220, a central controller 4230, a pressure generator 4140, a data communication interface 4280, and one or more output devices 4290. A separate controller may be provided for the treatment devices. The electrical components 4200 may be implemented on a single printed circuit board assembly (PCBA) 4202. In the alternative, the RPT device 40 may include multiple PCBAs 4202. Other components such as one or more protection circuits 4250, transducers 4270, data communication interfaces 4280, and storage devices may also be implemented on the PCBA 4202.

[0068] The RPT device may include one or more of the following components in an integrated unit: In the alternative, one or more of the following components may each be disposed as a separate unit.

[0069] An RPT device according to one form of the present technology may include an air filter 4110 or multiple air filters 4110. In one form, an inlet air filter 4112 is located at the beginning of the pneumatic path upstream of the pressure generator 4140. In one form, an outlet air filter 4114, e.g., an antibacterial filter, is located between the outlet of the pneumatic block 4020 and the user interface 100.

[0070] An RPT device in accordance with one form of the present technology may include a muffler 4120 or multiple mufflers 4120. In one form of the present technology, an inlet muffler 4122 is disposed in the pneumatic path upstream of a pressure generator 4140. In one form of the present technology, an outlet muffler 4124 is disposed in the pneumatic path between the pressure generator 4140 and the user interface 100 of FIG.

[0071] In one form of the present technology, the pressure generator 4140 for generating a positive pressure air flow or supply is a controllable blower 4142. For example, the blower 4142 may include a brushless DC motor 4144 having one or more impellers. The impellers may be arranged in a volute. The blower can deliver the supply of air at speeds up to about 120 L / min, at positive pressures in the range of about 4 cmH2O to about 20 cmH2O, or in other forms up to about 30 cmH2O, for example. The blower is described in any one of the following patents or patent applications: U.S. Patent No. 7,866,944; U.S. Patent No. 8,636,479; U.S. Patent No. 8,636,479; PCT Patent Application Publication No. WO 2013 / 020167, the contents of which are incorporated herein by reference in their entireties.

[0072] The pressure generator 4140 is under the control of the therapy device controller 4240. In other forms, the pressure generator 4140 may be a piston-driven pump, a pressure regulator connected to a high pressure source (e.g., a compressed air reservoir), or a bellows.

[0073] An air circuit 4170 according to one aspect of the present technology is a conduit or tube constructed and arranged to allow a flow of pressurized air to travel, in use, between two components, such as the humidifier 60 and the user interface 100. Specifically, the air circuit 4170 may be in fluid communication with an outlet of the humidifier 60 and the plenum chamber 120 of the user interface 100.

[0074] In one form of the present technology, the anti-spill valve 4160 is located between the humidifier 60 and the pneumatic block 4020. The anti-spill valve is constructed and positioned to reduce the risk of water flowing upstream from the humidifier 60, for example to the motor 4144.

[0075] The power supply 4210 may be located inside or outside the external housing 4010 of the RPT device 40. In one form of the present technology, the power supply 4210 provides power only to the RPT device 40. In another form of the present technology, the power supply 4210 provides power to both the RPT device 40 and the humidifier 60.

[0076] The RT system may include one or more transducers (sensors) 4270 configured to measure one or more of any number of parameters related to the RT system, its user, and / or its environment. The transducers may be configured to generate output signals representative of the one or more parameters that the transducers are configured to measure.

[0077] The output signal may be one or more of an electrical signal, a magnetic signal, a mechanical signal, a visual signal, an optical signal, an acoustic signal, or any number of other signals known in the art.

[0078] The transducer can be integrated with another component of the RT system, one exemplary configuration being the transducer located inside the RPT device. The transducer may also be a substantially "standalone" component of the RT system, an exemplary configuration being the transducer located outside the RPT device.

[0079] The transducer may be configured to transmit its output signal to one or more components of the RT system, such as an RPT device, a local external device, or a remote external device. An external transducer may be located, for example, on the user interface or in an external computing device, such as a smartphone. An external transducer may be located, for example, on the air circuit or may form part of the air circuit, such as the user interface.

[0080] 4D illustrates a system 4300 according to some embodiments of the present disclosure. The system 4300 includes a control system 4310 including a processor 4312, a memory device 4314, an electronic interface 4316, and one or more sensors 4270. In some embodiments, the system 4300 may further include a respiratory therapy system 4320, which may be the system of FIG. 2 including an RPT device 40, a user device such as a mobile device 234, and an activity tracker 4322. The device 234 may include a display 4344. In some cases, some or most of the system 4300 may be implemented as the mobile device 232, the RPT device 40, or other external devices such as an external computing device. As described above, the respiratory therapy system 4320 includes the RPT device 40, a user or user interface 100, a conduit 4326, a display such as an output 4290, and a humidifier 60.

[0081] The one or more sensors or transducers 4270 may be constructed and arranged to generate a signal representative of an attribute of the air, such as flow rate, pressure or temperature. The air may be air flow from the RPT device to a user, air flow from the user to the atmosphere, ambient air, or any other air. The signal may represent an attribute of the air flow at a particular point, such as the air flow in the air pressure path between the RPT device and the user. In one form of the present technology, the one or more transducers 4270 are positioned in the air pressure path of the RPT device, such as downstream of the humidifier 60.

[0082] According to one aspect of the present technology, the one or more sensors 4270 include a pressure sensor in fluid communication with the air pressure path. An example of a suitable pressure sensor is a transducer from the HONEYWELL ASDX series. An alternative suitable pressure sensor is a transducer from GENERAL ELECTRIC's NPA series. In one embodiment, the pressure sensor is located in the air circuit 4170 adjacent to the outlet of the humidifier 60.

[0083] The acoustic / audio sensor 4278 (e.g., microphone pressure sensor 4278) is configured to generate a sound signal representative of pressure changes in the air circuit 4170. "Acoustic sensor" or "audio sensor" are interchangeable terms and refer to a sensor capable of detecting sounds audible and / or inaudible to humans. The sound signal from the audio sensor 4278 may be received by the central controller 4230 for acoustic processing and analysis, as configured by one or more algorithms described below. The audio sensor 4278 may be directly exposed to the airway to increase sensitivity to sound, or may be encapsulated behind a thin layer of flexible film material. The film may serve to protect the audio sensor 4278 from the effects of heat and / or humidity. Alternatively, the audio sensor 4278 may be coupled to the RPT device 40, user interface 100 or conduit, or may be integrated into the RPT device 40, user interface 100 or external user device.

[0084] The audio data generated by the audio sensor 4278 can be played as one or more sounds (e.g., sounds from the user 10). The audio data from the audio sensor 4278 can also be used to identify (e.g., using the central controller 4230) or confirm characteristics associated with the user interface, such as the sound of air escaping a valve.

[0085] The speaker may output sound waves audible to a user, such as user 10. The speaker may be used to provide audio feedback, such as, for example, how to operate the RPT device 40 or other device to obtain desired sensor data or to indicate when sensor data collection has been sufficiently completed. In some embodiments, the speaker may be used to transmit audio data generated by the audio sensor 4278 to a user. The speaker may be coupled to or integrated into the RPT device 40, user interface 100, conduit or external device.

[0086] The audio sensor 4278 and the speaker may be used as separate devices. In some embodiments, the audio sensor 4278 and the speaker may be combined as an acoustic sensor (e.g., a sonar sensor), for example, as described in WO 2018 / 050913 and WO 2020 / 104465, each of which is incorporated herein by reference in its entirety. In such embodiments, the speaker generates or emits sound waves at predetermined intervals, and the audio sensor 4278 detects the reflection of the sound waves emitted from the speaker. The sound waves generated or emitted by the speaker have a frequency that is inaudible to the human ear (e.g., below 20 Hz or above about 18 kHz) so as not to interfere with the user 10. Based at least in part on data from the audio sensor 4278 and / or the speaker, the control system may determine positional information regarding the user and / or the user interface 100 (e.g., the position of the user's face, the position of features on the user's face, the position of the user interface 100, the position of features on the user interface 100), physiological parameters (e.g., respiratory rate), and the like. In this context, sonar sensors may be understood to involve active acoustic sensing, such as by generating and / or transmitting ultrasonic and / or low frequency ultrasonic sensing signals (e.g., within a frequency range of about 17-23 kHz, 18-22 kHz, or 17-18 kHz) in the air. Such systems are contemplated with respect to WO 2018 / 050913 and WO 2020 / 104465, mentioned above, each of which is incorporated herein by reference in its entirety. In some embodiments, an additional microphone pressure sensor may be used.

[0087] Data from transducers 4270, such as pressure sensors 4272, flow sensors 4274, motor speed sensors 4276, audio sensors 4278, etc., may be periodically collected by the central controller 4230. Such data generally relates to the operational status of the RPT device 40. In this example, the central controller 4230 encodes such data from the sensors in a proprietary data format. The data may also be encoded in a standardized data format.

[0088] The one or more sensors or transducers 4270 may include a temperature sensor 4330, a motion sensor 4332, a speaker 4334, a radio frequency (RF) receiver 4336, an RF transmitter 4338, a camera 4340, an infrared sensor 4342 (e.g., a passive infrared sensor or an active infrared sensor), a photoplethysmography (PPG) sensor 4344, an electrocardiogram (ECG) sensor 4346, an electroencephalogram (EEG) sensor 4348, a capacitance sensor 4350, a force sensor 4352, a strain gauge sensor 4354, an electromyogram (EMG) sensor 4356, an oxygen sensor 4358, an analyte sensor 4360, a humidity sensor 4362, a LiDAR sensor 4364, or any combination thereof. Generally, each of the one or more sensors or transducers 4270 is configured to output sensor data that is received and stored in a storage device.

[0089] The one or more sensors 4270 are shown and described as including each of a pressure sensor 4272, a flow sensor 4274, a motor speed sensor 4276, and an audio sensor 4278, however the other sensors described above may include any combination and any number of each sensor described and / or illustrated herein.

[0090] One or more sensors 4270 may be used to generate sensor data such as image data, audio data, ranging data, contour mapping data, thermal data, physiological data, environmental data, etc. The control system may use the sensor data to determine the user interface and to identify characteristics related to the current fitness of the user interface 100.

[0091] An exemplary pressure sensor 4272 is an air pressure sensor (e.g., a barometric sensor) that generates sensor data indicative of a user's respiration (e.g., inhalation and / or exhalation) and / or ambient pressure of the respiratory therapy system of FIG 1. In such an embodiment, the pressure sensor 4272 may be coupled to or integrated within the RPT device 40. The pressure sensor 4272 may be, for example, a capacitive sensor, an electromagnetic sensor, a piezoelectric sensor, a strain sensor, an optical sensor, a potentiometric sensor, or any combination thereof.

[0092] Examples of flow sensors (e.g., flow sensor 4274) are described in WO 2012 / 012835, the entire contents of which are incorporated herein by reference. In some embodiments, the flow sensor 4274 is used to determine the airflow rate from the RPT device 40, the airflow rate through the conduits of the air circuit 4170, the airflow rate through the user interface 100, or any combination thereof. In such embodiments, the flow sensor 4274 may be coupled to or integrated with the RPT device 40, the user interface 100, or the user interface 100 to the conduits of the RPT device 40. The flow sensor 4274 may be a mass flow sensor, such as a rotary flow meter (e.g., a Hall effect flow meter), a turbine flow meter, an orifice flow meter, an ultrasonic flow meter, a hot wire sensor, an eddy current sensor, a membrane sensor, or any combination thereof. In some embodiments, the flow sensor 4274 is configured to measure vent flow (e.g., intentional "leak"), unintentional leak (e.g., mouth leak and / or mask leak), user flow (e.g., air in and / or out of the lungs), or any combination thereof. In some embodiments, the flow data can be analyzed to determine cardiogenic oscillations of the user.

[0093] In some embodiments, the temperature sensor 4330 generates temperature data indicative of the core body temperature of the user 10, the user's local or average skin temperature, the local or average temperature of the air flowing from the RPT device 40 and / or through the conduit, the local or average temperature within the user interface 100, the ambient temperature, or any combination thereof. The temperature sensor 4330 may be, for example, a thermocouple sensor, a thermistor sensor, a silicon band gap temperature sensor or semiconductor based sensor, a resistance temperature detector, or any combination thereof. In some cases, the temperature sensor may be a non-contact temperature sensor, such as an infrared pyrometer.

[0094] The RF transmitter 4338 generates and / or emits radio waves having a predetermined frequency and / or amplitude (e.g., high frequency band, low frequency band, long wave signal, short wave signal, etc.). The RF receiver 4336 detects reflections of the radio waves emitted from the RF transmitter 4338, and this data can be analyzed by the control system to determine location information about the user 10 and / or the user interface 100, and / or one or more physiological parameters described herein. The RF receiver (RF receiver and RF transmitter or another RF pair) can also be used for wireless communication between external components and the RPT device 40, or any combination thereof. The RF receiver 4336 and the RF transmitter 4338 can be combined as part of an RF sensor (e.g., a RADAR sensor). In such an embodiment, the RF sensor includes a control circuit. The particular format of the RF communication may be WiFi, Bluetooth, etc.

[0095] In some embodiments, the RF sensor is part of a mesh system. One example of a mesh system is a WiFi mesh system, which may include mesh nodes, mesh routers, and mesh gateways, each of which may be mobile or fixed. In such an embodiment, the WiFi mesh system includes a WiFi router and / or a WiFi controller, and one or more satellites (e.g., access points), each of which includes an RF sensor identical or similar to the RF sensor. The WiFi router and satellite constantly communicate with each other using WiFi signals. The WiFi mesh system may be used to generate motion data based on changes in the WiFi signal between the router and the satellite (e.g., differences in received signal strength) due to moving objects or people locally disrupting the signal. The motion data may indicate movement, breathing, heart rate, gait, falls, behavior, etc., or any combination thereof.

[0096] The control system can use image data from the camera 4340 to determine information related to the user's face, the user interface 100, and / or one or more physiological parameters described herein. For example, image data from the camera 4340 can be used to identify the user's position, the local color of a portion of the user's face, the relative position of features on the user interface to the user's facial features, etc. In some embodiments, the camera includes a wide-angle lens or a fisheye lens. The camera can be, for example, a camera operating in the visible spectrum, such as at or about 380 nm and wavelengths between or about 740 nm.

[0097] The IR sensor 4342 may be a passive sensor or an active sensor. A passive infrared sensor may measure natural infrared emissions or reflections from a remote surface, such as measuring infrared energy emitted from the surface to determine the temperature of the surface. An active IR sensor may include an IR transmitter that generates an IR signal that is received by an IR receiver. Such active IR sensors may be used to measure IR reflections from and / or transmissions through an object. For example, an IR transmitter as a dot projector may use IR light to project an array of identifiable dots onto a user's face, and the IR receiver may then detect reflections of the IR light to determine ranging data related to the user's face (e.g., data related to the distance between the IR sensor 4342 and a remote surface (e.g., a portion of the user's face)) or contour data (e.g., data related to the relative height characteristics of the surface relative to the nominal height of the surface).

[0098] In general, infrared data from the IR sensor 4342 may be used to determine information about the user 10 and / or the interface 100 and / or one or more physiological parameters described herein. In one example, infrared data from the IR sensor may be used to detect a local temperature of a portion of the user's face or a portion of the interface 100. The IR sensor 4342 may also be used in conjunction with a camera, such as, for example, to correlate IR data (e.g., temperature data or ranging data) with camera data (e.g., local color). The IR sensor 4342 may detect infrared light having a wavelength between at or about 700 nm and 1 mm or about 1 mm.

[0099] The PPG sensor 4344 outputs physiological data associated with the user 10 that can be used to determine one or more sleep-related parameters, such as heart rate, heart rate variability, cardiac cycle, respiratory rate, inhalation amplitude, exhalation amplitude, inhalation-exhalation ratio, estimated blood pressure parameters, or any combination thereof. The PPG sensor 4344 may be worn by the user, embedded in clothing and / or fabric worn by the user, embedded in and / or coupled to the interface 100 and / or its associated headgear (e.g., straps, etc.).

[0100] The ECG sensor 4346 outputs physiological data related to the electrical activity of the heart of the user 10. In some embodiments, the ECG sensor 4346 includes one or more electrodes positioned on or around a portion of the user 10 during a sleep session. The physiological data from the ECG sensor 4346 can be used, for example, to determine one or more sleep-related parameters described herein.

[0101] The EEG sensor 4348 outputs physiological data related to electrical activity of the brain of the user 10. In some embodiments, the EEG sensor 4348 includes one or more electrodes placed on or around the scalp of the user 10 during a sleep session. The physiological data from the EEG sensor 4348 may be used, for example, to determine the sleep state and / or sleep stage of the user 10 at any time during the sleep session. In some embodiments, the EEG sensor 4348 may be integrated into the interface 100 and / or associated headgear (e.g., straps, etc.).

[0102] The EMG sensor 4356 outputs physiological data related to electrical activity generated by one or more muscles. The oxygen sensor 4358 outputs oxygen data indicative of the oxygen concentration of a gas (e.g., in the conduit or interface 100). The oxygen sensor 4358 may be, for example, an ultrasonic oxygen sensor, an electrical oxygen sensor, a chemical oxygen sensor, an optical oxygen sensor, a pulse oximeter (e.g., an SpO2 sensor), or any combination thereof. In some embodiments, the one or more sensors 4270 further include a galvanic skin response (GSR) sensor, a blood flow sensor, a respiration sensor, a pulse sensor, a blood pressure sensor, an oximeter sensor, or any combination thereof.

[0103] The analyte sensor 4360 can be used to detect the presence of an analyte, for example, in the breath of the user 10. The control system can use the data output by the analyte sensor 4360 to determine the identity and concentration of any analyte, for example, in the breath of the user 10. In some embodiments, the analyte sensor is placed near the mouth of the user 10 to detect the analyte in the breath exhaled from the mouth of the user 10. For example, if the interface 100 is a face mask that covers the nose and mouth of the user 10, the analyte sensor 4360 can be placed in the mask to monitor the mouth breathing of the user 10. In other embodiments, for example, if the interface 100 is a nasal mask or nasal pillows mask, the analyte sensor can be positioned near the nose of the user 10 to detect the analyte in the breath exhaled through the nose. In other embodiments, if the interface 100 is a nasal mask or nasal pillows mask, the analyte sensor can be placed near the mouth. In this embodiment, the analyte sensor 4360 can be used to detect whether there is an unintentional air leak from the mouth of the user 10. In some embodiments, the analyte sensor 4360 is a volatile organic compound (VOC) sensor that may be used to detect carbon-based chemicals or compounds. In some embodiments, the analyte sensor 4360 may be used to detect whether the user 10 is breathing through their nose or mouth. For example, if the presence of an analyte is detected by data output from an analyte sensor placed near the mouth or in a face mask (in embodiments where the interface e100 is a face mask), the control system may use this data as an indication that the user 10 is breathing through their mouth.

[0104] The humidity sensor 4322 may be used to detect humidity in various areas around the user 10 (e.g., inside the conduit or interface 100, near the face of the user 10, near the connection of the conduit to the interface 100, near the connection of the conduit to the RPT device 40, etc.). Thus, in some embodiments, the humidity sensor 4362 may be coupled to or integrated into the interface 100 or conduit to monitor the humidity of the pressurized air from the RPT device 40. In other embodiments, the humidity sensor 4362 is located near any area where humidity levels need to be monitored. The humidity sensor 4362 may also be used to monitor the humidity of the air in the environment around the user 10, for example, in a bedroom.

[0105] A Light Detection and Ranging (LiDAR) sensor 4364 can be used for depth sensing. This type of optical sensor (e.g., laser sensor) can be used to detect objects and build a three-dimensional (3D) map (e.g., a contour map) of an object, such as a user's face, the interface 100, or the surrounding environment (e.g., living space). LiDAR can typically utilize a pulsed laser to measure time of flight. LiDAR is also referred to as 3D laser scanning. In an example using such a sensor, a fixed or mobile device (e.g., a smartphone) with a LiDAR sensor can measure and map an area more than 5 m away from the sensor. For example, LiDAR data can be fused with point cloud data estimated by an electromagnetic RADAR sensor. LiDAR sensors can also use artificial intelligence (AI) to automatically geofence a radar system by detecting and classifying features in a space that may pose a problem for the radar system, such as glass windows (which can be highly reflective to radar). LiDAR may be used to provide an estimate of a person's height, changes in height when a person sits down or falls, etc. The LiDAR may be used to form a 3D mesh representation of the user's face, the user interface 100 (e.g., when worn on the user's face), and / or the environment. In a further application, the LiDAR may reflect off solid surfaces (e.g., radio-transparent materials) through which the radio waves pass, thereby enabling classification of various types of obstructions. Although a LiDAR sensor is described herein, in some cases, one or more other ranging sensors, such as, for example, ultrasonic ranging sensors, electromagnetic radar sensors, etc., may be used in place of or in addition to the LiDAR sensor.

[0106] In addition to the RPT device 40, conduit, or interface 100, any or all of the above sensors may be located on an external device, such as a mobile user device or an activity tracker. For example, the audio sensor 4278 and speaker 4334 may be integrated within and / or coupled to a mobile device, and the pressure sensor 4272 and / or flow sensor 4274 may be integrated within and / or coupled to the RPT device 40. In some embodiments, at least one of the one or more sensors may be located approximately adjacent to the user 10 during a sleep session (e.g., located on or in contact with a portion of the user 10, worn by the user 10, coupled or located on a nightstand, coupled to a mattress, coupled to a ceiling, etc.).

[0107] In one form of the present technology, the RPT device 40 includes one or more input devices 4220 in the form of buttons, switches, or dials to allow a person to interact with the device. The buttons, switches, or dials may be physical devices accessible via a touch screen, or software devices. In one form, the buttons, switches, or dials may be physically connected to the external housing 4010, and in another form, may be in wireless communication with a receiver electrically connected to the central controller 4230. In one form, the input devices 4220 may be constructed and arranged to allow a person to select values ​​and / or menu options.

[0108] In one form of the present technology, the central controller 4230 is one or more processors suitable for controlling the RPT device 40. Suitable processors may include x86 INTEL processors, processors based on ARM® Cortex®-M processors from ARM Holdings (e.g., STM32 series microcontrollers from ST MICROELECTRONIC). In certain alternative forms of the present technology, 32-bit RISC CPUs such as the STR9 series microcontrollers from ST MICROELECTRONICS manufactured by TEXAS INSTRUMENTS, or 16-bit RISC CPUs such as processors from the MSP430 family of microcontrollers may also be suitable. In one form of the present technology, the central controller 4230 is a dedicated electronic circuit. In one form, the central controller 4230 is an application specific integrated circuit. In another form, the central controller 4230 includes discrete electronic components. The central controller 4230 may be configured to receive input signals from one or more transducers 4270, one or more input devices 4220, and the humidifier 60.

[0109] The central controller 4230 may be configured to provide output signals to one or more of the output device 4290, the therapy device controller 4240, the data communication interface 4280, and the humidifier 60.

[0110] In some forms of the present technology, the central controller 4230 is configured to implement one or more methods described herein, such as one or more algorithms represented as a computer program stored in a non-transitory computer-readable storage medium on an internal memory. In some forms of the present technology, the central controller 4230 may be integrated into the RPT device 40. However, in some forms of the present technology, some methods may be performed by a remote device, such as a mobile computing device. For example, the remote device may analyze stored data, such as data from any of the sensors described herein, to determine ventilator control settings or detect respiratory-related events. As noted above, all data and operations of the external source or central controller 4230 are generally proprietary to the manufacturer of the RPT device 40. Thus, data from the sensors and other additional operational data are typically not accessible to other devices.

[0111] In one form of the present technology, a data communication interface is provided and connected to a central controller 4230. The data communication interface may be connectable to a remote external communication network and / or a local external communication network. The remote external communication network may be connectable to a remote external device such as a server or a database. The local external communication network may be connectable to a local external device such as a mobile device or a health monitoring device. Thus, the RPT device 40 or the mobile device can collect data from other devices using the local external communication network.

[0112] In one form, the data communication interface is part of the central controller 4230. In another form, the data communication interface 4280 can be separate from the central controller 4230 and can include an integrated circuit or processor. In one form, the remote external communication network is the Internet. The data communication interface may be connectable to the Internet using wired communication (e.g., via Ethernet or optical fiber) or wireless protocols (e.g., CDMA, GSM, 2G, 3G, 4G / LTE, LTE Cat-M, NB-IoT, 5G New Wireless, satellite, 5G and beyond). In one form, the local external communication network 4284 utilizes one or more communication standards, such as Bluetooth or consumer infrared protocols.

[0113] An exemplary RPT device 40 includes integrated sensors and communication electronics, as shown in FIG. 4C. Older RPT devices may incorporate a sensor module that may include communication electronics for transmitting collected data. Such a sensor module may be attached to the RPT device, thus transmitting operational data to a remote analysis engine in the server 210.

[0114] FIG. 5 is a schematic diagram of an audio sensor 4278 in an air circuit 4170 that connects the RPT device 40 to the interface 100 of FIG. 1. In this example, the conduit 180 (length L) effectively acts as an acoustic waveguide for the sound generated by the RPT device 40. In this example, the input signal is the sound emitted by the RPT device 40. The input signal (e.g., an impulse) enters the audio sensor 4278 located at one end of the conduit 180, propagates along the air path in the conduit 180 to the mask 100, and is reflected back again along the conduit 180 by features in the air path (including the conduit 180 and the mask 100). Thus, the system IRF (the output signal generated from the input impulse) includes an input signal component and a reflected component. One important feature is the time it takes for the sound to travel from one end of the air path to the other. This interval is expressed in terms of the system IRF, since the audio sensor 4278 receives an input signal from the RPT device 40, and then, after a certain time, receives an input signal that has been filtered by the conduit 180 and reflected and filtered by the mask 100 (and other systems 190 attached to the mask when the mask 100 is worn by a user, e.g., the human respiratory system). This means that the component of the system IRF associated with the reflection from the mask end of the conduit 180 (the reflected component) is delayed relative to the component of the system IRF associated with the input signal that arrives at the audio sensor 4278 after a relatively short delay (the input signal component). (For practical purposes, this short delay may be ignored, with zero time approximating the time the microphone first responded to the input signal). The delay is equal to 2L / c, where L is the length of the conduit and c is the speed of sound in the conduit.

[0115] Another feature is that, because the air pressure path is lossy, if the conduit is long enough, the input signal component will attenuate to a negligible amount when the reflected component of the system IRF begins. In this case, the input signal component can be separated from the reflected component of the system IRF. Alternatively, the input signal may be from a speaker at the device end of the air path.

[0116] Some embodiments of the disclosed acoustic analysis techniques may enable cepstral analysis. The cepstrum may be thought of as the inverse Fourier transform of the log spectrum of the forward Fourier transform of the decibel spectrum. The operation essentially converts the convolution of the impulse response function (IRF) and the sound source into an additive operation, thus isolating the data of the IRF for analysis, so that the sound source can be more easily interpreted or removed. The technique of cepstral analysis is described in detail in the academic paper "The Cepstrum: A Guide to Processing" (Childers et al., Proceedings of the IEEE, Vol. 65, No. 10, Oct 1977) and in "Frequency Analysis" by RANDALL RB, (Copenhagen: Bruel & Kjaer, p. 344 (1977, revised edition 1987)). The application of cepstral analysis in component identification in respiratory therapy systems is described in detail in PCT Publication No. WO2010 / 091462, entitled "Acoustic Detection of Respiratory Therapy Devices," the entire contents of which are incorporated herein by reference.

[0117] As previously mentioned, a respiratory therapy system typically includes a respiratory therapy device (e.g., an RPT device), such as the components shown in FIG. 1, a humidifier, an air delivery conduit, and a patient interface. A variety of different forms of patient interfaces can be used with a given RPT device, such as nasal pillows, nasal prongs, a nasal mask sealed to the nasal ridge, a nasal and nasal (oronasal) mask sealed to the lower rim of the nose but not to the nasal ridge, a nasal and nasal (oronasal) mask sealed to the lower rim of the nose but not to the bridge or nasal ridge as in a traditional full face mask, a tube down mask with a conduit connected to the forward facing portion of the mask, a mask with conduit head gear, a mask with an integrated short tube to which the main conduit is connected, a mask with a separate structure such as an elbow to which the main conduit is directly connected, and other types of masks and variations of the above. Additionally, various forms of air delivery conduits can be used. Measured treatment parameters, such as pressure and airflow within the user interface, can be analyzed or estimated to provide improved control of the therapy delivered to the user interface. In conventional systems, knowledge of the type of parts the user is using may be determined in order to determine the optimal interface for the user, as described below. Some RPT devices include a menu system that allows the user to select the type of system part being used (e.g., brand, form, model, etc.), including the user interface. Once the user enters the type of part, the RPT device may select the appropriate operating parameters of the flow generator that are optimally tuned to the selected part. Data collected by the RPT device may be used to evaluate the effectiveness of a particular selected part (e.g., user interface) with respect to the delivery of pressurized air to the user.

[0118] As discussed above with reference to Figure 5, acoustic analysis can be used to identify components of a respiratory pressure therapy system. As used herein, "identification" of a component refers to identification of the component type. In the following, for simplicity, "mask" is used synonymously with "user interface" even though some user interfaces are not commonly described as "masks."

[0119] The system can identify the length of conduit in use and the mask connected to the conduit by analyzing the sound signals acquired by the audio sensor 4278. This technology can identify the mask and conduit whether or not the user is wearing the mask at the time of identification.

[0120] The technology includes analytical methods that can separate acoustic mask reflections from other system noises and responses (including but not limited to blower noise), making it possible to identify differences between acoustic reflections from different masks (typically determined by mask shape, construction and material), and thus distinguish between different masks without the user or user intervention.

[0121] An exemplary method for identifying a mask is to sample the output sound signal y(t) acquired by the audio sensor 4278 at least at the Nyquist rate (e.g., 20 kHz), calculate a cepstrum from the sampled output signal, and separate the reflection component of the cepstrum from the input signal component of the cepstrum. The reflection component of the cepstrum is called the "acoustic signature" or "mask signature" of the mask, since it contains the acoustic reflection from the mask of the input sound signal. The acoustic signature is then compared to a predefined or pre-determined database of previously measured acoustic signatures obtained from systems containing known masks. If necessary, some criteria are set to determine the appropriate similarity. In an exemplary embodiment, the comparison can be made based on a single maximum data peak in the cross-correlation between the measured acoustic signature and the stored acoustic signature. However, this approach can be improved by making the comparison on several data peaks, or on only the cepstral feature set extracted.

[0122] Alternatively, the same method can be used to determine the conduit length by determining the delay between the sound received from the RPT device 40 and the reflection of the sound received from the mask 100. The delay can be proportional to the length of the tube 180. Additionally, changes in tube diameter can increase or decrease the amplitude of the reflected signal and can therefore be identified. Such an assessment can be made by comparing the current reflected data with previous reflected data. The diameter change can be considered as a percentage of the amplitude change from the reflected signal (i.e., reflected data).

[0123] According to the present technique, the data associated with the reflectance component may then be compared to similar data from previously identified mask reflectance components, such as data contained in a memory or database of mask reflectance components.

[0124] For example, the reflective component ("mask signature") of the mask under test may be isolated from the cepstrum of the output signal generated by the microphone. This mask signature may be compared to mask signatures of previous or pre-defined mask signatures of known masks stored as data templates in the device. One way to do this is to calculate the cross-correlation between the mask signature of the mask under test and the previously stored mask signatures for all known masks or data templates. The cross-correlation with the highest peak is likely to correspond to the mask under test, and the location of the peak should be proportional to the length of the conduit.

[0125] As described above, the RPT device 40 can provide data regarding the type of user interface in addition to operational data. The operational data may be correlated with the mask type and user related data to determine whether a particular mask type is effective. For example, the operational data reflects the amount of time the RPT device 40 has been used and whether its use has resulted in effective treatment. The type of user interface may be correlated with the level of user compliance or effectiveness of treatment as determined from the operational data collected by the RPT device 40. The correlated data may be used to better determine an interface that would be effective by a new user requiring respiratory therapy with a similar RPT device. This selection may be combined with facial dimensions obtained from a facial scan of the new user to aid in the selection of an interface.

[0126] Thus, examples of the present technology allow users to obtain mask-like user interfaces more quickly and conveniently by combining data on different masks collected from a user population using an RPT device with the facial features of the individual user determined by a scanning process. By using the scanning process, a user can quickly measure their facial anatomy in the comfort of their own home using a computing device such as a desktop, tablet, smartphone, or other mobile device. The computing device can then receive or generate recommendations regarding appropriate user interface sizes and types after analyzing the user's facial dimensions and data from the general user population using various interfaces.

[0127] In a beneficial embodiment, the technology may employ an application that is downloadable from a manufacturer or third-party server to a smartphone or tablet with an integrated camera. The application may provide visual and / or audio instructions when activated. As shown, a user (i.e., a user) may stand in front of a mirror and press a camera button on a user interface. The activated process may then take a series of images of the user's face, after which, within, for example, a few seconds, facial dimensions for selecting an interface (based on a processor that analyzes the images) may be obtained.

[0128] A user / user can capture an image or a series of images of their facial structure. An example may be instructions provided by an application program stored on a computer readable medium that, when executed by, for example, a processor, detects various facial landmarks in the image, measures and scales the distances between these landmarks, compares these distances to data records, and recommends an appropriate user interface size. Thus, a consumer automation device can enable accurate user interface selection, such as in the home, and allow customers to determine size without trained employees.

[0129] Other examples may include identifying three-dimensional facial features from an image. Identifying facial features is a matter of determining their size based on the "shape" of the various features. This shape is described as an approximately continuous surface of the user's face. In practice, a continuous surface is not possible, but an approximation of the continuous surface of the face can be obtained by collecting approximately 10k-100k points on the face. There are several exemplary techniques for collecting facial image data to identify three-dimensional facial features.

[0130] One method is to determine the face image from a 2D image. In this method, computer vision (CV) and trained machine learning (ML) models are used to extract the primary facial landmarks. For example, OpenCV and DLib libraries can be used for landmark comparison using a trained number of standard facial landmarks. Once the preliminary facial landmarks are extracted, the derived 3D features must be appropriately scaled. Scaling involves determining an object such as a coin, a credit card, or the user's iris to provide a known scale. For example, the Google Mediapipe Facemesh and Iris models can track the user's iris and scale the facial landmarks to adjust the size of the mask. These models contain 468 facial landmarks and 10 eye landmarks. The iris data is then used to scale the other identified facial features.

[0131] Another way to determine 3D features may be face data acquired from a 3D camera with a depth sensor. A 3D camera, such as those found in iPhone X and later, can perform a 3D scan of the face and convert it back into a meshed (triangulated) surface. The number of surface points is typically on the order of about 50k. In this example, a 3D camera such as an iPhone has two types of output: (a) raw scan data, and (b) a low-resolution blendshape model for face detection and tracking. The latter includes automatic landmarks, while the former does not. Scaling is required for the meshed surface data.

[0132] Another method is to generate a 3D model directly from a 2D image. This involves using a 3D deformable model (or 3DMM) and machine learning to adjust the shape of the 3DMM to fit the face in the image. Single or multiple image views can be acquired from multiple angles and can be acquired from videos captured by a digital camera. The three DMMs may be adapted to match data acquired from multiple 2D images by a machine learning matching routine. The 3DMMs may be adapted to take into account the shape, pose and expression shown in the face image to correct the facial features. Since scaling may still be necessary, detection and scaling of known objects such as eye features like irises can be used as a reference to account for scaling errors due to factors such as age.

[0133] The three-dimensional features or shape data can be used for sizing the mask. One way to match a mask is to align the identified facial surface to the known surface of the proposed mask. The surfaces are then aligned. This alignment can be performed by the Nearest Iterative Closest Point (NICP) technique. A goodness of fit score can then be calculated by determining the average distance, which is the average of the distances between the closest points or corresponding points of the facial features and the mask contact surface. A lower score corresponds to a better fit.

[0134] Another method for determining the size of the mask is to use 3D face scans collected from various users. In this example, it is possible to collect 3D data for over 1,000 users. These users are grouped based on their ideal mask size. In this example, the number of ideal mask sizes available is determined by the mask designer to cover the various user types. Such grouping methods may be based on other types of data, such as grouping based on traditional 2D landmarks or grouping based on principal components of face shapes. Principal component analysis can determine a reduced characteristic set of facial features. An average set of 3D facial features representing each mask size is calculated based on the mask size grouping.

[0135] To determine the size of a new user, 3D data is derived from a 3D face scan or 2D image and a suitability score for the new user for each average face is calculated. The selected mask size and mask type is the mask that corresponds to the average face with the lowest suitability score. Additional personal preferences can also be added. Specific facial features can also be used to create custom sizes based on modifying the available mask types.

[0136] FIG. 7 illustrates an exemplary system 200 that may be implemented for automated facial feature measurement and user interface selection. The system 200 may generally include one or more of a server 210, a communication network 220, and a computing device 230. The server 210 and the computing device 230 may communicate via the communication network 220, which may be a wired network 222, a wireless network 224, or a wired network with a wireless link 226. In some versions, the server 210 may communicate unidirectionally with the computing device 230 by providing information to the computing device 230, or vice versa. In other embodiments, the server 210 and the computing device 230 may share information and / or processing tasks. The system may be implemented to enable automated purchasing of a user interface, such as, for example, the mask 100 of FIG. 1. This process may include an automatic sizing process, which is described in more detail herein. For example, a customer can order a mask online after performing a mask selection process that automatically identifies the appropriate mask size, type, and / or model by image analysis of the customer's facial features in combination with RPT device operation data and operation data from other masks from a patient population that use masks of different types and sizes. The system 200 may include one or more databases. The one or more databases may include a patient database 260, a patient interface database 270, and any other databases described herein. It should be understood that in some examples of the present technology, all data accessed by the system or that needs to be accessed during the execution of a method may be stored in a single database. In other examples, data may be stored in two or more separate databases. Thus, when a particular database is referred to herein, it should be understood that the particular database may be a unique database in some examples and may be part of a larger database in other examples.

[0137] The server 210 and / or computing device 230 may also communicate with a respiratory therapy device, such as an RPT device 250 similar to the RPT device 40 shown in FIG. 1. The RPT device 250 in this example collects operational data, mask leakage, and other related data related to user use to provide feedback related to mask use. Data from the RPT device 250 is collected and correlated with personal user data of users using the RPT device 250 in a user database 260. A user interface database 270 includes data regarding different types and sizes of interfaces, such as masks that may be used by a new user. The user interface database 270 may also include acoustic signature data for each type of mask, which may allow the type of mask to be determined from audio data collected from the respiratory therapy device. A mask analysis engine executed by the server 210 is used to correlate and determine effective mask sizes and types from individual face dimension data and corresponding effectiveness of operational data collected from the RPT device 250 with the entire user population. For example, an effective fit may be evidenced by a minimum of leak detected, a maximum compliance with the treatment regimen (e.g., mask on-off times, frequency of on-off events, treatment pressure used), number of apneas per night, AHI levels, pressure settings used with the device, and predetermined pressure settings. This data may be correlated with facial dimensional data or other data based on a facial image of the new user.

[0138] For example, the facial shape obtained by imaging a user's face (e.g., 3D scan data) can be compared to the geometry of the proposed mask (cushion, conduits, headgear) features. Differences in shape and geometry can be analyzed to determine if there are fit issues that may cause leaks or areas of high contact pressure that may cause redness or pain in the contact areas. As described herein, data collected on a user population can be combined with other forms of data, such as detected leaks, to identify an optimal mask system for a particular facial shape (i.e., shape of mouth, nose, cheeks, head, etc.).

[0139] As described below, the server 210 collects data from a plurality of users stored in the database 260 and the respective mask size and type data stored in the database 270 to select an appropriate mask based on an optimal mask that best suits the scanned facial dimension data collected from the new user and a mask that achieves optimal operational data for a user having similar facial dimensions and other characteristics, sleep behavior data, and demographic data to the new user. In some examples, the system 200 includes one or more databases for storing a plurality of facial features from a user population, a plurality of corresponding user interfaces used by the user population, and operational data of a respiratory pressure therapy device used by the user population. A selection engine may be coupled to the one or more databases, the selection engine operable to select a user interface for the user based on a desired outcome and based on the stored operational data and the facial features of the user. The system 200 may be configured to execute a corresponding method for selecting a user interface. Thus, the system 200 may provide a mask recommendation to the new user by determining which masks have been shown to be optimal for existing users similar to the new user in various respects. The optimal mask may be, for example, the type, model, and / or size of mask that has been shown to be associated with maximizing treatment compliance, minimizing leaks, minimizing apneas, minimizing AHI, and most positive subjective user feedback. In various examples of the present technology, the influence of each of these outcomes in determining the optimal mask may be given various weights.

[0140] The computing device 230 may be a desktop or laptop computer 232, or a mobile device such as a smartphone 234 or a tablet 236. Figure 7 shows a general architecture 300 of the computing device 230. The device 230 may include one or more processors 310. The device 230 may also further include a display interface 320, a user control / input interface 331, a sensor 340 and / or a sensor interface for one or more sensors, an inertial measurement unit (IMU) 342, and non-volatile memory / data storage 350.

[0141] The sensor 340 may be one or more cameras (e.g., CCD charge-coupled device or active pixel sensors) built into the computing device 230, such as a camera provided in a smartphone or laptop. Alternatively, if the computing device 230 is a desktop computer, the device 230 may include a sensor interface for coupling to an external camera, such as the webcam 233 illustrated in FIG. 6. Other exemplary sensors that may be used to assist in the methods described herein include stereo cameras for capturing three-dimensional images, or photodetectors capable of detecting reflected light from a laser light source or a strobe / structured light source, and may be integrated into or external to the computing device.

[0142] User control / input interface 331 allows a user to input commands or respond to prompts or instructions provided to the user, and may include, for example, a touch panel, keyboard, mouse, microphone, speaker, etc.

[0143] The display interface 320 may include a monitor, LCD panel, etc. for displaying prompts, output information (such as facial measurements or interface size recommendations), and other information such as an input display, as described in further detail below.

[0144] The memory / data storage 350 may be an internal memory of the computing device, such as RAM, flash memory, or ROM. In some embodiments, the memory / data storage 350 may be an external memory linked to the computing device 230, such as an SD card, a server, a USB flash drive, or an optical disk. In other embodiments, the memory / data storage 350 may be a combination of external and internal memory. The memory / data storage 350 includes stored data 354 and processor control instructions 352 that direct the processor 310 to perform certain tasks. The stored data 354 may include data received by the sensor 340, such as captured images, and other data provided as part of an application. The processor control instructions 352 may also be provided as part of an application.

[0145] As mentioned above, the facial image may be captured by a mobile computing device such as a smartphone 234. An appropriate application running on the computing device 230 or the server 210 can provide three-dimensional associated facial data to assist in the selection of an appropriate mask. The application may use an appropriate method of facial scanning. Such applications may include Capture from StandardCyborg (https: / / www.standardcyborg.com / ), applications from Scandy Pro (https: / / www.scandy.co / products / scandy-pro), Beauty3D application from Qianxun3d (http: / / www.qianxun3d.com / scanpage), Unre 3D FaceApp (http: / / www.unre.ai / index.php?route=ios / detail) and Bellus3D (https: / / www.bellus3d.com / ). Detailed processes of facial scanning include those disclosed in WO 2017000031, the entirety of which is incorporated herein by reference.

[0146] One such application is an application 360 for facial feature measurement and / or user interface sizing, which may be an application downloadable to a mobile device such as the smartphone 234 and / or tablet 236. The application 360, which may be stored in a computer-readable medium such as the memory / data storage 350, includes programming instructions for the processor 310 to perform certain tasks related to facial feature measurement and / or user interface sizing. The application also includes data that may be processed by algorithms in an automated methodology. Such data may include data records, reference features, and correction factors, as described in more detail below.

[0147] Application 360 is executed by processor 310 to measure a user's facial features using two-dimensional or three-dimensional images and, based on the measurements obtained, select an appropriate user interface size and type, for example from a set of standard sizes. The method may be generally characterized as including three or four distinct stages: a pre-capture stage, a capture stage, a post-capture image processing stage, and a comparison and output stage.

[0148] In some cases, the application for facial feature measurement and sizing of the user interface may control the processor 310 to output a visual display including the reference feature on the display interface 320. The user may position the feature adjacent to their facial feature, such as by moving the camera. The processor may then capture and store one or more images of the facial feature in relation to the reference feature if certain conditions, such as alignment conditions, are met. This may be done with the assistance of a mirror. The mirror reflects the displayed reference feature and the user's face in the camera. The application may then control the processor 310 to identify specific facial features in the image and measure the distance between them. The image analysis process may convert facial feature measurements, such as pixel counts, into standard mask measurements based on the reference feature using a scaling factor. Such values ​​may be values ​​expressed in standardized units of measurement, such as meters or inches, and such units appropriate for sizing the mask.

[0149] Additional correction factors may be applied to the measurements. The facial feature measurements may be compared to a data record containing measurement ranges for different user interface sizes corresponding to a particular user interface format (e.g., nose mask and FFM). A recommended size is then selected and output to the user / patient as a recommendation based on the comparison. Such a process may be conveniently performed within any user's comfort zone. The application may perform this method within seconds. In one example, the application performs this method in real time.

[0150] During the pre-capture stage, the processor 310 assists the user in, among other things, establishing appropriate conditions for capturing one or more images for the sizing process. Some of these conditions include, for example, proper lighting and camera orientation, as well as motion blur caused by unsteady hands holding the computing device 230.

[0151] A user can easily download applications to perform automated measurement and sizing on a user device, such as computing device 230, from a server, such as a third-party application store server, to computing device 230. Once downloaded, such applications can be stored in the computing device's internal non-volatile memory, such as RAM or flash memory. Computing device 230 is preferably a mobile device, such as a smartphone 234 or tablet 236.

[0152] When a user launches the application, the processor 310 may prompt the user to provide user-specific information, such as age, gender, weight, and height, via the display interface 320. However, the processor 310 may prompt the user to input information at any time, such as after measuring the user's facial features. The processor 310 may also present a tutorial, which may be presented by audio and / or visuals provided by the application, to help the user understand their role in the process. The prompt may also require information about the user interface type, such as nose or full face, the type of device on which the user interface will be used, etc. Additionally, during the pre-capture phase, the application may infer user-specific information based on information collected by the user, such as after receiving a captured image of the user's face, based on machine learning techniques, or by artificial intelligence.

[0153] When the user is ready to proceed (which may be indicated by user input or by response to a prompt via user control / input interface 331), processor 310 activates sensor 340 according to the application's processor control instructions 352. Sensor 340 is preferably a front-facing camera of a mobile device located on the same side as the display device interface 320. The camera is typically configured to capture two-dimensional images. Mobile device cameras that capture two-dimensional images are ubiquitous. The present technology takes advantage of this ubiquity to avoid burdening users with the need to obtain specialized equipment.

[0154] At approximately the same time that the sensor / camera 340 is activated, as directed by the application, the processor 310 presents a capture display on the display interface 320. The capture display may include a live-action preview of the camera, a reference feature, a target box, and one or more status indicators, or any combination thereof. In this example, the reference feature is displayed in the center of the display interface and has a width corresponding to the width of the display interface 320. The vertical position of the reference feature may be such that the top edge of the reference feature abuts the top edge of the display interface 320 or the bottom edge of the reference feature abuts the bottom edge of the display interface 320. A portion of the display interface 320 displays a live-action preview 324 of the camera, typically displaying the user's facial features captured by the sensor / camera 340 in real time when the user is in the correct position and orientation.

[0155] Reference features are features known (predetermined) to the computing device 230 and provide the processor 310 with a frame of reference that allows the processor 310 to scale a captured image. The reference features are preferably features other than the user's facial or anatomical features. Thus, during the image processing stage, the reference features assist the processor 310 in determining when certain alignment conditions are met, such as during the pre-capture stage. The reference features may be Quick Response (QR) codes, known swatches or markers, etc. that can provide the processor 310 with certain information, such as scaling information, orientation, and / or any other desired information that can optionally be determined from the structure of the QR code. The shape of a QR code is a square or rectangle. When displayed on the display interface 320, the reference features have predetermined dimensions, such as in millimeters or centimeters, whose values ​​can be encoded in the application and communicated to the processor 310 at the appropriate time. The actual size of the reference features 326 can vary among various computing devices. In some versions, the application may be configured as model specific computing device, and the size of the reference feature 326 is known when displayed on the particular model. However, in other embodiments, the application may instruct the processor 310 to obtain specific information from the device 230, such as, for example, scaling characteristics that enable the processor 310 to calculate the real-world / actual size of the reference feature displayed on the display interface 320 depending on the display size and / or scaling. Nevertheless, the actual dimensions of the reference feature displayed on the display interface 320 of such a computing device are generally known prior to post-capture image processing.

[0156] Along with the reference features, a targeting box may be displayed on the display interface 320. The targeting box allows the user to align a particular part in the capture display 322 within the targeting box, which is desirable for successful image capture.

[0157] A status indicator provides the user with information about the status of the process, preventing the user from having to make significant adjustments to the sensor / camera alignment before image capture is complete.

[0158] Thus, when a user holds the display interface 320 parallel to the facial feature to be measured and presents the user display interface 320 to a mirror or other reflective surface, the reference feature is highlighted and covers the live image seen by the camera / sensor 340 as reflected by the mirror. This reference feature may be fixed near the top of the display interface 320. This at least locally highlighting of the reference feature allows the sensor 340 to clearly see the reference feature and the processor 310 to easily identify the feature. Additionally, the reference feature may cover the real-time view of the user's face, helping to avoid user confusion.

[0159] The user may also be instructed by the processor 310, via the display interface 320, via the speaker of the computing device 230, or pre-directed by a tutorial to position the display interface 320 in the plane of the facial feature to be measured. For example, the user may be instructed to position the display interface 320 so that it faces forward and is positioned under, against, or adjacent to the user's chin in a plane aligned with the particular facial feature to be measured. For example, the display interface 320 may be positioned so that it is planarly aligned with the cerulean and supreme. Since the final captured image is two-dimensional, the planar alignment helps ensure that the scale of the reference feature 326 is applied to the facial feature measurement as well. In this regard, the distance between the mirror and both the user's facial features and the display will be approximately the same.

[0160] When the user is positioned in front of a mirror and the display interface 320 including the reference features is positioned in approximately planar alignment with the facial features to be measured, the processor 310 checks for conditions that help ensure sufficient alignment. As previously mentioned, one exemplary condition that may be established by the application is that the reference features must be found entirely within the target box 328 before proceeding. If the processor 310 detects that the reference features are not located entirely within the target box, the processor 310 may inhibit or delay image capture. The user may then move their face along the display interface 320 to maintain planarity until the reference features are located within the target box as displayed in the live action preview. This facilitates optimal alignment of the facial features and the display interface 320 relative to the mirror for image capture.

[0161] Once the processor 310 detects all the reference features within the target box, the processor 310 can read the IMU 342 of the computing device to detect the device tilt angle. The IMU 342 can include, for example, an accelerometer or a gyroscope. Thus, the processor 310 can evaluate the device tilt, such as by comparison with one or more thresholds, to ensure that it is within an appropriate range. For example, if it is determined that the computing device 230, and thus the display interface 320 and the user's facial features, are tilted in any direction within about 5 degrees, the process can proceed to the capture stage. In other embodiments, the tilt angle for continuation may be within about ±10 degrees, ±7 degrees, ±3 degrees, or ±1 degree. If excessive tilt is detected, a warning message may be displayed or sounded to correct the undesired tilt. This is particularly useful to assist the user in inhibiting or reducing excessive tilt, especially in the forward and backward directions. If this is not corrected, the captured reference image may not have the proper aspect ratio, which can cause measurement errors.

[0162] Once the alignment has been determined by the processor 310, as controlled by the application, the processor 310 enters a capture phase. The capture phase is preferably performed automatically once the alignment parameters and other preceding conditions are met. However, in some embodiments, the user may initiate capture in response to a prompt to do so.

[0163] When image capture is initiated, the processor 310 captures n images, preferably more than one, via the sensor 340. For example, the processor 310 may capture approximately 5-20 images, 10-20 images, or 10-15 images, etc., via the sensor 340. The amount of captured images may be time-based. In other words, the number of images captured may be based on the number of images of a given resolution that can be captured by the sensor 340 during a given time interval. For example, if the number of images that the sensor 340 can capture at a given resolution per second is 40, and the given time interval for capture is 1 second, the sensor 340 will capture 40 images for processing by the processor 310. The number of images may be user-defined or may be determined by the server 210 based on detected environmental conditions or based on expected accuracy targets. For example, if high accuracy is required, more captured images may be required. While capturing multiple images for processing is preferred, one image may be considered and used successfully to obtain accurate measurements. However, using multiple images may allow for an average measurement to be obtained. This may reduce errors / inconsistencies and improve accuracy. The images may be placed by the processor 310 into stored data 354 in memory / data storage 350 for post-capture processing.

[0164] Once an image is captured, the image is processed by the processor 310 to detect or identify facial features / landmarks and measure distances between landmarks. The resulting measurements may be used to recommend an appropriate user interface size. Alternatively, this processing may be performed by the server 210 receiving the transmitted captured image and / or on the user's computing device (e.g., a smartphone). Processing may also be performed by a combination of the processor 310 and the server 210. In one example, the recommended user interface size may be based primarily on the width of the user's nose. In another example, the recommended user interface size may be based on the size of the user's mouth and / or nose.

[0165] The processor 310, as controlled by the application, retrieves one or more captured images from the stored data 354. The images are then extracted by the processor 310 to identify each pixel that makes up the two-dimensional captured image. The processor 310 then detects certain pre-designated facial features within the pixel arrangement.

[0166] The detection may be performed by the processor 310 using edge detection such as Canny, Prewitt, Sobel or Robert. These edge detection techniques / algorithms facilitate identifying the location of specific facial features within the pixel configuration that correspond to the actual facial features of the user presented for image capture. For example, the edge detection technique may first identify the user's face within the image and may also identify pixel locations within the image that correspond to specific facial features, such as each eye and its border, the mouth and its corners, the left and right alae of the nose, therion, supramenton, glabella, and the left and right nasolabial folds. The processor 310 may then mark, tag, or store the specific pixel locations of each of these facial features. Alternatively, or if such detection by the processor 310 / server 210 is not successful, the pre-designated facial features may be manually detected, marked, tagged, or stored by a human operator having viewing access to the captured images via the user interface of the processor 310 / server 210.

[0167] Once the pixel coordinates of these facial features are identified, the application controls the processor 310 to measure pixel distances between certain identified features. For example, the distances are generally determined by the number of pixels in each feature and may include scaling. For example, measurements between the left and right alars may be taken to determine the pixel width of the nose, and / or measurements between the selions and supramentons may be taken to determine the pixel height of the face. Other examples include pixel distances between each eye, between the corners of the mouth, and between the left and right nasolabial folds to obtain additional measurement data for certain structures such as the mouth. Distances between facial features may also be measured further. In this example, several facial dimensions are used in the user interface selection process.

[0168] Once pixel measurements of the pre-specified facial features are obtained, an anthropometric correction factor can be applied to the measurements. It should be understood that this correction factor can be applied before or after applying the scaling factor, as described below. The anthropometric correction factor can correct for errors that may occur during the automated process, and these errors are observed to occur consistently from user to user. In other words, without the correction factor, an individual automated process may produce consistent results between users, but may result in user interfaces that vary in size to some degree. The correction factor, which can be empirically extracted from population testing, brings the results closer to the actual measurements, helping to reduce or eliminate size errors. As each user's measurement and sizing data is transmitted from the corresponding computing device to the server 210, this correction factor can be refined or improved in accuracy over time, and such data may be further processed in the server 210 to refine the correction factor. The anthropometric correction factor may also vary depending on the form of the user interface. For example, the correction factor for a particular user seeking an FFM may be different from the correction factor when seeking a nose mask. Such a correction factor may be derived from tracking of mask purchases, for example, by monitoring mask returns and determining the size difference between replacement masks and returned masks.

[0169] To apply the facial feature measurements to the sizing of the user interface, the measurements may be scaled from pixels to other values ​​that accurately reflect the distance between the user's facial features for image capture, whether or not they have been corrected by an anthropometric correction factor. The reference features may be used to obtain one or more scaling values. Thus, the processor 310 determines the dimensions of the reference feature, which may similarly include pixel width and / or pixel height (x and y) measurements (e.g., number of pixels) of the entire reference feature. The pixel dimensions of the many squares / dots that make up the QR Code reference feature, and / or more detailed measurements of the pixel area occupied by the reference feature and its constituent parts may also be determined. Thus, each square or dot of the QR Code reference feature may be measured in pixels to determine a scaling factor based on the pixel measurements of each dot, and then averaged among all the measured squares or dots. This improves the accuracy of the scaling factor compared to measuring the full size of the QR Code reference feature once. However, it should be understood that whatever measurements are taken of the reference feature, the measurements may be utilized to scale the pixel measurements of the reference feature to the corresponding known dimensions of the reference feature.

[0170] Once the measurements of the reference feature are obtained by the processor 310, a scaling factor is calculated by the processor 310 under the control of the application. The pixel measurements of the reference feature are related to a known corresponding dimension of the reference feature, such as the reference feature 326 displayed by the display interface 320 for image capture, to obtain a conversion factor or scaling factor. Such a scaling factor may be in the form of length / pixel or area / pixel A2. In other words, the known dimension may be divided by the corresponding pixel measurement (e.g., number).

[0171] The processor 310 then applies a scaling factor to the facial feature measurements (number of pixels) to convert them from units of pixels to other units to reflect the distance between the user's actual facial features appropriate for sizing the mask, which may typically involve multiplying the scaling factor by the number of pixels of facial feature distance associated with sizing the mask.

[0172] The steps of measuring and calculating both facial features and reference features are repeated for each captured image until each image in the set contains scaled and / or corrected facial feature measurements.

[0173] The corrected and scaled measurements of the set of images may then be optionally averaged by processor 310 to obtain a final measurement of the user's facial anatomy. Such measurements may reflect the distances between the user's facial features.

[0174] In the comparison output stage, the results from the post-capture image processing stage can be output (displayed) directly to the interested party or compared with the data records to obtain automatic recommendations for the size of the user interface.

[0175] Once all measurements have been determined, the results (e.g., average values) may be displayed to the user by the processor 310 via the display interface 320. In one embodiment, this may end the automated process. The user / patient may record the measurements for further use by the user.

[0176] Alternatively, the final measurements may be transferred, either automatically or at the command of the user, from the computing device 230 to the server 210 via the communications network 220. The server 210 or an individual at the server can perform further processing and analysis to determine an appropriate user interface and user interface size.

[0177] In further embodiments, the final facial feature measurements reflecting the distances between the user's actual facial features are compared by the processor 310 to the user interface size data as in the data record. The data record may be part of an application for automated measurement of facial features and sizing of user interfaces. The data record may include a lookup table accessible by the processor 310 that may contain user interface sizes corresponding to facial feature distance / value ranges, for example. The data record may include multiple tables, many of which may correspond to particular forms of user interfaces and / or particular models of user interfaces provided by a manufacturer.

[0178] An exemplary process for selecting a user interface identifies important landmarks from the facial image captured by the methods described above. In this example, as shown by lines 3010, 3020, and 3030 in Figures 3A-3B, the initial correlation with the potential interface involves facial landmarks including face height, nose width, and nose depth. Measurements of these three facial landmarks are collected by the application to help select a compatible mask size via the lookup table or tables described above. Alternatively, other data about the 3D shape of the face can be used to match the derived shape data to the surface of the mask available as described above. For example, the landmarks or any region of the face (mouth, nose, etc.) can be obtained by fitting a 3D deformable model (3DMM) to the 3D face scan of the user. This fitting process is also referred to as non-rigid registration or (shrink) wrapping. Once the 3DMM is registered to the 3D scan, any number of methods can be used to determine the mask size since the points and surfaces of the user's face are known.

[0179] FIG. 8A is a face image 800 used to determine the face height dimension, nose width dimension, and nose depth dimension, for example captured by the application described above. The image 800 includes a set of landmark points 810 that can be determined from the image 800 by any standard known method. In this example, there are 100 standard cyborg gland mark points that are identified and displayed on the face image 800. In this example, the method requires seven landmarks on the face image to determine the face height, nose width, and nose depth, and to determine the size of the mask for the user. As will be explained later, two existing landmarks (e.g., based on standard cyborg gland marks) can be used. To confirm the three-dimensional position, five additional landmarks need to be identified on the image by the processing method. New landmarks can be determined based on the imaging data and / or existing landmarks. The two existing landmarks used include a point on the selion (bridge of the nose) and a point on the tip of the nose. The five new landmarks required include a point on the supramenton (top of the chin), a point on the left and right nasal alar, and a point on the left and right nasofacial sulcus.

[0180] 8B shows a face image 800 in which the face height dimension (from therion to the supplementon) is defined via landmarks 812 and 814. Landmark 812 is an existing landmark on therion. Landmark point 814 is a point on the supplementon. The face height dimension is determined based on the distance between landmark points 812 and 814.

[0181] FIG. 8C shows a face image 800 with new landmark points 820 and 822 to identify the nose width measurement. This requires two new marks, one on each side of the nose. These are called the right and left alar points and can correspond to the right and left nostrils. The distance between these points determines the nose width measurement. The alar points are different from, but similar to, the nasofacial groove points.

[0182] 8D shows a face image 800 with landmarks 830, 832, 834 for determining the nose depth dimension. Any suitable landmark can be used for the nose tip landmark 830. Landmark points 832 and 834 are determined on the left and right side of the nose. The landmark points 832, 834 are in the nasofacial groove on the left and right side of the nose. These are similar to the alar points but are located behind the nose.

[0183] As discussed above, operational data for each RPT device can be collected for multiple users. This can include usage data based on when each user operated the RPT device and the duration of operation. Thus, compliance data, such as the time and frequency with which the user used the RPT device within a given time period, the therapeutic pressure used, and / or whether the amount and manner of use of the RPT device is consistent with the user's respiratory therapy prescription, can be determined from the collected operational data. For example, one compliance criterion may be that the user acceptably uses the RPT device for a 90-day period. Leakage data can be determined from operational data, such as analysis of flow or pressure data. Mask switching data can be derived using analysis of acoustic signals to determine whether the user is switching masks. The RPT device is operable to determine mask type utilizing the cepstral analysis described above based on an internal or external audio sensor, such as audio sensor 4278 of FIG. 4B. Alternatively, for older masks, the operational data can be used to determine the mask type by correlating the collected acoustic data with the acoustic signatures of known masks.

[0184] In this example, user input of other data may be collected via a user application running on the computing device 230. The user application may be part of the user application 360 that prompts the user to capture facial landmark features or may be a separate application. This may also include subjective data obtained by a questionnaire survey with questions to collect data regarding comfort preferences, whether the user is a mouth or nose breather (e.g., "Is your mouth dry when you wake up?"), and mask material preferences (silicone, foam, textile, gel, etc.). For example, user input may be collected by the user answering subjective questions regarding the comfort of the user interface via the user application. Other questions may relate to user behavior regarding sleep characteristics, etc. For example, subjective questions may include questions such as, "Do you have a dry mouth when you wake up?", "Are you a mouth breather?", "What is your comfort preference?", etc. Such sleep information may include the duration of sleep, how the user sleeps, external influences such as temperature and stress factors. The subjective data may be as simple as a point rating regarding comfort, or a more detailed response. Such subjective data may also be collected from a graphical user interface (GUI). For example, input data regarding leakage from the user interface experienced by the user during treatment may be collected by the user selecting a portion of the user interface displayed on the user interface graphic on the GUI. The collected user input data may be delivered to a user database 260 of FIG. 7. The subjective input data from the user may be used as input for selecting an exemplary mask type and size. Other subjective data regarding the user's psychological safety may be collected. For example, questions such as whether the user feels claustrophobic in a particular mask or how psychologically comfortable it is to wear a mask next to a bed partner may be asked and input collected.If the answers to these questions are low, indicating a negative response, the system can recommend a less obtrusive interface from the interface database 270, which may be a mask that is smaller than the user's existing mask, such as a nasal cradle mask (a mask that is sealed to the user's face at the lower periphery of the user's nose and does not cover the user's mouth and nose bridge), or an oronasal mask that is sealed around the user's mouth and also sealed to the lower periphery of the user's nose and does not bond the nose bridge (sometimes known as an ultra-compact full-face mask). Other questions regarding preferred sleeping positions and whether the user prefers to move a lot during the night and whether the user prefers the "freedom" of a tube mask (e.g., a mask with conduit headgear) that can allow for more movement may also be considered. Alternatively, if the user tends to sleep on their back or side and not move, a tube-down mask (e.g., a traditional style mask with tubes extending forward and / or downward from the mask near the user's nose or mouth) may be acceptable.

[0185] Other data sources may collect data outside of RPT device use that may correlate to mask selection. This may include user demographics such as age, sex, or location, AHI severity, which indicates the level of sleep apnea the user has experienced. Another example may be soft tissue thickness determined based on a computed tomography (CT) of the face. Other data may be a predetermined pressure setting for a new user of the RPT device. If a user is given a lower pressure, such as 10 cmH2O, the user may be able to wear a lighter mask suitable for the lower pressure, which may result in increased comfort or less mask on the face. On the other hand, a full face mask may have a very robust seal suitable for 20 cmH2O, but may be less comfortable. However, if the user has a higher pressure requirement, such as 20 cmH2O, a full face mask with a very robust seal may be recommended to the user.

[0186] After a mask is selected, the system continues to collect operational data from the RPT device 250. The collected data is added to the databases 260, 270. Feedback from new and existing users can be used to tailor recommendations of mask options to subsequent users. For example, if the operational data determines that the recommended mask has a high level of leakage, a different mask type can be recommended to the user. Through the feedback loop, the selection algorithm can be refined to learn the specific aspects of the facial geometry that are best suited for a particular mask. This correlation can refine mask recommendations to new users with the facial geometry. Thus, the collected data and the correlated mask type data can provide further updates to the mask selection criteria. Thus, the system can provide additional insights to improve the user's mask selection.

[0187] In addition to mask selection, the system can enable analysis of mask selection with respect to respiratory therapy effectiveness and user compliance. Additional data can be provided through a feedback loop to optimize respiratory therapy based on the data.

[0188] Machine learning can be applied to optimize the mask selection process and provide a correlation between mask type and increasing user compliance with respiratory therapy. Such machine learning can be performed by the server 210. The training data set can be used to train a mask selection algorithm based on the output of favorable operational outcomes and inputs including user demographics, mask size and type, and subjective data collected from the user. Machine learning can be used to discover correlations between desired mask size and predictive inputs, such as facial dimensions, user demographics, operational data from the RPT device, and environmental conditions. Machine learning can use techniques such as neural networks, clustering, or traditional regression techniques. Test data can be used to test different types of machine learning algorithms and determine which algorithm is most accurate in predicting the correlations.

[0189] The model for selecting the optimal interface may be continually updated with new input data from the system of Figure 7. Thus, as use of the analytics platform increases, the model may become more accurate.

[0190] As mentioned above, part of the system of FIG. 6 relates to recommending an interface to a user using an RPT device. A second function of the system is a validation process that includes optimizing the interface selection. Once a recommended mask has been provided to the user and used for a predetermined period of time, such as two days, two weeks, or other period, the system can monitor the use of the RPT device and collect other data. If, based on the data collected in this way, the mask is not performing to a high standard as determined from adverse data indicating leakage, insufficient or reduced compliance, or unsatisfactory feedback, the system can reevaluate the mask selection and update the database 260 and machine learning algorithms with the user's results. The system can then recommend a new mask to fit the newly collected data. For example, if a relatively high leakage rate is determined from data based on acoustic signatures and other sensors, the user may be chin-down during REM sleep, which may indicate that a different type of interface is needed, such as a full-face mask, rather than the initially selected nasal mask only or small full-face mask.

[0191] The system may also adjust recommendations in response to good subsequent data. For example, if operational data indicates no leakage from the selected full face mask, the routine may recommend trying a smaller mask for a better experience. Subsequent recommendations may be provided using trade-offs of style, material, variation, and correlation with user preferences to maximize user compliance with the regimen. The trade-offs for an individual user may be determined by an input tree displayed to the user by the application. For example, if the user indicates from a menu of potential problems that skin irritation is an issue, a graphic showing the potential irritation location over the facial image may be displayed to collect data from the user, such as the specific location of the irritation. The particular data may provide a better correlation with the best mask for the particular user.

[0192] FIG. 9 is a flow diagram of a user interface selection process that may be performed by an interface selection engine executed by the server 210 of FIG. 7. The process may collect an image of a user's face (facial image) and then store it in a storage device. The system 200 may include a facial contour engine operable to determine the patient's facial features based on the facial image. In some examples, any step involving facial analysis may be performed by the facial contour engine. In this example, the user's face is scanned by a depth camera of a mobile device, such as the smartphone 234 or tablet 236 of FIG. 7, to generate a 3D image of the face (900). Alternatively, 3D face scan data having a 2D or 3D facial image of the scanned user may be obtained from a storage device. Landmark points in a face mesh are determined based on the 3D scan of the face, and a set of important dimensions and points related to the fit of the user interface, such as face height, nose depth, and nose width, are measured based on the image (902). The measured important dimensions are then correlated to potential interfaces by size and type (904). For example, the correlation may include fitting or non-rigid registration with a 3D MMM. An irregular triangular surface mesh from the 3D scan is fitted to a 3D MMM surface that contains information about the face, including the location and area of ​​key facial regions.

[0193] Operational data is collected from a database relating to respiratory devices and corresponding user interfaces (906). The operational data is analyzed for desired outcomes, such as reduced leakage and optimal compliance, in association with respiratory therapy using the respiratory therapy device. The operational data for the desired outcomes for the determined facial dimensions and points is then correlated with available interfaces of different sizes and types (908). The model is used to select an appropriate interface based on the correlation between the operational data and the facial dimensions and points to obtain the desired outcome, taking into account minimization of leakage, best fit for use of the respiratory therapy device, and best compliance (910). Other factors may also be considered, including user input, such as satisfaction with the selected interface. The selected interface is then stored and transmitted to the user (912). As described above, once the user uses an RPT device with the selected interface, the scanned facial dimension data and the data collected from the RPT device are added to the databases 260 and 270 to further refine the analysis of the selection process.

[0194] FIG. 10 is a subsequent routine that may be executed during one or more specific periods of time after the initial selection of an interface detailed in FIG. 9. For example, the subsequent routine may be executed during the first two days of using the selected interface with the RPT device. As described below, the routine of FIG. 10 may provide a recommendation to maintain the initially selected interface or to switch to another type of interface. The collection of additional objective and subjective data is recorded in the interface database 270. Thus, the routine of FIG. 10 records ongoing usage, switching, and feedback data in the interface database 270 along with a mark in the initially selected mask type as "success" or "failure." This data continually updates the exemplary machine learning driven recommendation engine executed by the server 210.

[0195] The routine begins by collecting 1010 operational data over a set period of time, such as two days of use. For example, the system of Figure 2 may collect objective data compiled from two days of use, such as hours of use and leakage data from the RPT device 250. Of course, other suitable periods of time, greater than or less than two days, may be used as the period for collecting operational data and other relevant data from the RPT device.

[0196] Additionally, subjective feedback data such as seal quality / performance, comfort, general likes and dislikes, etc., can be collected from the interface of a user application executed by the computing device 230 (1012). The subjective data may be collected by subjective questions asked via the two-way communication device or a client application executed on the computing device 230. Thus, subjective data may include responses regarding discomfort and leakage, as well as psychological safety questions such as whether the user is psychologically comfortable wearing the mask. Subjective data may also include information provided by the patient regarding fatigue experienced while awake.

[0197] The routine then correlates the objective and subjective data with the selected mask size / type and the user's facial scan data (1014). In the case of a good outcome, the routine determines that the performance data indicates high compliance, low leakage, and good subjective outcome data from the user (1016). The routine then updates the database and learning algorithm to better match the correlated data (1018). The routine also analyzes the correlated data to determine whether the results could be improved with a more desirable mask (1020). The routine then suggests trying a more suitable mask according to the routine of FIG. 9.

[0198] If the correlation produces an undesirable result (1014), the routine determines that the undesirable result is from at least one of low compliance, high leakage, or unsatisfactory subjective outcome data (1022). The routine then updates the database 270 and the learning algorithm to account for the discrepancies between the user data and the selected mask (1024). The routine then suggests trying a more suitable mask (1026) according to the routine of FIG.

[0199] If the user uses the user interface as prescribed and the user's compliance with the therapy can be determined to be high based on a number of factors, these factors may be weighted equally or differently. High compliance with therapy can be indicated if the user uses the user interface as frequently as instructed. Good compliance can also be indicated if the user uses the user interface for a sufficiently long period of time, e.g., overnight, rather than abandoning treatment overnight. Good compliance can also be indicated by adhering to the prescription (e.g., minimal absences from work). Additionally, if the user uses the prescribed treatment pressure rather than a lower pressure, this may contribute to an assessment of good compliance.

[0200] In some examples, a user may need to replace the user interface or parts thereof according to a predetermined replacement schedule. For example, a user may need to replace the closure module of the user interface according to a predetermined interval based on a predetermined pressure and intensity of use, as wear on the seals and vents due to use and cleaning can adversely affect performance and impact the effectiveness of the therapy. In some examples of the present technology, timely replacement of parts may indicate good compliance with the therapy. If prescription data is not available, user compliance may be determined based on using the user interface and corresponding respiratory therapy device for at least a predetermined number of nights (e.g., at least 4, 5, 6, or 7 nights) and at least a predetermined number of hours (e.g., 4, 5, 6, or 7 hours) per week while utilizing a sustained high therapeutic pressure (at least 4, 6, 8, or 10 cmH2O) and / or (e.g., once or more than once a week) and / or replacing the user interface or parts thereof in a timely manner.

[0201] As can be seen, there are several ways to make a user comply with a treatment. In some examples of the present technology, the degree of compliance is determined as "non-compliant" or "compliant" based on whether one or more criteria are met. In some examples, compliance may be one of "low", "medium", or "high". In further examples, compliance may be represented by a score up to 100%. In some examples, a good level of compliance is not necessarily 100%, but may be at least 75% or 80%.

[0202] When a physical user interface, such as a mask, is made available to the user, the systems and mobile devices described above may also be used to determine the proper fit.

[0203] FIG. 11 is a perspective view of a user, such as user 10, controlling a user device, such as smartphone 234 of FIG. 7, to collect sensor data related to the current fit of a user interface, such as mask 100, according to some implementations of the disclosure. For example, user 10 may be a new user of the respiratory therapy system of FIG. 1. User 10 may have one or more sensors of user device 234 aimed at the user's face immediately after donning user interface 100. Mask 100 may be a mask selected based on the motion data and facial data described above. Although shown in FIG. 11 as smartphone 234, any suitable user device may be used.

[0204] To obtain a measurement, the user 10 can press an appropriate button or interact with the smartphone 234 to initiate the measurement process. Once initiated, the smartphone 234 can optionally provide cues (e.g., prompts and / or instructions) to the user 10 indicating different actions the user 10 should take or stop to achieve the desired measurement. For example, the prompts and / or instructions may include text instructions such as "hold the phone at face height and move it slowly in a figure eight", auditory prompts 382 such as a particular bell sound being played when the measurement process is complete, and / or haptic feedback 380 such as an increased vibration pattern that signals the user 10 to slow down the movement of the smartphone 234. The use of non-visual prompts (e.g., auditory prompts, haptic feedback, etc.) is particularly useful when the user 10 must hold the smartphone 234 in an orientation that prohibits the user 10 from looking at the smartphone 234 display. In some cases, the prompts and / or instructions may be presented to overlay on the image of the user 10, for example in the form of an augmented reality overlay. For example, icons, highlighting, text, and other indicia may be overlaid on an image (eg, live or non-live) of the user 10 to provide instructions on how to perform the measurement process.

[0205] The smartphone 234 may include one or more sensors. In some cases, the user 10 may be instructed to hold the smartphone 234 in a particular orientation to ensure that the desired sensor or sensors are obtaining the desired data. For example, if the smartphone uses an infrared sensor on the front side (e.g., an infrared sensor for unlocking the phone), the user 10 may be instructed to hold the smartphone 234 facing away from the user 10 so that the infrared sensor faces the user's face, and data of the user's face may be obtained. In another example, the smartphone 234 may include a LiDAR sensor on the back side, in which case the user 10 may be instructed to hold the phone facing away from the user 10 so that the LiDAR sensor faces the user's face, and data of the user's face may be obtained. In some cases, the user 10 may be instructed to take multiple measurements with the smartphone 234 positioned in different orientations.

[0206] The smartphone 234 can provide feedback regarding the current fit of the user interface 100, whether during the measurement process (e.g., real-time feedback) and / or after completion of the measurement process (e.g., non-real-time feedback). In one example, while the user 10 is holding the smartphone 234 to take measurements, the smartphone 234 can provide feedback indicating that the user 10 should make an adjustment, such as, for example, pulling on a particular strap, to improve the current fit of the user interface 100. In such an example, the user 10 can make an adjustment while continuing to hold the smartphone 234 so that measurements of the user's face and / or the user interface 100 can continue to be taken. In such a case, the smartphone 234 can provide dynamic feedback indicating how the current fit has improved or changed.

[0207] 12 is a user view of a smartphone 234 for identifying thermal characteristics associated with a current fitness of a user interface 100, according to some embodiments of the present disclosure. The smartphone 234 and user interface 100 may be any suitable user device and user interface. The view shown in FIG. 12 may be implemented during the measurement process (e.g., viewed in real time) or after the measurement has been made (e.g., viewed after the measurement process is completed).

[0208] The user 10 may hold a user device (e.g., smartphone 234) and view a display device 472 (e.g., a display screen) of the smartphone 234. The display device 472 may render a graphical user interface (GUI) that may provide feedback related to the current fit of the user interface 100. The GUI may include an image of the user 10 wearing the user interface 100. The image may be acquired by an infrared sensor and may be a thermal map of the user 10 wearing the user interface 100. The thermal map may depict local temperatures at different points on the user's face and the user interface 100.

[0209] As shown in the close-up, a region 482 of the user's face near the seal between the user interface 100 and the user's face is shown as being much cooler than the surrounding area of ​​the user's face. This cooler region 482 may indicate an unintentional air leak, as air leaking through the seal of the user interface 100 can cool the user's skin by an amount that is perceived by an IR sensor.

[0210] In some cases, the GUI may provide feedback in the form of a score 484 related to the current fitness of the user interface 100. For example, the score 484 may be depicted as a filler bar that may be filled between 0% and 100%. As shown in FIG. 12, the current fitness score 484 of the user interface 100 is currently about 65%.

[0211] In some cases, the GUI may provide feedback in the form of text instructions 486 to improve the current fit. The text instructions 486 may provide instructions to the user 10 to take an action to improve the current fit, such as, for example, tightening the top left strap as shown in Figure 12. The instruction to tighten the top left strap was selected because doing so should reduce or eliminate any air leaks detected by temperature differences in region 482.

[0212] 13 is a user view of a user device, such as a smartphone 234, used to identify contour-based characteristics associated with a current fit of a user interface, according to some embodiments of the present disclosure. The user device may be any suitable user device, such as that shown in FIG.

[0213] A display device 572 (e.g., a display screen) of the smartphone 234 can display a GUI that includes a live image of the user 10. To create the live image of the user 10, the camera 550 of the smartphone 234 can be pointed at the user 10.

[0214] In the illustration shown in FIG. 13, the user 10 has just removed the user interface, leaving behind an indentation 590 that captures a portion of the user's face. This indentation 590 may be detected by visual sensing (e.g., sensing the color of the indentation 590, or the color difference between the indentation 590 and the adjacent surface of the user's face), distance sensing (e.g., communicating the local depth difference between the indentation 590 and the adjacent surface of the user's face), and / or the like. As shown, a region 592 of the indentation 590 is less noticeable than other regions. This region 592 may be identified as an area where the user interface has not been sufficiently pressed against the facial skin, thereby not establishing an effective seal.

[0215] In some cases, the GUI can provide feedback in the form of a score 584 related to the current fit of the user interface. For example, the extent of region 592 can indicate that the current fit of the user interface (e.g., the fit of the user interface removed prior to collecting the sensor data leading to the identification of region 592) is not optimal. The score 584 can be generated and depicted, for example, by a numeric score of "65%". In some cases, an alert can additionally be provided, such as alert 585, indicating that a poor fit was detected. In some cases, the GUI can provide feedback in the form of textual instructions 586 to improve the current fit. The textual instructions 586 can provide instructions to the user 10 to take action to improve the current fit by using a different type of user interface (e.g., a nasal pillows mask instead of a full face user interface). The instruction to use a nasal pillows mask instead of a full face user interface may have been selected due to the nature of the poor fit detected and / or one or more previous attempts to improve the fit of the current user interface. For example, if the user attempts to improve the fit of the current user interface more than a threshold number of times, the system may determine that it may be prudent to attempt to use a different type of user interface to establish a better fit. The available user interfaces may be selected using the routines described above that combine face and motion data from the RPT device.

[0216] In FIG. 13, when depression 590 is detected as a detected contour on the user's face, a change in color on the user's face (e.g., bleaching as some blood is pushed out of the tissue close to the skin surface where the user interface seal rests on the skin surface, or a change in color) can be detected to identify where and how the user interface seal engages the user's face.

[0217] FIG. 14 is a flow chart illustrating a process 1400 for evaluating the fitness of a user interface across multiple user interface transition events, according to some embodiments of the present disclosure. The user interface may be any suitable user interface, such as the user interface 100 of FIG. 1. The process 1400 may be executed using a control system, such as a processor for the smartphone 234 of FIG. 7 or a processor of a server, such as the server 210. The sensor data collected during the process 1400 may be collected from one or more sensors (e.g., one or more sensors of the RPT device 40 of FIGS. 4A-4C), one or more sensors, or all of the sensors may be incorporated into and / or coupled to a user device (e.g., the smartphone 234 of FIG. 7). Examples of user interface transition events include putting on the user interface, removing the user interface, adjusting the user interface (e.g., adjusting an adjustable portion of the user interface, e.g., an adjustable strap, moving the user interface to various positions or orientations on the user's face, etc.), and adjusting a respiratory therapy system such as RPT device 40 coupled to the user interface (e.g., turning the respiratory therapy system on or off, adjusting parameters of the respiratory therapy system, e.g., humidity or heating, etc.).

[0218] In block 1402, first sensor data may be collected prior to wearing the user interface. The first sensor data may be collected of the user's face and, optionally, the user interface (prior to being worn by the user).

[0219] In some cases, the first sensor data may be used in block 1412 to identify one or more characteristics associated with the potential suitability of the user interface. For example, the system may determine that the user would be best suited to use a nasal pillows mask instead of a full face user interface based on characteristics identified from the first sensor data alone (e.g., if a particularly large contour is detected around where a full face user interface seal would normally be located, or if a particularly thick beard is detected around where a full face user interface seal would normally be located).

[0220] However, in some cases, the first sensor data is used in block 1412 to compare with other sensor data. For example, in some cases, the first sensor data may establish a baseline for future comparison, such as a baseline contour map of the face, a baseline thermal map of the face, a baseline detection of one or more features of the face (e.g., detection of eyes, mouth, nose, ears, etc.). The first sensor data may be considered sensor data relevant to the current fit of the user interface, such as when collected prior to wearing the user interface to establish a baseline, which is compared to further sensor data to assess the fit of the user interface when worn. For example, such data may be stored from a face scan performed as described with reference to FIGS. 8A-8D.

[0221] In block 1404, the user interface may be donned. Donning the user interface may include the user placing the user interface on the user's face as if using it (e.g., for respiratory therapy). In some cases, donning the user interface in block 1404 may include adjusting the user interface before donning the user interface, but this is not necessarily the case. When adjusting the user interface before donning the user interface, further analysis of the sensor data may be compared to historical sensor data, historical characteristic data, historical fit scores, etc. For example, donning the user interface in block 1404 may occur immediately after a previous evaluation of the fit of the user interface that identifies actions to be taken to improve the fit. In such an example, the user may perform an action as part of block 1404, and then compare the resulting fit of the user interface to a previous fit of the user interface to determine whether the fit has improved.

[0222] In some cases, in block 1404, putting on the user interface may include wearing the user interface for a threshold duration. For example, if the fit of the user interface is evaluated by comparing sensor data before and after wearing the user interface, process 1400 may proceed from block 1404 to block 1408. In such cases, putting on the user interface in block 1404 may include wearing the user interface for a threshold duration (e.g., at least 10 seconds, 20 seconds, 30 seconds, 40 seconds, 50 seconds, 1 minute, 1.5 minutes, 2 minutes, 5 minutes, 10 minutes, or 15 minutes) to ensure that the user interface is worn for a sufficient time to affect the user's face (e.g., a sufficient time to establish a dip or color change on the user's face).

[0223] In block 1406, second sensor data may be collected while the user interface is being worn (e.g., while worn on the user's face). The second sensor data may be collected from the user's face and the user interface worn by the user. In some cases, the second sensor data may be collected while making one or more adjustments to the user interface and / or the respiratory therapy system coupled to the user interface. In such cases, in block 1412, the second sensor data obtained over time may be used to detect changes in the identified characteristics, and these second sensor data may be used to evaluate the current fit of the user interface. In one example, the thermal data collected in block 1406 may identify temperature changes on the exterior of the user interface and in an area of ​​the user's face adjacent to the user interface when a heater of the respiratory therapy system of FIG. 1 is activated, and these thermal data indicate that there may be an unintentional air leak in the seal of the user interface near the user's facial area. In such examples, additional sensor data, such as audio data collected by audio sensor 4278 of FIG. 4C, can be used to confirm the presence of a leak (e.g., by detecting a characteristic acoustic signal, e.g., an audible or inaudible signal, associated with an unintentional leak).

[0224] The user interface may be removed at block 1408. Removing the user interface may, but is not always, performed while one or more sensors are collecting sensor data.

[0225] Third sensor data may be collected after the user removes the user interface at block 1410. The third sensor data may be similar to the first sensor data from block 1402, except that the third sensor data is affected by changes in the user's face due to wearing the user interface.

[0226] Collecting sensor data at blocks 1402, 1406, and 1410 may include collecting the same type of sensor data, including sensor data from the same sensor or different sensors, and / or collecting different types of sensor data. For example, the first sensor data and the second sensor data collected at blocks 1402 and 1410, respectively, may include ranging sensor data and image data in the visible and IR spectrum, respectively, and the second sensor data collected at block 1406 may include audio data and image data in the visible and IR spectrum.

[0227] At block 1412, one or more characteristics are identified from the sensor data (e.g., the first sensor data, the second sensor data, and / or the third sensor data). Identifying the characteristics at block 1412 may include analyzing a given sensor data set (e.g., the separately analyzed second sensor data) and / or comparing the sensor data sets (e.g., comparing the first sensor data with the third sensor data). Identifying the characteristics at block 1412 may include identifying characteristics indicative of a quality of fit of the user interface. For example, some characteristics indicate a poor fit of the user interface and other characteristics indicate a good fit of the user interface. In one example, characteristics such as a sound indicative of an unintentional air leak indicate a poor fit, while characteristics such as a heat map of the user's face wearing the user interface indicate a good fit indicative of a consistency and / or expected temperature on the surface of the user's face adjacent to the user interface.

[0228] The characteristic may be output in any suitable manner, including as a value (e.g., a numeric or Boolean value), a set of values, and / or a signal (e.g., a set of values ​​that change over time). In one example, a characteristic related to the thermal mapping of a user's face when wearing the user interface may be output as i) a Boolean value of whether a local temperature change (e.g., temperature change over time or distance on the surface of the face) exceeding a threshold has been detected, ii) a set of temperature values ​​taken at various locations on the user's face surrounding the perimeter of the seal of the user interface, iii) a thermal image or video of the user's face, iv) or any combination of I-III.

[0229] In some cases, a score related to the suitability of the user interface may be generated at block 1414. Generating the suitability score may include analyzing the characteristics identified at block 1412 and generating a score from the characteristics. In some cases, the score may be based, or at least in part, on a calculation that uses one or more characteristics of block 1412 as input and / or sensor data from blocks 1402, 1406, and / or 1410.

[0230] In some cases, the score may be calculated using a machine learning algorithm that uses one or more characteristics of block 1412 and / or sensor data from blocks 1402, 1406, and / or 1410 as input. Such a machine learning algorithm may be trained using characteristics and / or sensor data associated with a training set of fitness ratings of users wearing the user interface. The fitness ratings of the training set may be based on subjective user ratings, objective values ​​collected using other equipment (e.g., equipment such as laboratory sensors or user interface outfitted with dedicated sensors and / or dedicated sensing equipment), etc.

[0231] An output may be generated at block 1416. The output or output feedback may include any suitable output that relays information regarding the current fitness of the user interface. The output may be based on one or more characteristics of block 1412, sensor data from blocks 1402, 1406, and / or 1410, and / or a fitness score from block 1414. In some cases, the output may be a fitness score generated in block 1414. In some cases, the output may be raw or processed sensor data from blocks 1402, 1406, and / or 1410 (e.g., a portion of a thermal image or audio recording presented to a user).

[0232] In some cases, the output may be instructions or suggestions for a selected action to improve the current fit of the user interface. For example, if the characteristics identified in block 1412 indicate that an unintentional air leak exists for the user interface at a particular location, the output may include instructions to adjust the user interface (e.g., tighten a strap) to reduce the unintentional air leak and improve the fit of the user interface. In some cases, the improvement of the fit of the user interface may be based on the current fit score generated in block 1414 and past or future fit scores generated in past or future instances of block 1414.

[0233] In some cases, a machine learning algorithm using one or more characteristics of block 1412, sensor data from blocks 1402, 1406, and / or 1410, and / or fitness scores from block 1414 as inputs may be used to select the selected output to improve the current fitness. Such a machine learning algorithm may be trained using the characteristics, sensor data, and / or fitness scores associated with a training set of fitness ratings of users wearing a particular user interface. The training data may include information regarding adjustments made to the user interface, adjustments made to the user's face (e.g., shaving), and / or adjustments such as various selections for the user interface.

[0234] In one example, the user interface fit is evaluated by comparing sensor data before wearing the user interface and sensor data when wearing the user interface, and process 1400 includes only blocks 1402, 1404, 1406, 1412, 1414. In another example, the current fit is evaluated when the user is wearing the user interface, and process 1400 includes only blocks 1406, 1412, and 1414. In another example, the current fit of the user interface is evaluated by comparing facial characteristics of the user before and after wearing the user interface, and process 1400 includes only blocks 1402, 1404, 1408, 1410, 1412, and 1414. Other arrangements may be used. Additionally, in some cases, aspects presented as separate blocks may be incorporated into one or more other blocks. For example, in some cases, the generation of the fit score in block 1414 occurs as part of the generation of the output in block 1416. In another example, in some cases, process 1400 may proceed from block 1412 to block 1416 without generating a fitness score at block 1414.

[0235] 15 is a flow chart illustrating a process 1500 for evaluating the suitability of a user interface according to some embodiments of the disclosure. The user interface may be any suitable user interface, such as the user interface 100 of FIG. 1. The process 1500 may be performed using a control system of a processor for the smartphone 234 or a processor of a server, such as the server 210 of FIG.

[0236] In block 1402, sensor data is received from a user device, for example, related to a current engagement of a user interface. The sensor data received in block 702 may have been collected from one or more sensors (e.g., one or more sensors / transducers 4270 of RPT device 40 of FIGS. 4A-4C), one, more, or all of which may be incorporated into and / or coupled to the user device (e.g., smartphone 234 of FIG. 7). The sensor data may be collected at the same or different frequencies and at any suitable resolution. In some cases, the detection frequency may be based on the sensor used. For example, image data in the visible spectrum may be collected at a frame rate of 25-60 frames per second (fps) or more, while thermal data in IR frequencies may be collected at a frame rate of 10 fps or less.

[0237] In some cases, receiving the sensor data in block 1502 may include calibrating, adjusting, or stabilizing the sensor data in block 1504. Calibrating, adjusting, or stabilizing the sensor data may include adjusting the sensor data to account for undesirable artifacts in the data. Calibrating, adjusting, or stabilizing the sensor data may include adjusting using a portion of the sensor data. In one example, the image data may be stabilized in block 1502, for example, by applying image stabilization software to the image data or by applying inertial data obtained from an inertial measurement unit (IMU) or a sub-sensor of a smartphone. In some cases, stabilizing the sensor data may include receiving image stabilization information associated with a first portion of the sensor data (e.g., image stabilization information associated with image data from a visible spectrum camera) and applying the stabilization information to a second portion of the sensor data (e.g., sensor data from another sensor, such as thermal data from an IR sensor). In one example, image stabilization information associated with stabilizing the visible spectrum camera data (e.g., obtained by image stabilization software and / or inertial data) may be applied to the thermal data from the IR sensor, thereby stabilizing the thermal data beyond the extent possible when using the thermal data alone.

[0238] Calibration can be performed by acquiring sensor data of a known or nominal object or event. For example, calibration of exposure and / or color temperature in image data can be achieved by first collecting reference image data from a calibration surface (e.g., a surface of known color such as a white balance card) and then using the reference image data to adjust settings on the sensor acquiring the image data and / or to adjust the image data itself.

[0239] Adjustments may be made by using some of the sensor data to inform adjustments made to other sensor data. For example, a thermal imager (e.g., an IR sensor) may collect thermal data to generate a thermal map of the user's face. The thermal map may identify local temperatures at various locations on the user's face and some surfaces of the surrounding environment, such as a surface behind the user. In such an example, the thermal data may indicate that the surface behind the user is 19° C., but ambient temperature data collected from another temperature sensor may indicate that the surface behind the user and / or the surrounding room temperature is closer to 21° C. Thus, the system may automatically adjust the thermal data obtained from the thermal imager such that the temperature sensed for the surface behind the user is equal to the temperature measured by the other temperature sensor (e.g., 21° C.). Such adjustments may thus continue to the local temperature values ​​at various locations on the user's face.

[0240] In another example, sensor data from one or more sensors may be used to correlate and align with sensor data from one or more other sensors. For example, image data from a visible spectrum camera may be correlated with thermal mapping data from an IR sensor. In such an example, some detected features in the image data (e.g., eyes, ears, nose, mouth, vents in a user interface, etc.) may be compared to similarly detected features in the thermal mapping data. Thus, the data may be correlated even if the image data and the thermal mapping data were not collected at the same scale and field of view. For example, every pixel of the thermal mapping data may be adjusted (e.g., stretched in the X and / or Y directions) so that the location of a detected feature in the image data matches the location of a correlated feature in the thermal mapping data.

[0241] In some cases, adjusting the sensor data may include adjusting the sensor data using portions of the sensor data related to i) movement of the user device, ii) user device specific noise, iii) user breathing noise, iv) user speech (or other vocal) noise, v) changes in ambient lighting, vi) temporary shadows projected on the detected user's face, vii) temporary colored light projected on the detected user's face, or viii) any combination of i-vii.

[0242] Other techniques can be used to calibrate, tune, stabilize, or otherwise correct the sensor data based in part on the sensor data and / or other data (e.g., historical sensor data, sensor data obtained from sensors not coupled to the system, etc.).

[0243] In some cases, sensory feedback may be provided at block 1506 as part of receiving the sensor data at block 1502. Providing sensory feedback may include presenting feedback to the user regarding the process of receiving the sensor data at block 1502. In a first example, the sensory feedback may include text or images on a display showing instructions regarding the ongoing sensor data collection. For example, the sensory feedback may take the form of an outline or other marking designating an area of ​​the screen superimposed over a live camera feed of a camera facing the user, instructing the user to keep their face within the outline or area of ​​the screen.

[0244] In another example, receiving sensor data at block 1502 may continue for a set duration or until any sensor data is successfully acquired. In such a case, the user may be required to hold the user device in a particular orientation. Once the duration has elapsed or any sensor data has been acquired, the user device may present sensory feedback 1506 to indicate that the user may stop holding the user device in the particular orientation (e.g., the display of the user device is facing away from the user). The sensory feedback may be in the form of visual and / or non-visual cues (e.g., stimuli such as hints, commands, notifications, etc.). In some cases, such as when the display of the user device is currently not visible to the user, it may be particularly useful to use non-visual cues (e.g., audio cues or haptic feedback).

[0245] In some cases, providing sensory feedback in block 1506 may include presenting a command to the user to perform some action to evoke an effect that may be detected by one or more sensors. For example, the user may be instructed to hold their breath for a period of time, smile at the camera, chew, yawn, speak, put on or take off the user interface, etc. The detectable effect may be used to detect the characteristic in block 1514. In one example, the detectable effect may be the amount of movement of the user interface relative to the user's face when the user speaks, chews, or yawns. In such an example, excessive movement of the user interface may indicate a poor fit. In some cases, receiving sensor data in block 1502 may include receiving a completion signal associated with the user completing the action instructed in block 1506. Examples of completion signal detection include sensing a button press (e.g., a user pressing a "Done" button) and automatically detecting the completion of the action (e.g., via camera data).

[0246] In some cases, providing sensory feedback in block 1506 may include providing a command to the user to move to a desired position (e.g., move indoors or to a room with sufficient light), adjust the environment (e.g., turn lights on or off in the room), and / or adjust an orientation or posture (e.g., sit up straight).

[0247] In some cases, receiving the sensor data in block 1502 may include controlling the respiratory system in block 1508. Controlling the respiratory therapy system in block 1508 may include sending a control signal to a respiratory therapy device coupled to the user interface. When the control signal is received by the respiratory therapy device, the respiratory therapy device may adjust a parameter or take some action. For example, the control signal may cause the respiratory therapy device to turn on and / or off, to deliver air at a predetermined pressure or preset pressure pattern, to start and / or stop a heater and / or humidifier, and / or to take other appropriate action.

[0248] At block 1510, a facial mapping may be generated using the sensor data. The facial mapping may be generated using any suitable sensor data, such as ranging data (e.g., from a LiDAR or IR sensor), image data (e.g., from a camera), and / or thermal data (e.g., from an IR sensor). The facial mapping may identify one or more features of the user's face, such as the eyes, nose, mouth, ears, irises, etc. The facial mapping may include measuring the distance between any combination of the identified features. The resulting facial mapping may be a two-dimensional or three-dimensional facial mapping. Alternatively, the facial mapping and resulting data may be obtained from stored data collected from a facial scan performed as described above with reference to FIGS. 8A-8C.

[0249] In some cases, the facial mapping is a contour map that indicates contours and elevations associated with the user's face. In some cases, the facial mapping is a heat map that indicates local temperatures at various locations on the user's face.

[0250] In some cases, generating the facial mapping in block 1510 may include identifying a first individual and one or more additional individuals. In this case, generating the facial mapping may include selecting the first individual (e.g., based on the first individual's proximity to one or more sensors or user devices, based on a comparison with previously recorded images or characteristics of the first user, or other such analysis), and continuing the facial mapping generation using one or more features detected on the first individual's face.

[0251] In some cases, in optional block 1512, the sensor data may be used to generate a user interface mapping. Any suitable sensor data, such as ranging data (e.g., from a LiDAR sensor or an IR sensor), image data (e.g., from a camera), and / or thermal data (e.g., from an IR sensor), may be used to generate the user interface mapping. The user interface mapping may identify one or more features of the user interface, such as vents, contours, conduits, conduit connections, straps, etc. The user interface mapping may include measuring the distance between any combination of the identified features. The resulting user interface map may be a two-dimensional or three-dimensional user interface mapping.

[0252] In some cases, generating a facial mapping at block 1510 may include generating a user interface mapping at block 1512 .

[0253] At block 1514, the sensor data from block 1502 and the face mapping from block 1510 are used to identify one or more characteristics associated with the current fit. In some cases, identifying the characteristics at block 1514 may include using the interface mapping generated at block 1512.

[0254] Any suitable characteristic may be identified. The characteristics may include characteristics of the user's face, characteristics of the interaction between the user's face and the user interface, characteristics of the user interface, and characteristics of the environment. In some cases, the characteristics may be discernible aspects of the user's face, the user interface, the interaction between the user's face and the user interface, or the environment detected from the sensor data and having probative value for determining the quality of the fit of the user interface.

[0255] In some cases, but not necessarily, the feature may be associated with a location. A location-related characteristic may be associated with a location on the facial mapping and / or user interface mapping or a location that references the facial mapping and / or user interface mapping. For example, an exemplary characteristic may be an audio signal indicative of an unintentional air leak (e.g., an audio signal having a characteristic frequency distribution associated with an unintentional air leak), or an unintentional air leak itself. In some cases, the characteristic may be used alone (e.g., the presence or absence of an unintentional air leak may be used, such as to generate a fit score) or in combination with location information associated with the characteristic. In such an example, an unintentional air leak may be detected at a particular location on the user's face (e.g., between the nose and the left cheek, as shown in FIG. 12). In this case, the location information may indicate a location on the facial mapping or user interface map that correlates to the location on the user's face where the unintentional air leak was. Adding location information along with the user interface helps provide useful information, such as a more accurate fit score and / or more precise instructions to improve the fit.

[0256] In some cases, the set of available characteristics may be pre-determined. In such cases, identifying the characteristics in block 1514 may include analyzing the sensor data and / or face mapping to determine which of the set of available characteristics, if any, have been detected. For example, the set of available characteristics may include a local temperature rebound of the user's face, a local color rebound of the user's face, and a local temperature of the user interface. In such an example, if the sensor data does not include thermal data (e.g., if an IR sensor or a temperature sensor is not available), the only characteristic that may be determined from the list is a local color rebound that may be detected from image data collected by the camera. In another example, if sufficient thermal data is present, a local temperature rebound and a local temperature of the user interface may be additionally detected.

[0257] In some cases, a characteristic may be a beneficial characteristic or a detrimental characteristic. A beneficial characteristic may be a characteristic associated with a good fit of the user interface. For example, a characteristic such as a local contour on the user's face in the form of a seal of the user interface may indicate that the user interface maintains a good fit with the user's face. Thus, the presence or more of such a characteristic may be beneficial. On the other hand, a detrimental characteristic may be associated with a poor fit of the user interface. For example, a characteristic such as a local temperature change in the area of ​​the user's face may indicate that cold air is flowing through that area of ​​the user's face, thereby indicating an unintentional air leak associated with a poor fit. Thus, the presence or more of such a characteristic may be detrimental.

[0258] An exemplary characteristic is the local temperature of the user's face. This characteristic may be a temperature detected on the user's face, which may be compared to a previously detected temperature at the same location (e.g., a previous time the user wore the user interface) or a current temperature detected at an adjacent location. For example, if a cold spot is detected near a seal on the user interface, but the cold spot is surrounded by higher temperatures, the cold spot may indicate an unintentional air leak.

[0259] Another exemplary characteristic is local temperature rebound of the user's face. Local temperature rebound may include a change in temperature over a period of time at a particular location (e.g., a particular location on the facial mapping). Local temperature rebound may follow a transient event, such as a user interface transient event (e.g., removal of the user interface), a user transient event (e.g., moving from a warm room to a cold room), a respiratory therapy system transient event (e.g., activating a respiratory therapy system heater, such as a flow generator heater, a humidifier heater, and / or a conduit heater), or other transient event. For example, a longer temperature rebound time detected at a particular location relative to the facial mapping after the user interface is removed may indicate a poor fit. A "rebound" of a measured attribute (e.g., temperature, color, contour, etc.) may include a change in the measured attribute following a transient event, such as removal or adjustment of the user interface, starting or stopping pressurized airflow from the RPT device, or the occurrence of another event that causes the measured attribute to change over time.

[0260] Another exemplary characteristic is the local color of the user's face. This characteristic may be a color detected on the user's face, which may be compared to a color previously detected at the same location (e.g., a previous time the user wore the user interface) or a current color detected at an adjacent location. For example, if a redder spot is detected near a seal on the user interface, but surrounded by a whiter color, this redder spot may indicate an unintentional air leak (e.g., due to the seal being less pressed against the user's face and / or due to irritation from flowing air).

[0261] Another exemplary characteristic is local color rebound on the user's face. Local color rebound may include a change in color over a period of time at a particular location (e.g., a particular location on the face mapping). Local color rebound can follow a transient event, such as a user interface transient event (e.g., removal of the user interface), a user transient event (e.g., moving from a warm room to a cold room), a respiratory therapy system transient event (e.g., a respiratory therapy device being turned on or off), or other transient event. For example, a longer color rebound time detected at a particular location relative to the face mapping after the user interface is removed can indicate a good fit.

[0262] Another exemplary characteristic is the local contour of the user's face. This characteristic may be a contour detected on the user's face, which may be compared to a contour previously detected at the same location (e.g., a previous time the user wore the user interface) or a current contour detected at an adjacent location. For example, if a certain amount of indentation is detected in the user's skin around the entire seal of the user interface, this may indicate a strong seal and good fit.

[0263] Another exemplary characteristic is local contour rebound of the user's face. Local contour rebound may include a change in contour over a period of time at a particular location (e.g., a particular location on the facial mapping). Local contour rebound may follow a temporal event, such as a user interface temporal event (e.g., removal of the user interface), a user temporal event (e.g., moving from a warm room to a cold room), a respiratory therapy system temporal event (e.g., a respiratory therapy device being turned on or off), or other temporal event. For example, a longer contour rebound time detected at a particular location relative to the facial mapping after the user interface is removed may indicate a good fit.

[0264] Another exemplary characteristic is the local contour of the user interface. This characteristic may be a contour detected on the user interface (e.g., on a portion of the user interface, such as a seal on the user interface) and may be compared to a contour previously detected at the same location (e.g., the previous time the user interface was worn) or a current contour detected at another location on the user interface. For example, if a certain amount of indentation is detected on a seal on the user interface, this may indicate a strong seal and good fit.

[0265] Another exemplary characteristic is local rebound of the user interface. Local contour rebound may include a change in contour over time at a particular location on the user interface (e.g., a particular location on a user interface map). Local contour rebound may follow a transient event, such as a transient event of the user interface (e.g., removal of the user interface), a transient event of the user (e.g., moving from a warm room to a cold room), a transient event of the respiratory therapy system (e.g., turning a respiratory therapy device on or off), or other transient event. For example, a long contour rebound time detected at a particular location on a seal of the user interface may indicate a poor fit and / or seal failure (e.g., a seal failure may not rebound quickly).

[0266] Another exemplary characteristic is the local temperature of the user interface. This characteristic may be a temperature detected at a particular location on the user interface (e.g., on a portion of the user interface, such as a seal), and may be compared to a temperature previously detected at the same location (e.g., the previous time the user interface was worn), or a current temperature detected at another location on the user interface. For example, if a cold spot is detected at a location on a seal of the user interface, but other portions of the user interface are detected as being relatively hot, the cold spot may indicate an unintentional air leak. In another example, the length of time required for a portion of the user interface or conduit to heat to a predetermined temperature may indicate a good or bad fit.

[0267] Another example characteristic is a vertical position of the user interface relative to one or more features of the user's face. The vertical position of the user interface relative to one or more features of the user's face may be based on facial mapping, sensor data, and / or user interface mapping. In one example, if the user interface or a detected feature of the user interface is detected to be too high or too low relative to one or more features of the user's face (e.g., the chin, mouth, and / or nose), this may indicate a poor fit (e.g., a strap may pull the user interface up too much or down too much).

[0268] Another example characteristic is a horizontal position of the user interface relative to one or more features of the user's face. The horizontal position of the user interface relative to one or more features of the user's face may be based on facial mapping, sensor data, and / or user interface mapping. In one example, if the user interface or a detected feature of the user interface is detected to be too far from one side or the other of the user's face, this may indicate a poor fit (e.g., a strap may pull the user interface too far to the right or left side of the user).

[0269] Another example characteristic is a rotational orientation of the user interface relative to one or more features of the user's face. The rotational orientation may be a measure of how much the user interface rotates relative to the user's face around an axis of rotation extending outward from the user's face, e.g., an axis of rotation extending anteriorly or ventrally from the face. The rotational orientation of the user interface relative to one or more features of the user's face may be based on facial mapping, sensor data, and / or user interface mapping. In one example, if the degree to which the user interface or a detected feature of the user interface is excessively rotated relative to one or more features of the user's face is detected, this may indicate a poor fit (e.g., a strap may distort the user interface on the user's face).

[0270] Another exemplary characteristic is the distance between an identified feature of the user interface to one or more features of the user's face. This distance may be based on facial mapping, sensor data, and / or user interface mapping. In one example, if a detected feature of the user interface (e.g., a vent) is detected to be too misaligned from a detected feature of the user's face (e.g., the bridge of the nose), this may indicate a poor fit. This distance may be measured in one, two, or three dimensions. In one example, a user interface that fits too loosely to the user's face may have vents located at a relatively large distance from the surface of the user's face, which may indicate a poor fit. In such a case, a measurement of the distance between the vents and the user's facial features may indicate a poor fit, which may be corrected by tightening a strap of the user interface.

[0271] In another example, identifying the leak characteristic in block 1514 may include determining a breathing pattern of the user based on the sensor data received from block 1502, determining a thermal pattern associated with the user's face based on the sensor data received from block 1502 and the facial mapping from block 1510, and using the breathing pattern and thermal pattern to generate a leak characteristic signal indicative of a balance between intentional airflow leaks and unintentional seal leaks. For example, the breathing pattern and thermal pattern may be used to generate an intentional airflow leak signal indicative of instances and any strength of intentional airflow leaks. A similar unintentional seal leak signal may be generated for unintentional seal leaks. The leak characteristic signal may be a ratio of the unintentional seal leak signal to the intentional seal leak signal.

[0272] In some cases, identifying the characteristic in block 1514 may include identifying and confirming the potential characteristic in block 1516. Identifying and confirming the potential characteristic may include identifying the characteristic using a first sensor data set and then confirming the characteristic using a second sensor data set. The first sensor data set and the second sensor data set may be collected from the same sensor at different times or from different sensors. For example, collecting from the same sensor at different times may include identifying a potential unintentional air leak from thermal data while the user is wearing the user interface (e.g., detecting a localized temperature change of the user's facial surface in an area surrounding the user interface) and confirming the unintentional air leak using thermal data collected after the user removes the user interface (e.g., detecting an absence of change in the user's facial surface temperature in a particular area for a time that is not near the potential air leak, such as during a time when blood is backing up into previously compressed tissue at another location).

[0273] In another example, collecting sensor data from various sensors to identify and confirm potential characteristics may include identifying a potential unintentional air leak using thermal data collected while a user is wearing the user interface, and then confirming the unintentional air leak using simultaneously collected audio data (e.g., by detecting a characteristic audio or acoustic signal indicative of the unintentional air leak, corresponding to the same time that the potential unintentional air leak was detected in the thermal image).

[0274] In some cases, additional sensor data may be received from the respiratory device in block 1520 and used for identification of the characteristic in block 1514. The additional sensor data from the respiratory therapy device may include data obtained from a respiratory therapy system (e.g., the respiratory therapy system of FIG. 1). Such additional sensor data may include data such as flow rate, pressure, delivery air temperature, ambient temperature, delivery air humidity, ambient humidity, ambient audio signals, audio signals transmitted through the conduit or air in the conduit, etc. In some cases, the additional sensor data may include data collected by sensors in the user interface and / or the conduit. In some cases, the additional sensor data from block 1520 may be related to air delivery by the respiratory therapy system. For example, in block 1514, the additional sensor data related to the delivery air temperature may be used to identify and / or confirm unintentional air leaks by comparing the delivery air temperature to the temperature of the user's skin around the seal of the user interface.

[0275] However, in some cases, the additional sensor data from block 1520 may be related to the current fit of the user interface. In one example, the respiratory therapy system may include one or more sensors capable of detecting information about the user's face or the user interface. For example, an RF sensor in the respiratory therapy device may detect ranging information related to the current fit of the user interface to the user. In another example, one or more optical sensors (e.g., a visible light camera, a passive infrared sensor, and / or an active infrared sensor) may be used to detect information about the user's face and / or the user interface.

[0276] In some cases, historical data may be received in block 1522 and used in block 1514 for identifying the characteristic. The historical data may include historical sensor data (e.g., sensor data collected or received in a previous instance of block 1502), historical characteristic data (e.g., information about the characteristic identified in a previous instance of block 1514), and / or other historical data (e.g., previous fit scores). In block 1518, identifying the characteristic in block 1514 may include comparing the historical data received in block 1522 with the sensor data received in block 1502. For example, receiving the historical data in block 1522 may include accessing a memory including a thermal mapping of the user's face prior to donning the user interface. In such an example, in block 1518, that thermal mapping may be compared to a current thermal mapping of the user's face, which may be performed while the user is donning the user interface and / or after the user has removed the user interface. A difference detected in the two thermal mappings may be used to identify the characteristic (e.g., unexpected localized temperature and / or unintentional air leakage).

[0277] In another example, receiving the historical data at block 1522 may include accessing a memory that includes a previous contour rebound time associated with a particular user interface seal. In such an example, the previous contour rebound time may be compared to a newly obtained contour rebound time for the same portion of the user interface seal. This comparison may indicate that the seal is deteriorating over time, which may cause a poor fit and, inevitably, a seal failure. Thus, this degradation may be detected as a characteristic and a user may be notified to replace the seal.

[0278] In some cases, machine learning algorithms can be used to facilitate or achieve the identification of the characteristics. Such machine learning algorithms can be trained using a training set of sensor data collected by one or more users wearing the user interface or participating in a temporal event of the user interface. The training data can include information about the characteristics determined to be present, although this is not always the case. In some cases, the training data can include information about the fit quality of the user interface. The quality of such fit information can be based on a subjective assessment of the user, objective values ​​collected using other equipment (e.g., equipment such as laboratory sensors or user interfaces with dedicated sensors and / or dedicated sensing equipment), etc. In some cases, the characteristics identified in block 1514 are the features used by the machine learning algorithm.

[0279] In some cases, identifying characteristics related to the current fit may include determining a predicted fit quality for a given user face (e.g., the facial mapping from block 1510) and a given user interface (e.g., the user interface mapping from block 1512 or other user interface identification information). In such cases, the facial mapping of the user's face may be applied to design parameters of the given user interface to determine a predicted fit quality (e.g., the most likely fit of the particular user interface to the user's face). If that predicted fit quality falls below a threshold, it may be determined that the given user interface is not suitable for use by the user. The predicted fit quality may also be used to determine whether a given assessment of the current fit (e.g., the assessment described with reference to block 1528) may be improved (e.g., whether an improvement of the current fit may be achieved or possible). For example, if the predicted fit quality of a given user interface on a particular user's face is "good" (e.g., from "poor," "good," "good," "very good," although other measures can be used to represent the quality of fit) and the current fit is rated as "good," it may be determined that no further improvement in the quality of the fit is possible without modifying the user interface. In another example, if the predicted fit quality is "very good" and the current fit is rated as "good," it may be determined that adjustment of other factors, such as trimming facial hair, removing makeup, changing position on the bed, or other changes, could improve the current fit.

[0280] At block 1524, output feedback may be generated based on the identified characteristic(s) from block 1514 and optionally the characteristic location (e.g., location relative to the face mapping and / or user interface mapping). Generating the output feedback at block 1524 may include presenting the output feedback via, for example, a GUI, a speaker, a haptic feedback device, etc. In some cases, the output feedback may be presented as an overlay on the user image, for example in the form of an augmented reality overlay. For example, icons, highlights, text, and other marks may be overlaid on the user's image (e.g., live or non-live) to provide feedback regarding the quality of the current fit and / or instructions on how to improve the current fit.

[0281] In some cases, the output feedback may include the identified characteristic or characteristics and optional location information. For example, the output feedback may be an image or representation of the user's face covered with graphics or text indicating the presence of the detected characteristic (e.g., obtained from a visible spectrum camera, a thermal imager, or graphically generated from facial mapping). In one example, a detected unintentional air leak may be displayed on the image or representation of the user's face with an arrow, highlighted circle, or other visual element calling attention to the location of the detected unintentional air leak.

[0282] In some cases, generating output feedback in block 1524 may include determining and presenting a suggested action to improve the fit in block 1526. Determining and presenting a suggested action to improve the fit may include using the identified characteristics and their location information to select an action designed to improve the fit and presenting the action (e.g., via text, graphics, audio feedback, haptic feedback, etc.). In some cases, the action to improve the fit may be identified based on a lookup table or algorithm that can select an action to take based on the identified characteristics, optionally including its location information. For example, detection of an unintentional air leak can be correlated with a suggested action to adjust a strap of a user interface, in which case the location of the unintentional air leak can be used to determine which strap to adjust.

[0283] However, in some cases, the determination of the action to be taken may be based on a machine learning algorithm. Such a machine learning algorithm may be trained using a training set of actions to be taken and the quality of fitness information before and / or after the action is taken. The quality of such fitness information may be based on a subjective assessment by a user, objective values ​​collected using other equipment (e.g., equipment such as laboratory sensors or user interfaces with dedicated sensors and / or dedicated sensing equipment), etc. In some cases, the characteristics identified in block 1514 are the features used by the machine learning algorithm.

[0284] In some cases, generating the output feedback in block 1524 may include generating a current goodness of fit assessment in block 1526. Generating the current goodness of fit assessment in block 1526 may include using the sensor data, the identified characteristics, and / or the characteristic location information. The assessment may be a numerical score (e.g., a goodness of fit score such as a numerical score between 0 and 100), a categorical score (e.g., a textual score such as "good", "average", "poor", a color score such as green, yellow, red, or a graphical score such as depicting a seal with no air leaks, depicting a small seal, depicting a seal with a large air leak, etc.).

[0285] In some cases, generating the assessment may include generating the assessment based on an equation algorithm using the sensor data, the identified characteristics, and / or the characteristic location information as inputs. For example, the goodness of fit score may be a weighted calculation of values ​​associated with various identifying characteristics (e.g., amount of temperature difference, distance of the temperature difference from the detected user interface seal edge, duration of the detected unintentional leak in the audio signal, etc.).

[0286] In some cases, the generated assessment may be based at least in part on a comparison to historical data (e.g., the historical data received in block 1522). In this case, the assessment may be based at least in part on whether a value associated with the identified characteristic indicates an improvement or a deterioration, where an improvement ensures that the fitness score is an increase from a previous fitness score and a deterioration ensures that the fitness score is a decrease from a previous fitness score.

[0287] In some cases, the generated assessment may be based on a machine learning algorithm. Such a machine learning algorithm may be trained using a training set of sensor data, identified characteristic(s), and / or characteristic location information. In some cases, the training set may include fit quality information. The quality of such fit information may be based on a user's subjective assessment, objective values ​​collected using other equipment (e.g., equipment such as laboratory sensors or user interfaces with dedicated sensors and / or dedicated sensing equipment), etc. In some cases, the characteristics identified in block 1514 are the features used by the machine learning algorithm.

[0288] Although process 1500 is shown and described herein as occurring in a particular order, more generally, the individual blocks of process 1500 may be performed in any suitable order and may have fewer and / or additional blocks, for example, blocks 1520, 1522, 1512, and 1518 may not be used in some cases.

[0289] The flow charts of FIGS. 9-10 and 14-15 are representative of exemplary machine-readable instructions for collecting and analyzing data for selecting an optimal interface for respiratory pressure therapy, which is used to perform subsequent analysis, such as matching the selected user interface. In this example, the machine-readable instructions include an algorithm executed by (a) a processor, (b) a controller, and / or (c) one or more other suitable processing device(s). The algorithm may be embodied in software stored on a tangible medium, such as a flash memory, a CD-ROM, a floppy disk, a hard drive, a digital video (versatile) disk (DVD), or other storage device. However, those skilled in the art will readily appreciate that the entire algorithm and / or portions thereof may alternatively be executed by devices other than a processor and / or embodied in firmware or dedicated hardware in a well-known manner (e.g., the entire algorithm and / or portions thereof may be implemented by an application specific integrated circuit (ASIC), a programmable logic device (PLD), a field programmable logic device (FPLD), a field programmable gate array (FPGA), discrete logic, etc.). For example, any or all of the components of the interface may be implemented by software, hardware, and / or firmware. Also, some or all of the machine-readable instructions represented by the flow charts may be implemented manually. Furthermore, an example of an algorithm is described with reference to the flow charts shown in Figures 19-10 and 14-15, but is described as follows. In fact, it will be readily apparent to those skilled in the art that many other ways of implementing exemplary machine-readable instructions may be used instead. For example, the order of execution of the blocks may be changed, and / or some of the described blocks may be modified, deleted, or combined.

[0290] As used herein, the terms "component," "module," "system," and the like generally refer to computer-related entities, hardware (e.g., circuitry), a combination of hardware and software, software, or an entity related to an operating machine having one or more specific functions. For example, a component may be, but is not limited to, a process running on a processor (e.g., a digital signal processor), a processor, an object, an executable file, a thread of execution, a program, and / or a computer. For example, an application running on a controller and a controller may be a component. One or more components may be present within an executing process and / or thread, and components may be localized on one computer and / or distributed among two or more computers. Furthermore, a "device" may be in the form of specially designed hardware; generalized hardware that is dedicated by the execution of software on the hardware to enable the hardware to perform a specific function; software stored on a computer-readable medium; or a combination thereof.

[0291] The terms used herein are used only for the purpose of describing particular embodiments and are not intended to limit the present invention. As used herein, the singular forms "a," "an," and "the" may be intended to include the plural forms unless the context clearly indicates otherwise. Furthermore, to the extent the terms "including," "includes," "having," "has," "with," or variations thereof are used in either the Detailed Description and / or the Claims, such terms are intended to be as inclusive as the term "comprising."

[0292] Unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as those of ordinary skill in the art would normally understand. Furthermore, terms, for example, terms defined in common dictionaries, should be interpreted as having a meaning consistent with their meaning in the context of the relevant field, and should not be interpreted as idealized or overly formal unless expressly defined herein.

[0293] One or more steps, aspects or steps, or any portion thereof, from any one or more of claims 1-100 below may be combined with one or more steps, aspects or steps, or any portion thereof, from any one or more of other claims 1-100 to form one or more additional implementations and / or claims of the present disclosure.

[0294] Although various embodiments of the present invention have been described above, it should be understood that they are presented by way of example only, not by way of limitation. Although the present invention has been illustrated and described with respect to one or more implementations, those skilled in the art will recognize equivalent changes and modifications upon reading and understanding this specification and the accompanying drawings. In addition, while a particular feature of the present invention may be disclosed with respect to only one of several implementations, such feature may be combined with one or more other features of other implementations, which may be desirable and advantageous for any given application or particular application. Thus, the breadth and scope of the present invention should not be limited by any of the above-described embodiments. Rather, the scope of the present invention should be defined based on the following claims and their equivalents.

Claims

1. 1. A system for selecting an interface for respiratory therapy suited to a user's face, comprising: a storage device for storing a facial image of the user; a facial contour engine operable to determine facial features of the user based on the facial image; A plurality of facial features from a user population and a plurality of corresponding interfaces used by the user population; and storing operational data of respiratory therapy devices used by said user population having said plurality of corresponding interfaces; One or more databases; a selection engine coupled to the one or more databases, the selection engine operable to select an interface for a user from among the plurality of corresponding interfaces based on a desired outcome from the stored operational data and the determined facial characteristics.

2. The system of claim 1, wherein the interface is a mask and the respiratory therapy device is configured to provide one or more of positive airway pressure (PAP) or non-invasive ventilation (NIV).

3. 3. The system of claim 1 or claim 2, wherein at least one of the respiratory therapy devices includes an audio sensor for collecting audio data during operation of the at least one of the respiratory therapy devices, and the selection engine is operable to analyze the audio data to determine the type of the corresponding interface of the at least one of the respiratory therapy devices based on matching the audio data with acoustic signatures of known interfaces.

4. The system of any one of claims 1 to 3, wherein the selection engine selects the interface based on user demographic data compared with demographic data of a user population stored in the one or more databases.

5. The system of any one of claims 1 to 4, wherein the operational data includes one of flow rate, motor speed, and treatment pressure.

6. The system of any one of claims 1 to 5, wherein the facial image is determined based on a scan from a mobile device including a camera.

7. 7. The system of claim 6, wherein the mobile device includes a depth sensor, the camera is a 3D camera, and the facial features are three-dimensional features derived from a mesh surface derived from the facial image.

8. The system of any one of claims 1 to 7, wherein the desired outcome is a seal between an interface and a face to prevent leakage or compliance with use of the respiratory therapy device.

9. The system of any one of claims 1 to 8, further comprising a mobile device operable to collect subjective data input from the user, and wherein the selection of the interface is based in part on the subjective data.

10. The system of any one of claims 1 to 9, further comprising a machine learning module operable to determine a type of behavioral data that correlates to an interface that achieves the desired outcome.

11. The system of any one of claims 1 to 10, wherein the selection engine is further operable to receive feedback from the user based on operation of the selected interface and based on undesirable outcomes, and to select another interface based on the desired outcomes, the undesirable outcomes being one of: low user compliance with therapy, high leakage, or unsatisfactory subjective outcome data.

12. 1. A method for selecting an appropriate respiratory therapy interface for a user's face, comprising: storing a facial image of the user in a storage device; determining facial characteristics based on the facial image; storing a plurality of facial features and a plurality of corresponding interfaces from a population of users in one or more databases; storing operational data of the respiratory therapy devices used by the user population along with the plurality of corresponding interfaces in one or more databases; selecting an interface for a user from among the plurality of corresponding interfaces based on a desired outcome from the stored motion data and the determined facial characteristics.

13. 1. A method comprising: receiving sensor data comprising an image of a user's face, the sensor data relating to a current fit of an interface to the user's face, the sensor data being collected by one or more sensors of a mobile device including an image sensor; using the sensor data to generate a facial mapping indicative of one or more features of the user's face; using the sensor data and the facial mapping to identify a first characteristic associated with the current fit of an interface to the face of the user, the characteristic indicating a quality of the current fit, the first characteristic associated with a distinctive location on the facial mapping; generating output feedback that adjusts a fitness of the interface based on the identified first characteristic and the characteristic location.

14. The step of generating output feedback comprises: determining a proposed action that, if implemented, affects the first characteristic to improve the current fitness; and presenting the proposed action using an electronic interface of the mobile device.

15. The method of claim 13 or claim 14, wherein the sensor data includes distance data, the distance data indicating one or more distances between one or more sensors of the mobile device and a face of the user.

16. 16. The method of claim 15, wherein the one or more sensors include: i) a proximity sensor; ii) an infrared-based dot matrix sensor; iii) a LiDAR sensor; iv) a MEMS micromirror projector-based sensor; or v) any combination of i-iv.

17. The first characteristic is i) the local temperature of the user's face; ii) local temperature changes on the user's face; iii) the local color of the user's face; iv) local color changes on the user's face; v) the local contours of the user's face; vi) local contour changes of the user's face; vii) the local contour of the interface; viii) the local temperature of the interface; ix) a change in the local temperature of the interface, or The method of any one of claims 13 to 16, comprising any combination of x) i to ix.

18. The characteristic position is i) the vertical position of an interface relative to one or more features of the user's face; ii) a horizontal position of an interface relative to one or more features of the user's face; iii) a rotational orientation of the interface relative to one or more features of the user's face; iv) the distance between an identified feature of the interface to one or more features of the user's face; or The method of any one of claims 13 to 17, comprising any combination of v) i to iv.

19. A method according to any one of claims 13 to 18, further comprising using the sensor data to generate an interface map indicating a relative position of one or more features of the interface with respect to the facial mapping, and wherein identifying the first characteristic comprises using the interface map.

20. using the sensor data and the facial mapping to identify a second characteristic associated with a future potential failure of the interface; 20. The method of claim 13, further comprising: generating output feedback based on the identified second characteristic that can be used to reduce the likelihood of a potential future fault occurring or delay the occurrence of a potential future fault.

21. 21. The method of claim 13, further comprising, prior to receiving the sensor data, accessing historical sensor data relating to one or more historical fits of an interface on the user's face, and wherein identifying the first characteristic further comprises accessing the historical sensor data.

22. 22. The method of any one of claims 13 to 21, further comprising generating a current suitability score using the sensor data and the face mapping, and wherein the output feedback is generated to improve a subsequent suitability score.

23. 23. The method of any one of claims 13 to 22, wherein receiving the sensor data comprises receiving audio data from the one or more audio sensors, and identifying the first characteristic comprises identifying an unintentional leak using the audio data.

24. The one or more sensors include a camera, an audio sensor, and a thermal sensor, the sensor data includes camera data, audio data, and thermal data, and identifying the first characteristic includes: identifying potential characteristics using at least one of the camera data, the audio data, and the thermal data; and using at least one other data of the camera data, the audio data, and the thermal data to ascertain potential characteristics.

25. presenting user instructions indicating an action to be performed by the user; 25. The method of claim 13, further comprising: receiving a completion signal indicating that the user has performed an action, wherein a first portion of the sensor data is collected before receiving the completion signal and a second portion of the sensor data is collected after receiving the completion signal; and wherein identifying the characteristic comprises comparing the first portion of the sensor data with the second portion of the sensor data.

26. The method of any one of claims 13 to 25, wherein the output feedback is useful for improving a current fitness.

27. using the sensor data to generate an initial score based on the current fitness; receiving subsequent sensor data related to a subsequent fit of the interface to the user's face, the subsequent fit being based on a current fit after performing the output feedback; using the subsequent sensor data to generate a subsequent score based on a subsequent goodness of fit; 27. The method of claim 26, further comprising: evaluating the subsequent scores as indicating an improvement in quality over the initial scores.

28. The step of identifying a first characteristic associated with the current fitness includes: determining a breathing pattern of the user based on the received sensor data; determining a heat pattern associated with the user's face based on the received sensor data and the facial mapping; and using the breathing pattern and the thermal pattern to determine a leakage characteristic indicative of a balance between intentional vent leakage and unintentional seal leakage.

29. The method of any one of claims 14 to 28, further comprising the step of receiving additional sensor data from one or more additional sensors of the respiratory device, and wherein identifying the characteristic comprises using the additional sensor data.

30. 30. The method of any one of claims 14 to 29, further comprising the step of transmitting a control signal which, when received by the respiratory device, causes the respiratory device to operate using a defined set of parameters, wherein a first portion of the sensor data is collected while the respiratory device is operating using a defined set of parameters, a second portion of the sensor data is collected while the respiratory device is not operating using a defined set of parameters, and the second portion of the sensor data is collected while the respiratory device is operating using a defined set of parameters, and wherein identifying the characteristic comprises comparing the first portion of the sensor data with the second portion of the sensor data.

Images