Mask sizing tool using mobile application

Abstract

Devices and methods for automatically selecting one or more patient interfaces according to the size of the patient interface (s), such as with processing in one or more processors or one or more servers. The captured image data may be received by an image sensor. The captured image data may contain one or more facial features of an intended user of the patient interface. These facial features may be captured in association with predetermined reference features of one or more known dimensions. One or more facial features and reference features of the user may be detected in the captured image data. Image pixel data of an image may be processed to measure an aspect spacing of the detected facial feature (s) based on a reference feature. A patient interface size is selected from the standard patient interface sizes based on a comparison of the measured aspect spacing of the facial feature (s) and the data record with the measured aspect spacing of the facial feature (s) of size information regarding the standard patient interface sizes.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the benefit of U.S. Provisional Application No. 62 / 186,461, filed June 30, 2015, and U.S. Provisional Application No. 62 / 313,202, filed March 25, 2016, the entire disclosures of which are incorporated herein by reference.

[0003] STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH OR DEVELOPMENT

[0004] not applicable

[0005] Names of Joint Research and Development Participants

[0006] not applicable

[0007] Sequence Listing

[0008] not applicable Technical Field

[0009] The present technology relates to one or more of the detection, diagnosis, treatment, prevention and improvement of respiratory-related diseases. The present technology also relates to medical devices or equipment and their uses. Background Art

[0010] Related technical notes

[0011] Human respiratory system and its diseases

[0012] The body's respiratory system facilitates gas exchange. The nose and mouth form the entrance to a patient's airway.

[0013] The respiratory tract includes a series of branch tubes, which become narrower, shorter, and more numerous as they penetrate deep into the lungs. The main function of the lungs is gas exchange, thereby allowing oxygen to move from the air into the venous blood and remove carbon dioxide. The trachea is divided into the left main bronchus and the right main bronchus, and is eventually further divided into the terminal bronchi. The bronchi constitute the conducting airways and do not participate in gas exchange. Further division of the respiratory tract leads to the respiratory bronchi, and eventually to the alveoli. The alveolar region of the lungs is where gas exchange occurs and is called the respiratory zone. Referring to "Respiratory Physiology" by John B. West, Lippincott Williams and Wilkins, the 9th edition published in 2011.

[0014] There is a range of respiratory diseases. Some diseases can be characterized by specific conditions, such as apnea, hypopnea, and hyperpnea.

[0015] Obstructive sleep apnea (OSA) is a form of sleep-disordered breathing (SDB) characterized by conditions that include occlusion or obstruction of the upper airway during sleep. This is caused by a combination of an abnormally small upper airway and normal loss of muscle tone in the area of ​​the tongue, soft palate, and posterior oropharyngeal wall during sleep. The condition causes affected patients to stop breathing for periods of time that typically last 30 to 120 seconds, sometimes 200 to 300 times per night. It often leads to excessive daytime sleepiness and can cause cardiovascular disease and brain damage. The syndrome is a common disease, especially in middle-aged overweight men, although those affected may not be aware of the problem. See U.S. Patent No. 4,944,310 (Sullivan).

[0016] Cheyne-Stokes respiration (CSR) is another form of sleep-disordered breathing. CSR is a disease of the patient's respiratory controller in which there are rhythmic alternating cycles of increased and decreased ventilation called CSR cycles. CSR is characterized by repeated deoxygenation and reoxygenation of arterial blood. Due to repeated hypoxia, CSR may be harmful. In some patients, CSR is associated with repeated awakenings from sleep, which lead to severe sleep disruption, increased sympathetic activity, and increased afterload. See U.S. Patent No. 6,532,959 (Berthon-Jones).

[0017] Obesity hyperventilation syndrome (OHS) is defined as the combination of severe obesity and awake chronic hypercapnia in the absence of other known causes of hypoventilation. Symptoms include dyspnea, morning headaches, and excessive daytime sleepiness.

[0018] Chronic obstructive pulmonary disease (COPD) includes any of a group of lower respiratory tract diseases that have certain common features. These include increased resistance to air movement, prolonged expiratory phase of breathing, and loss of normal elasticity of the lungs. Examples of COPD are emphysema and chronic bronchitis. COPD is caused by chronic smoking (the main risk factor), occupational exposure, air pollution, and genetic factors. Symptoms include: dyspnea on exertion, chronic cough, and sputum production.

[0019] Neuromuscular disease (NMD) is a broad term that covers many diseases and disorders that impair muscle function directly via intrinsic muscle pathology or indirectly via neuropathology. Some NMD patients are characterized by progressive muscle damage, resulting in loss of mobility, wheelchair confinement, dysphagia, respiratory muscle weakness, and ultimately death from respiratory failure. Neuromuscular diseases can be divided into rapidly progressive and slowly progressive: (i) rapidly progressive disease: characterized by muscle damage that worsens within a few months and leads to death within a few years (e.g., amyotrophic lateral sclerosis (ALS) and juvenile Duchenne muscular dystrophy (DMD)); (ii) variable or chronic progressive disease: characterized by muscle damage that worsens within a few years and only slightly reduces life expectancy (e.g., limb girdle, facioscapulohumeral and myotonic dystrophy). Symptoms of NMD respiratory failure include: increased general weakness, dysphagia, dyspnea during exercise and rest, fatigue, drowsiness, morning headache, difficulty concentrating, and difficulty with mood changes.

[0020] Chest wall disease is a group of thoracic deformities that result in inefficient coupling between the respiratory muscles and the thorax. These diseases are often characterized by restrictive defects and share the potential for prolonged hypercapnic respiratory failure. Scoliosis and / or kyphoscoliosis may lead to severe respiratory failure. Symptoms of respiratory failure include: dyspnea on exertion, peripheral edema, orthopnea, recurrent chest infections, morning headaches, fatigue, poor sleep quality, and loss of appetite.

[0021] A range of therapies have been used to treat or improve this condition. In addition, otherwise healthy individuals can take advantage of this therapy to prevent respiratory illness from occurring.

[0022] Positive pressure therapy

[0023] Continuous positive airway pressure (CPAP) therapy has been used to treat obstructive sleep apnea (OSA). It is hypothesized that continuous positive airway pressure acts as an inflatable splint and can prevent upper airway obstruction by pushing the soft palate and tongue forward and away from the posterior oropharyngeal wall. Treatment of OSA by CPAP can be voluntary, and therefore patients may choose not to comply with treatment if they find the device used to provide such treatment to be one or more of the following: uncomfortable, difficult to use, expensive, and unsightly.

[0024] Non-invasive ventilation (NIV) provides ventilation support to patients through the upper airway in order to assist the patient in breathing adequately and / or maintaining adequate oxygen levels in the body by doing some or all of the work of breathing. Ventilatory support is provided via a patient interface. NIV has been used to treat CSR, OHS, COPD, MD, and chest wall disease. In some forms, the comfort and effectiveness of these therapies can be improved.

[0025] Invasive ventilation (IV) provides ventilation support to patients who are no longer able to breathe effectively and can be provided using a tracheostomy tube. In some forms, the comfort and effectiveness of these therapies can be improved.

[0026] Diagnostic and therapeutic systems

[0027] These treatments can be provided by a treatment system or device. Systems and devices can also be used to diagnose a condition without treating it.

[0028] The therapy system may include a respiratory pressure therapy device (RPT device), air circuit, humidifier, patient interface, and data management.

[0029] Another form of treatment system is a mandibular repositioning device.

[0030] Patient Interface

[0031] A patient interface may be used to connect a respiratory device to its wearer, for example by providing an air flow to the entrance of the respiratory tract. The air flow may be provided to the nose and / or mouth via a mask, to the mouth via a tube, or to the patient's trachea via a tracheostomy tube. Depending on the therapy to be applied, the patient interface may, for example, form a seal with an area of ​​the patient's face so as to deliver gas at a pressure sufficiently different from the ambient pressure to achieve therapy, for example, at a positive pressure of about 10 cmH2O relative to the ambient pressure. For other forms of therapy, such as delivering oxygen, the patient interface may not include a seal sufficient to facilitate delivery of a supply of gas to the respiratory tract at a positive pressure of about 10 cmH2O.

[0032] The design of a patient interface presents many challenges. The face has a complex three-dimensional shape. The size and shape of the nose varies greatly between individuals. Because the head is composed of bones, cartilage, and soft tissue, different areas of the face respond differently to mechanical forces. The jaw or mandible can move relative to other bones of the skull. The entire head may move during respiratory therapy.

[0033] As a result of these challenges, some masks suffer from being offensive, aesthetically undesirable, expensive, inappropriate, difficult to use, and uncomfortable, one or more of these, particularly when worn for extended periods of time or when the patient is unfamiliar with the system. For example, masks designed for pilots, masks designed as part of personal protective equipment (e.g., filtering masks), SCUBA masks, or masks for the administration of anesthetics may be tolerable for their original applications, but such masks may not be expected to be worn uncomfortably for extended periods of time, such as for several hours. This discomfort may lead to a reduction in the patient's compliance with treatment. This is even more true if the mask is worn during sleep.

[0034] CPAP therapy is very effective for treating certain respiratory conditions, as long as the patient complies with the treatment. If the mask is uncomfortable or difficult to use, the patient may not comply with the treatment. Since patients are often advised to wash their masks regularly, if the mask is difficult to clean (e.g., difficult to assemble or disassemble), the patient may not clean their mask, which may affect patient compliance.

[0035] Although masks used for other applications (eg, pilots) may not be suitable for treating sleep-disordered breathing, masks designed for treating sleep-disordered breathing may be suitable for other applications.

[0036] For these reasons, patient interfaces for delivering CPAP during sleep represent a unique field.

[0037] Seal forming part

[0038] The patient interface may include a seal forming portion. Since it is in direct contact with the patient's face, the shape and configuration of the seal forming portion may directly affect the effectiveness and comfort of the patient interface.

[0039] The patient interface may be characterized in part according to the design intent of the position at which the seal-forming portion engages the face during use. In one form of the patient interface, the seal-forming portion may include two sub-portions that engage with the respective left and right nostrils. In one form of the patient interface, the seal-forming portion may include a single element that surrounds both nostrils during use. Such a single element may be designed, for example, to cover the upper lip region and the bridge of the nose region of the face. In one form of the patient interface, the seal-forming portion may include an element that surrounds the mouth region during use, for example, by forming a seal on the lower lip region of the face. In one form of the patient interface, the seal-forming portion may include a single element that surrounds both nostrils and the mouth region during use. These different types of patient interfaces may be known by their manufacturers by a variety of names, including nasal masks, full-face masks, nasal pillows, nasal sprays, and oronasal masks.

[0040] A seal-forming portion that may be effective in one area of ​​a patient's face may not be suitable in another area, for example because of different shapes, structures, variability, and sensitive areas of the patient's face. For example, a seal on swimming goggles that covers a patient's forehead may not be suitable for use on the patient's nose.

[0041] Certain seal forming portions may be designed for mass production such that one design fits, is comfortable and effective for a wide range of different face shapes and sizes. To the extent there is a mismatch between the shape of a patient's face and the seal forming portion of a mass-manufactured patient interface, one or both must adapt in order to form a seal.

[0042] One type of seal forming portion extends around the periphery of the patient interface and is intended to seal against the patient's face when a force is applied to the patient interface with the seal forming portion in facing engagement with the patient's face. The seal forming portion may include a cushion filled with air or fluid, or a molded or formed surface of a resilient sealing element made of an elastomer such as rubber. With this type of seal forming portion, if the fit is inadequate, there will be a gap between the seal forming portion and the face, and additional force will be required to force the patient interface against the face in order to achieve a seal.

[0043] Another type of seal forming portion includes a flap seal of thin material positioned around the periphery of the mask to provide a self-sealing action against the patient's face when positive pressure is applied within the mask. As with the previous type of seal forming portion, if the fit between the face and the mask is not good, then additional force may be required to achieve a seal, or the mask may leak. In addition, if the shape of the seal forming portion does not match the shape of the patient, then it may wrinkle or bend during use, causing leaks.

[0044] Another type of seal-forming portion may include a friction fit element, for example for insertion into a nostril, however some patients find this uncomfortable.

[0045] Another form of seal forming part can use adhesive to achieve the seal. Some patients may find it inconvenient to constantly apply and remove adhesive on the face.

[0046] A range of patient interface seal forming part technologies are disclosed in the following patent applications assigned to ResMed Ltd: WO 1998 / 004,310; WO 2006 / 074,513; WO 2010 / 135,785.

[0047] One form of nasal pillow is found in the Adam Circuit manufactured by Puritan Bennett. Another nasal pillow or nasal spray is the subject of U.S. Patent 4,782,832 (Trimble et al.) assigned to Puritan-Bennett Company.

[0048] ResMed Ltd. has manufactured the following products, including nasal pillows: SWIFT TM Nasal pillow mask, SWIFT TM II nasal pillow mask, SWIFT TM LT nasal pillow mask, SWIFT TM FX Nasal Pillows Mask and MIRAGE LIBERTY TMFull face mask. The following patent applications assigned to ResMed Ltd describe examples of nasal pillow masks: International Patent Application WO 2004 / 073,778 (describing ResMed Ltd's SWIFT TM Other aspects of nasal pillows), U.S. Patent Application 2009 / 0044808 (describing ResMed Ltd.'s SWIFT TM LT nasal pillow); International Patent Applications WO 2005 / 063,328 and WO 2006 / 130,903 (describing the MIRAGE LIBERTY TM Other aspects of full face masks); International Patent Application WO 2009 / 052,560 (describing ResMed Ltd.'s SWIFT TM Other aspects of the FX Nasal Pillows).

[0049] Positioning and stabilization

[0050] The seal forming part of the patient interface for positive pressure therapy is subjected to corresponding air pressure to break the seal. Therefore, various techniques have been used to position the seal forming part and maintain its sealing relationship with the appropriate part of the face.

[0051] One technique is to use an adhesive. See, for example, US Patent Application Publication No. US2010 / 0000534. However, the use of an adhesive may be uncomfortable for some people.

[0052] Another technique is to use one or more straps and / or stabilizing harnesses. Many of these harnesses suffer from being one or more of ill-fitting, bulky, uncomfortable, and awkward in use.

[0053] Patient interface sizing

[0054] The patient interface as described above can be provided to the patient in various forms, such as a nasal mask or a full face mask / oronasal mask (FFM) or a nasal pillow mask. Such patient interfaces are manufactured with various sizes to accommodate the anatomical features of specific patients so as to facilitate a comfortable interface for providing, for example, positive pressure therapy. The size of such a patient interface can be customized to correspond to the specific facial anatomy of a specific patient, or can be designed to accommodate a population of individuals with anatomy that falls within a predetermined spatial boundary or range. However, in some cases, the mask can have multiple standard sizes from which the appropriate size is selected.

[0055] In this regard, sizing a patient interface for a patient is typically performed by a trained individual, such as a durable medical equipment (DME) provider or physician. Typically, a patient requiring a patient interface to begin or continue positive pressure therapy will visit a trained individual at a housing facility who will diligently perform a series of measurements to determine the appropriate patient interface size from standard sizes. The appropriate size is intended to represent a specific combination of dimensions of certain features of the patient interface, such as a seal-forming structure, that provide adequate comfort and seal to achieve positive pressure therapy. Sizing in this manner is not only labor intensive but also inconvenient. The inconvenience of taking time out of a busy schedule or, in some cases, having to travel long distances is a barrier to many patients receiving a new or replacement patient interface and ultimately a barrier to receiving treatment. Nonetheless, selecting the most appropriate size is important for quality of treatment and compliance. Summary of the invention

[0056] The present technology is intended to provide a medical device for diagnosing, ameliorating, treating or preventing respiratory diseases, which has one or more of improved comfort, cost, efficacy, ease of use and manufacturability.

[0057] A first aspect of the present technology relates to a device for diagnosing, ameliorating, treating or preventing respiratory diseases.

[0058] Another aspect of the present technology relates to methods for diagnosing, ameliorating, treating or preventing respiratory diseases.

[0059] An aspect of some forms of the present technology is to provide methods and / or apparatus for improving patient compliance with respiratory therapy.

[0060] One form of the present technology involves automatically adjusting the size of a patient interface without the assistance of trained individuals.

[0061] Another aspect of one form of the present technology is automatically measuring facial features of a patient based on data collected from the patient.

[0062] Another aspect of one form of the present technology is automatically recommending a patient interface size based on a comparison between data collected from the patient and a corresponding data record.

[0063] Another aspect of one form of the present technology is a mobile application that conveniently determines an appropriate patient interface size for a particular patient based on a two-dimensional image.

[0064] Some versions of the present technology include one or more automatic methods for selecting a patient interface based on a patient interface size. The method or methods may be operated in one or more processors. The method may include receiving image data captured by an image sensor. The captured image data may contain one or more facial features of an intended user of the patient interface associated with predetermined reference features having known dimensions. The method may include detecting one or more facial features of the user in the captured image data. The method may include detecting a predetermined reference feature in the captured image data. The method may include processing image pixel data of the image to measure a longitudinal and lateral spacing of one or more facial features detected in the image based on the predetermined reference feature. The method may include selecting a patient interface size from a set of standard patient interface sizes based on a comparison of one or more facial features and a data record's measured longitudinal and lateral spacing of size information about a set of standard patient interface sizes with the measured longitudinal and lateral spacing of one or more facial features.

[0065] In some versions, the longitudinal and lateral spacing of one or more facial features may include the distance between the bridge of the nose and the supramenton of the user. The method may include calculating the value of the measured longitudinal and lateral spacing based on a scaling factor derived from a reference feature. The method may include adjusting the value of the measured longitudinal and lateral spacing with an anthropometric correction factor. The anthropometric correction factor may be calculated based on the patient interface return data. The method may include calculating the scaling factor as a function of the known size of a predetermined reference feature and the number of detected pixels of the detected reference feature. The predetermined reference feature may be a coin. Detecting the reference feature may include applying a cascade classifier to the captured image data. The method may include calculating the value of the measured longitudinal and lateral spacing based on the scaling factor derived from the coin. The method may include calculating the scaling factor as a function of the known size of the coin in the captured image data and the number of detected pixels of the detected coin. The number of detected pixels of the detected coin may be the width of an ellipse fitted to the coin. The predetermined reference feature may be the user's cornea.

[0066] In some versions, for image capture, the method may include displaying a reference feature on a display interface of a display device coupled to an image sensor. The display interface may include a target guide and a live action preview of content detected by the image sensor. The content may include a reference feature displayed on the display interface. The method may include controlling the capture of image data so as to satisfy at least one alignment condition. The at least one alignment condition may include detecting the positioning of the reference feature of the live action preview within a box of the target guide. The at least one alignment condition may include detecting a tilt condition within approximately + / - 10 degrees of an axis extending vertically. The at least one alignment condition may include detecting a tilt condition within approximately + / - 5 degrees of an axis extending vertically. The detection of the tilt condition may be performed by reading an inertial measurement unit (IMU).

[0067] In some versions, the predetermined reference feature can be a QR code. The patient interface can include a facial mask. The patient interface can include a nasal mask. Optionally, processing the image pixel data can include counting pixels. The method can include generating an automatic electronic quote for a patient interface for purchase based on a selected patient interface size. The method can include calculating an average of measured longitudinal and lateral spacings of the facial feature from a plurality of captured images of the one or more facial features.

[0068] Some versions of the present technology include one or more systems for automatically recommending a patient interface size that is complementary to a particular patient's facial features. The system or systems may include one or more servers. The one or more servers may be configured to communicate with a computing device over a network. The one or more servers may be configured to receive image data captured by an image sensor, wherein the captured image data may contain one or more facial features of an intended user of a patient interface associated with predetermined reference features of known dimensions. The one or more servers may be configured to detect one or more facial features of a user in the captured image data. The one or more servers may be configured to detect a predetermined reference feature in the captured image data. The one or more servers may be configured to process the image pixel data of the image to measure a longitudinal and lateral spacing of one or more facial features detected in the image based on the predetermined reference feature. The one or more servers may be configured to select a patient interface size from a set of standard patient interface sizes based on a comparison of one or more facial features and a data record's measured longitudinal and lateral spacing of size information about a set of standard patient interface sizes with the measured longitudinal and lateral spacing of one or more facial features.

[0069] In some versions, the longitudinal and transverse spacing of one or more facial features may include the distance between the bridge of the nose point and the suprachinal point of the user. The one or more servers may be configured to calculate the value of the measured longitudinal and transverse spacing based on a scaling factor derived from a reference feature. The one or more servers may be configured to adjust the value of the measured longitudinal and transverse spacing with an anthropometric correction factor. The anthropometric correction factor may be calculated based on the patient interface return data. The one or more servers may be configured to calculate the scaling factor as a function of the known size of a predetermined reference feature and the number of detected pixels of the detected reference feature. The predetermined reference feature may include a coin. The one or more servers may be configured to detect the reference feature by applying a cascade classifier to the captured image data. The one or more servers may be further configured to calculate the value of the measured longitudinal and transverse spacing based on the scaling factor derived from the coin. The one or more servers may be configured to calculate the scaling factor as a function of the known size of the coin in the captured image data and the number of detected pixels of the detected coin. The number of detected pixels of the detected coin may be the width of an ellipse fitted to the coin. The predetermined reference feature may be the user's cornea.

[0070] In some versions, the system may include a computing device. The computing device may be configured to generate a display of reference features on a display interface of a display device that may be coupled to an image sensor for image capture. The display interface may include a target guide and a live action preview of content detected by the image sensor. The content may include a reference feature displayed on the display interface. The computing device may be further configured to control the capture of image data so as to satisfy at least one alignment condition. The at least one alignment condition may include detecting the positioning of the reference feature of the live action preview within a box of the target guide. The at least one alignment condition may include detecting a tilt condition within approximately + / -10 degrees of an axis extending up and down. The at least one alignment condition may include detecting a tilt condition within approximately + / -5 degrees of an axis extending up and down. The detection of the tilt condition may be performed by reading an inertial measurement unit (IMU).

[0071] In some versions, the predetermined reference feature may include a QR code. The patient interface may include a facial mask. The patient interface may include a nasal mask. In some cases, to process the image pixel data, the one or more servers may be configured to count pixels. The one or more servers may be configured to generate an automatic electronic quote for a patient interface for purchase based on the selected patient interface size. The one or more servers may be configured to calculate an average of the measured longitudinal and lateral spacings of the facial features from multiple captured images of the facial features. The one or more servers may be configured to transmit the selected patient interface size to a computing device via a network.

[0072] Some versions of the present technology include one or more systems for automatically recommending a patient interface size that is complementary to a particular patient's facial features. The system(s) may include a mobile computing device. The mobile computing device may be configured to communicate with one or more servers over a network. The mobile computing device may be configured to receive captured image data of an image. The captured image data may contain one or more facial features of a user associated with predetermined reference features of known dimensions. The image data may be captured with an image sensor. The mobile computing device may be configured to detect one or more facial features of a user in the captured image data. The mobile computing device may be configured to detect predetermined reference features in the captured image data. The mobile computing device may be configured to process the image pixel data of the image to measure a longitudinal and lateral spacing of one or more facial features detected in the image based on the predetermined reference features. The mobile computing device may be configured to select a patient interface size from a set of standard patient interface sizes based on a comparison of one or more facial features and a data record's measured longitudinal and lateral spacing of size information about a set of standard patient interface sizes with the measured longitudinal and lateral spacing of the one or more facial features.

[0073] In some versions, the longitudinal and transverse spacing of one or more facial features may include a distance between a nose bridge point and a suprachinal point of the user. The mobile computing device may be configured to calculate a value of the measured longitudinal and transverse spacing based on a scaling factor derived from a reference feature. The mobile computing device may be further configured to adjust the value of the measured longitudinal and transverse spacing with an anthropometric correction factor. The anthropometric correction factor may be calculated based on the patient interface return data. The mobile computing device may be configured to calculate a scaling factor as a function of a known size of a predetermined reference feature and a number of detected pixels of the detected reference feature. The predetermined reference feature may be a coin. The mobile computing device may be configured to detect the reference feature by applying a cascade classifier to the captured image data. The mobile computing device may be configured to calculate a value of the measured longitudinal and transverse spacing based on a scaling factor derived from the coin. The mobile computing device may be configured to calculate a scaling factor as a function of a known size of the coin in the captured image data and a number of detected pixels of the detected coin. The number of detected pixels of the detected coin may be the width of an ellipse fitted to the coin. In some versions, the predetermined reference feature may be a cornea of ​​the user.

[0074] The mobile computing device may be configured to generate a display of reference features for image capture on a display interface of a display device that may be coupled to an image sensor. The display interface may include a target guide and a live action preview of content detected by the image sensor. The content may include a reference feature displayed on the display interface. The mobile computing device may be configured to control the capture of image data so as to satisfy at least one alignment condition. The at least one alignment condition may include detecting the positioning of the reference feature of the live action preview within a box of the target guide. The at least one alignment condition may include detecting a tilt condition within approximately + / - 10 degrees of an axis extending vertically. The at least one alignment condition may include detecting a tilt condition within approximately + / - 5 degrees of an axis extending vertically. The detection of the tilt condition may be performed by reading an inertial measurement unit (IMU).

[0075] In some versions, the predetermined reference feature may be a QR code. The patient interface may be a facial mask. The patient interface may be a nasal mask. In some cases, to process the image pixel data, the mobile computing device may be configured to count pixels. The mobile computing device may be configured to request an automatic electronic quote for a patient interface for purchase based on the selected patient interface size. The mobile computing device may be configured to calculate an average of measured longitudinal and lateral spacings of the facial feature from a plurality of captured images of the facial feature. The mobile computing device may be configured to transmit the selected patient interface size to a server over a network.

[0076] Some versions of the present technology include an apparatus for automatically recommending a patient interface size that is complementary to facial features of a particular patient. The apparatus may include means for receiving image data captured by an image sensor. The captured image data may contain one or more facial features of an intended user of the patient interface associated with predetermined reference features having known dimensions. The apparatus may include means for detecting one or more facial features of the user in the captured image data. The apparatus may include means for detecting the predetermined reference features in the captured image data. The apparatus may include means for processing image pixel data of the image to measure longitudinal and lateral spacings of one or more facial features detected in the image based on the predetermined reference features. The apparatus may include means for selecting a patient interface size from a set of standard patient interface sizes based on a comparison of one or more facial features and a data record's measured longitudinal and lateral spacings of size information about the set of standard patient interface sizes with the measured longitudinal and lateral spacings of the one or more facial features.

[0077] Of course, a part of these aspects may form a sub-aspect of the present technology. In addition, different aspects in the sub-aspects and / or various aspects may be combined in various ways, and also constitute additional aspects or sub-aspects of the present technology.

[0078] Other features of the technology will become apparent by considering the information contained in the following detailed description, abstract, drawings, and claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0079] The present technology is illustrated by way of example and not by way of limitation in the figures of the accompanying drawings, in which like reference numerals indicate similar elements, including:

[0080] Treatment system

[0081] Figure 1A A system is shown including a patient wearing a patient interface in the form of nasal pillows;

[0082] Figure 1B A system is shown including a patient wearing a patient interface in the form of a nasal mask;

[0083] Figure 1C A system is shown including a patient wearing a patient interface in the form of a full face mask;

[0084] Figure 2 A patient interface in the form of a nasal mask with a head cap is shown in accordance with one form of the present technology.

[0085] Figure 3A is a front view of the face with several features of surface anatomy identified including the upper lip, upper lip vermilion, lower lip vermilion, lower lip, mouth width, medial canthus, nasal wing, nasolabial groove, and corner of mouth. The directions of superior, inferior, radially inward, and radially outward are also indicated.

[0086] Figure 3B is a side view of the head with several features of surface anatomy identified including the glabella, nasal bridge point, nasal tip point, subseptal point, upper lip, lower lip, supramental point, nasal ridge, nasal vertex, supraauricular diameter, and subauricular diameter. The superior-inferior and anterior-posterior directions are also indicated.

[0087] Figure 3C It is a basic view of the nose with several features identified including the nasolabial folds, lower lip, upper lip, nostrils, subseptal point, columella, tip point, long axis of the nostrils, and the sagittal plane.

[0088] Figure 4 is a diagram of an exemplary system for automatically determining the size of a patient interface including a computing device.

[0089] Figure 5 is used for Figure 4 A block diagram of an exemplary architecture of a computing device of a system including exemplary components suitable for implementing the methods of the present technology.

[0090] Fig. 6Ais a flow chart of a pre-capture phase method of an exemplary version of the present technology.

[0091] Figure 6B is a flow chart of the capture phase method of some versions of the present technology.

[0092] Figure 6C is a flow chart of the post-capture image processing stage methods of some versions of the present technology.

[0093] Fig.6D is a flow chart of the comparison and output phase methods of some versions of exemplary method embodiments of the present technology.

[0094] Fig. 7A Exemplary display interfaces for some versions of the present technology are shown.

[0095] Figure 7B Another exemplary display interface of some versions of the present technology is shown.

[0096] Figure 7C A user employing a method of the present technology is shown.

[0097] Figure 8 is a flow chart of a method of implementing a post-capture image processing stage according to another implementation of the present technology.

[0098] Fig. 9 It shows the implementation Figure 8 A flow chart of the method of the facial feature detection steps of the method.

[0099] Fig.10 It shows the implementation Figure 8 A flow chart of the method of the reference feature detection steps of the method.

[0100] Fig.11 It shows the implementation Figure 8 A flow chart of the method of the scaling factor calculation steps.

[0101] Fig.12 It shows the implementation Figure 8 A flow chart of the method of the facial feature measurement steps of the method.

[0102] Fig.13A is an illustration of the positioning of a coin on a user's forehead according to another implementation of the present technology.

[0103] Fig. 13B is based on Figure 8 Illustration of the detection boxes and subsequent facial feature measurements of our approach.

[0104] Fig.14 is based on Figure 8Illustration of detected facial features in a captured image of an alternative implementation of the method. DETAILED DESCRIPTION

[0105] Before describing the present technology in further detail, it should be understood that the present technology is not limited to the specific examples described herein, which may vary. It should also be understood that the terminology used in the present disclosure is only used to describe the specific examples discussed herein and is not intended to be limiting.

[0106] The following description is provided with respect to different examples that can share one or more common characteristics and / or features. It should be understood that one or more features of any one example can be combined with one or more features of another example or other examples. In addition, any single feature or combination of features in any example can constitute another example.

[0107] Treatment system

[0108] In one form, the present technology includes an apparatus or device for treating respiratory diseases. The apparatus or device may include an RPT device 40 for supplying pressurized air to a patient 10 via an air circuit 50 to a patient interface 100 .

[0109] Figure 1A A system is shown including a patient 10 wearing a patient interface 100 in the form of nasal pillows receiving a supply of air at a positive pressure from an RPT device 40. Air from the RPT device is humidified in a humidifier 60 and travels along an air circuit 50 to the patient 10. A bed partner 20 is also shown.

[0110] Figure 1B A system is shown including a patient 10 wearing a patient interface 100 in the form of a nasal mask receiving a supply of air at a positive pressure from an RPT device 40. The air from the RPT device is humidified in a humidifier 60 and travels along an air circuit 50 to the patient 10.

[0111] Figure 1C A system is shown including a patient 10 wearing a patient interface 100 in the form of a full face mask (FFM) receiving a supply of air at a positive pressure from an RPT device 40. The air from the RPT device is humidified in a humidifier 60 and travels along an air circuit 50 to the patient 10.

[0112] Patient Interface

[0113] Figure 2A patient interface 100 is depicted according to one aspect of the present technology, comprising the following functional aspects: a seal-forming structure 160, an inflatable chamber 120, a positioning and stabilizing structure 130, a vent 140, a forehead support 150, a connection port 170 for connecting to a form of air circuit 50. In some forms, the functional aspects may be provided by one or more physical components. In some forms, one physical component may provide one or more functional aspects. In use, the seal-forming structure 160 is arranged to surround the entrance of the patient's airway so as to facilitate the supply of air to the airway at a positive pressure.

[0114] Sealing structure

[0115] In one form of the present technology, the seal-forming structure 160 provides a seal-forming surface and may additionally provide a cushioning function.

[0116] The seal-forming structure 160 in accordance with the present technology may be comprised of a soft, flexible, resilient material such as silicone.

[0117] In one form, the seal-forming portion of the non-invasive patient interface 100 includes a pair of nasal puffs or pillows, each nasal puff or pillow constructed and arranged to form a seal with a corresponding nostril of the patient's nose.

[0118] A nasal pillow according to the present technology includes: a frustoconical body, at least a portion of which forms a seal on the underside of a patient's nose; a handle, a flexible region on the underside of the frustoconical body and connecting the frustoconical body to the handle. In addition, the structure to which the nasal pillow of the present technology is connected includes a flexible region adjacent to the base of the handle. The flexible regions can act in concert to facilitate the formation of a universal joint structure that accommodates relative movement (both displacement and angle) of the frustoconical body and the structure to which the nasal pillow is connected. For example, the frustoconical body can be axially displaced toward the structure to which the handle is connected.

[0119] In one form, the non-invasive patient interface 100 includes a seal-forming portion that forms a seal when used on the upper lip region (ie, upper lip) of a patient's face.

[0120] In one form, the non-invasive patient interface 100 includes a seal-forming portion that forms a seal when used on the chin area of ​​the patient's face.

[0121] Inflatable room

[0122] Preferably, the plenum 120 has a perimeter that is shaped to complement the surface contour of the average person's face in the area where a seal will be formed in use. In use, the boundary edge of the plenum 120 is positioned proximate to the adjacent surface of the face. The actual contact with the face is provided by the seal-forming structure 160. The seal-forming structure 160 may extend around the entire perimeter of the plenum 120 in use.

[0123] Positioning and stabilizing structures

[0124] Preferably, the seal-forming structure 160 of the patient interface 100 of the present technology may be maintained in a sealed position by the positioning and stabilising structure 130 during use.

[0125] Vent

[0126] In one form, patient interface 100 includes a vent 140 constructed and arranged to allow clearance of exhaled carbon dioxide.

[0127] One form of vent 140 according to the present technology includes a plurality of holes, for example, from about 20 to about 80 holes, or from about 40 to about 60 holes, or from about 45 to about 55 holes.

[0128] Terminology used in relation to the patient interface

[0129] Plenum: A mask plenum will be taken to mean a portion of the patient interface having walls enclosing a volume of space having air therein which is pressurized above atmospheric pressure in use. The housing may form part of the walls of the mask plenum.

[0130] Seal: The noun form ("seal") will be understood to mean a structure or barrier that intentionally prevents the flow of air across the interface of two surfaces. The verb form ("seal") will be understood to mean to prevent the flow of air.

[0131] Facial anatomy

[0132] Figure 3A A front view of a human face including the inner canthus, nose wing, nasolabial groove, upper and lower lips, upper and lower lip vermilion borders, and corners of the mouth is shown. Also shown is the mouth width, the longitudinal plane dividing the head into left and right parts, and a direction indicator. The direction indicator indicates radial inward / outward and up / down directions.

[0133] Figure 3B A side view of a human face including the brow, nose bridge point, nose ridge, nose tip point, nasal septum subpoint, lip upper part, lip lower part, chin suprapoint, nose vertex, ear suprapoint diameter and ear subpoint diameter is shown. Direction indicators indicating the up / down direction and the front / back direction are also shown.

[0134] Figure 3CA basic view of the nose is shown with several features identified including the nasolabial folds, lower lip, upper lip, nostrils, subseptal point, columella, tip point, long axis of the nostrils, and the sagittal plane.

[0135] Terms used in relation to facial anatomy

[0136] The outer lining or "wing" of each nostril (plural: alars)

[0137] Alar tip: The outermost point on the wing of the nose.

[0138] Alar curvature (or nasal roof) point: The posteriormost point of the base line of curvature of each ala, found in the wrinkle formed by the union of the ala and cheek.

[0139] Auricle: The entire external visible part of the ear.

[0140] Columella: The band of skin that separates the nostrils and extends from the tip of the nose to the upper lip.

[0141] Nasal columella angle: The angle between a line drawn through the midpoint of the nostril aperture and a line drawn perpendicular to the auricular plane and intersecting the lower point of the nasal septum.

[0142] Glabella: The most prominent point on the forehead in the midsagittal plane located on the soft tissue.

[0143] Nostril (nose hole): The approximately oval-shaped hole that forms the entrance to the nasal cavity. The singular form of nostril is a nostril (nose hole). The nostrils are separated by the nasal septum.

[0144] Nasolabial Folds or Nasolabial Folds: The skin folds or depressions that extend from the sides of the nose to the corners of the mouth, separating the cheeks from the upper lip.

[0145] Nasolabial angle: The angle between the nasal columella and the upper lip, which also intersects the lower point of the nasal septum.

[0146] Suprauricular diameter: The lowest point where the auricle attaches to the facial skin.

[0147] Auricularia: The highest point where the auricle attaches to the facial skin.

[0148] Tip: The most prominent point or tip of the nose which can be identified in a side view of the rest of the head.

[0149] Philtrum: The midline depression extending from the lower edge of the nasal septum to the top of the lip in the upper lip area.

[0150] Chin point: Located on the soft tissue, it is the front midpoint of the chin.

[0151] Ridge (nose): The nasal ridge is the midline prominence of the nose, extending from the bridge point to the tip point.

[0152] Septum Plane: The vertical plane passing from front (anterior) to back (posterior) dividing the body into left and right halves.

[0153] Nasal point: The most concave point on the soft tissue covering the frontonasal suture area.

[0154] Septal Cartilage (Nose): Septal cartilage forms part of the nasal septum and separates the front part of the nasal cavity.

[0155] Posterior superior lateral flap: The point at the lower edge of the alar floor where the alar base joins the skin of the upper (upper) lip.

[0156] Subnasal point: The point on the soft tissue where the columella joins the upper lip in the midsagittal plane.

[0157] Supramental point: the point of greatest depression in the midline of the lower lip between the midpoint of the lower lip and the anterior chin point of the soft tissue

[0158] Automatic patient interface resizing

[0159] Overview

[0160] Obtaining a patient interface allows the patient to undergo positive pressure therapy. Patients seeking their first patient interface or a new patient interface to replace an older interface typically consult a DME to determine a recommended patient interface size based on the measurement of the patient's facial anatomy, which is typically performed by the DME. This can be inconvenient, which can prevent some patients from receiving the desired patient interface and from undergoing positive pressure therapy. The present technology allows patients to obtain a patient interface faster and more conveniently. It can allow a more efficient method to quickly measure their facial anatomy and receive recommendations for appropriate patient interface size from the comfort of their own home using a computing device such as a desktop computer, tablet computer, smart phone, or other mobile device.

[0161] In a beneficial embodiment, the technology can employ an application that can be downloaded from the manufacturer or a third-party server to a smartphone or tablet computer with an integrated camera. Upon startup, the application can provide visual and / or audio instructions. As indicated, the user (i.e., the patient) can stand in front of the mirror 330 and press the camera button on the user interface. The startup process can then take a series of pictures of the user's face and then recommend a patient interface size to the user (based on the processor analyzing these pictures), for example, within a few seconds. This is a huge improvement over the traditional method of visiting a DME using a series of measurements with calipers, because it allows users anywhere in the world to quickly and easily find a patient interface that fits their needs. Therefore, it can allow patients to start treatment more quickly. In addition, because the user can control the process, the client can repeat as needed, without interference and with satisfaction, thereby increasing the user's confidence and sense of responsibility.

[0162] As further described below, the present technology allows a user / patient to capture an image or series of images of their facial structure. Instructions provided by an application stored on a computer-readable medium (such as when executed by a processor) detect various facial landmarks within the image, measure and scale the distances between these landmarks, compare these distances to a data record, and recommend an appropriate patient interface size. Thus, a consumer's automated device can allow accurate patient interface selection, such as at home, to allow the consumer to determine size without the need for a trained companion.

[0163] system

[0164] Figure 4 An exemplary system 200 that can be implemented for automatic facial feature measurement and patient interface sizing is depicted. System 200 may generally include one or more of server 210, communication network 220, and computing device 230. Server 210 and computing device 230 may communicate via communication network 220, which may be a wired network 222, a wireless network 224, or a wired network with a wireless link 226. In some versions, server 210 may communicate unidirectionally with computing device 230 by providing information to computing device 230, and vice versa. In other embodiments, server 210 and computing device 230 may share information and / or processing tasks. For example, the system may be implemented to allow automatic purchase of an interface (mask) for a patient, wherein the process may include an automatic sizing process described in more detail herein. For example, a consumer may order a mask online after running a mask selection process that automatically identifies a suitable mask size by performing image analysis of the consumer's facial features.

[0165] Computing Devices

[0166] Computing device 230 may be a desktop or laptop computer 232 or a mobile device such as a smartphone 234 or a tablet computer 236 . Figure 5 A general architecture 300 of a computing device 230 is depicted. The device 230 may include one or more processors 310. The device 230 may also include a display interface 320, a user control / input interface 331, a sensor 340 and / or a sensor interface for one or more sensors, an inertial measurement unit (IMU) 342, and a non-volatile memory / data storage device 350.

[0167] Sensor 340 may be one or more cameras (e.g., CCD charge coupled devices or effective pixel sensors) integrated into computing device 230, such as the cameras provided in a smartphone or laptop computer. Alternatively, where computing device 230 is a desktop computer, device 230 may include a camera for communicating with an external camera (e.g., a Figure 4 Other exemplary sensors that may be used to assist the methods described herein may be integrated with or external to the computing device, including a stereo camera for capturing three-dimensional images, or a light detector capable of detecting reflected light from a laser or a strobed / structured light source.

[0168] The user control / input interface 331 allows the user to provide commands or responses to prompts or instructions provided to the user. For example, this may be a touch panel, keyboard, mouse, microphone and / or speaker.

[0169] Display interface 320 may include a monitor, LCD panel, or the like to display prompts, output information (such as facial measurements or interface size recommendations), and other information, such as a capture display, as described in further detail below.

[0170] The memory / data storage device 350 may be an internal memory of the computing device, such as RAM, flash memory, or ROM. In some embodiments, for example, the memory / data storage device 350 may also be an external memory linked to the computing device 230, such as an SD card, a server, a USB flash drive, or an optical disk. In other embodiments, the memory / data storage device 350 may be a combination of external memory and internal memory. The memory / data storage device 350 includes stored data 354 and processor control instructions 352 that instruct the processor 310 to perform certain tasks. The stored data 354 may include data received by the sensor 340 (such as a captured image) and other data provided as part of the application. The processor control instructions 352 may also be provided as a component of the application.

[0171] Application of facial feature measurement and patient interface sizing

[0172] One such application is an application for facial feature measurement and / or patient interface sizing 360, which may be an application downloadable to a mobile device such as a smartphone 234 and / or a tablet computer 236. The application 360, which may be stored on a computer-readable medium such as a memory / data storage device 350, includes programming instructions for the processor 310 to perform certain tasks related to facial feature measurement and / or patient interface sizing. The application also includes data that may be processed by the algorithms of the automated method. Such data may include data records, reference features, and correction factors, as explained in additional detail below.

[0173] Methods for automatic measurement and sizing

[0174] like Figures 6A-6D As shown in the flowchart of , one aspect of the present technology is a method for controlling a processor such as processor 310 to measure a patient's facial features using a two-dimensional or three-dimensional image and recommend or select an appropriate patient interface size based on the obtained measurements, such as from a set of standard sizes. The method can generally be characterized as including three or four different stages: a pre-capture stage 400, a capture stage 500, a post-capture image processing stage 600, and a comparison and output stage 700.

[0175] In some cases, the application for facial feature measurement and patient interface sizing can control the processor 310 to output a visual display including a reference feature on the display interface 320. The user can position the feature adjacent to his facial feature, such as by movement of the camera. The processor can then capture and store one or more images of the facial feature associated with the reference feature when certain conditions such as alignment conditions are met. This can be done with the help of the mirror 330. The mirror 330 reflects the displayed reference feature and the user's face to the camera. The application then controls the processor 310 to identify certain facial features within the image and measure the distance between them. Through image analysis processing, a scaling factor can then be used to convert the facial feature measurement value, which can be a number of pixels, into a standard mask measurement value based on the reference feature. Such a value can be, for example, a standardized unit of measurement, such as meters or inches, and a value expressed in such units suitable for mask size. Additional correction factors can be applied to the measurement. For example, the facial feature measurement value can be compared with a data record including a measurement range of different patient interface sizes corresponding to a specific patient interface form (such as a nasal mask and an FFM). A recommended size can then be selected and output to the user / patient as a recommendation based on the comparison result or results. If the user so chooses, then this process can be implemented in the comfort and convenience of the user's own home. The application can perform this method in a few seconds. In one example, the application performs this method in real time.

[0176] Pre-capture stage 400

[0177] In by Fig. 6A In the pre-capture stage represented by the flowchart of , the processor 310 or the like assists the user in establishing appropriate conditions for capturing one or more images for the resizing process. For example, some of these conditions include appropriate lighting and camera orientation and motion blur caused by an unsteady hand holding the computing device 230.

[0178] In one version of the method, a user may conveniently download an application for performing automatic measurement and sizing at the computing device 230 from a server, such as a third-party application store server, to their computing device 230. When downloaded, such an application may be stored on an internal non-volatile memory (such as RAM or flash memory) of the computing device. The computing device 230 is preferably a mobile device, such as a smartphone 234 or a tablet computer 236. When the user launches the application, the processor 310 may prompt the user to provide patient-specific information such as age, gender, weight, and height via the display interface 320 of the computing device. However, the processor 310 may prompt the user to enter this information at any time, such as after the user's facial features are measured. The processor 310 may also present a tutorial, which may be presented audibly and / or visually as provided by the application to help the user understand their role in the process. These prompts may also require information for the type of patient interface, such as nose or full face, etc., and the type of device with which the patient interface will be used. Additionally, in the pre-capture phase 400, the application may infer patient-specific information based on information already collected about the user (such as after receiving a captured image of the user's face, and based on machine learning techniques or through artificial intelligence).

[0179] Sensor Start 410

[0180] When the user is ready to proceed, processor 310 activates sensor 340 as directed by processor control instructions 352 of the application, which may be indicated by user input or in response to a prompt via user control / input interface 331. Sensor 340 is preferably a forward-facing camera of the mobile device located on the same side of the mobile device as display interface 320. Cameras are typically configured to capture two-dimensional images. Mobile device cameras that capture two-dimensional images are ubiquitous. The present technology exploits this ubiquity to avoid the burden of users needing to obtain specialized equipment.

[0181] Display 420

[0182] At approximately the same time the sensor / camera 340 is activated, the processor 310 presents the captured display content on the display interface 320 as directed by the application. Fig. 7AAn example of a capture display 322 and its contents is depicted, which may include a camera live action preview 324, a reference feature 326, a target box 328, and one or more status indicators 327, or any combination thereof. In this example, the reference feature 326 is displayed centered on the display interface and has a width corresponding to the width of the display interface 320. The vertical position of the reference feature 326 may be such that the top edge of the reference feature 326 abuts the uppermost edge of the display interface 320, or the bottom edge of the reference feature 326 abuts the lowermost edge of the display interface 320. A portion of the display interface 320 will display the camera live action preview 324, and if the user is in the correct position and orientation, then the user's facial features captured by the sensor / camera 340 are generally displayed in real time.

[0183] The live action preview 324 is a stream of images / content as seen / detected in real time, for example, by the camera / sensor 340. Thus, if the user points the front-facing camera 340 at the user's facial features, the user's facial features may be presented on the display interface 320. Similarly, if the user points the front-facing camera 340 at the mirror 330, the reflection in the mirror 330 is displayed on the display interface 320 as part of the live action preview 324, which will preferably include the display interface 320 and one or more of its contents (including the reference feature 326). However, it should be understood that while the live action preview 324 may include the patient's facial features, such facial features need not be displayed on the display interface 320, as the user may be directed to the patient's facial features. Fig. 7A Nonetheless, sensor 340 does capture facial features during this aspect of the process.

[0184] Reference features 326 are features that are known (predetermined) by computing device 230 and provide processor 310 with a frame of reference that allows processor 310 to scale the captured image. Reference features may preferably be features other than the user's facial or anatomical features. Thus, during image processing stage 600, they help processor 310 determine when certain alignment conditions are met, such as during pre-capture stage 400. Fig. 7AAs shown, the reference feature 326 can be a quick response (QR) code or a known example or mark, which can provide certain information to the processor 310, such as scaling information, orientation and / or any other desired information, which can be optionally determined by the structure of the QR code. The QR code can have a square or rectangular shape. When displayed on the display interface 320, the reference feature 326 has a predetermined size, such as in millimeters or centimeters, and its value can be encoded into the application and transmitted to the processor 310 at the appropriate time. The actual size of the reference feature 326 can vary between various computing devices. In some versions, the application can be configured to a specific computing device model, where the size of the reference feature 326 is known when displayed on a specific model. However, in other embodiments, the application can instruct the processor 310 to obtain certain information from the device 230, such as a display size and / or zoom characteristics that allow the processor 310 to calculate the real world / actual size of the reference feature 326 displayed on the display interface 320 via scaling. Regardless, the actual size of reference feature 326 displayed on display interface 320 of such a computing device is typically known prior to post-capture image processing.

[0185] Along with the reference feature 326, a target box 328 may be displayed on the display interface 320 and overlay the live action preview 324. The target box 328 allows the user to aim at certain features within the capture display 322 in the target box 328, which may be desirable for successful image capture. Fig. 7A In one example shown, the application can include a capture condition where the reference feature 326 is to be completely within the target box 328 prior to image capture. Alignment of the reference feature 326 within the target box 328 can improve detection during later processing and ensure good location and alignment of the reference feature 326 within the captured image. Additionally, alignment within the target box 328 can help ensure that the display interface 320 is aligned along the upper-lower axis to avoid excessive radial inward or outward tilting and rotationally about the upper-lower axis to maintain the display interface 320 approximately parallel to the mirror 330, for example.

[0186] Status indicator 327 provides information to the user about the status of the process. This helps ensure that the user does not make major adjustments to the positioning of the sensor / camera before image capture is complete.

[0187] Therefore, in Fig. 7AIn the depicted embodiment, when the user holds the display interface 320 parallel to the facial feature to be measured and presents the user display interface 320 to a mirror 330 or other reflective surface, the reference feature 326 is highlighted and overlays the real-time image seen by the camera / sensor 340 and as reflected by the mirror 330. This reference feature 326 can be fixed near the top of the display interface 320. The reference feature 326 is at least partially highlighted in this manner so that the sensor 340 can clearly see the reference feature 326, so that the processor 310 can easily identify the feature 326. In addition, the reference feature 326 can overlay the live view of the user's face, which helps avoid user confusion. The movable reference feature image 329, which is a reflection of the real-time live view of the reference feature 326, can be positioned within the alignment box by movement of the display device having the image sensor. Finally, when movable reference feature image 329 is positioned within target box 328, alignment box and movable reference feature image 329 will be positioned and aligned in an offset manner relative to reference feature 326 (eg, directly underneath), which aids usability.

[0188] Other features or information may be displayed by the processor 310 on the display interface 320. For example, the application may establish parameters that must be met regarding lighting conditions. If the lighting conditions are unsatisfactory, the processor 310 may display a warning on the display interface or output an audible warning to the user and indicate steps that the user may take to help correct the unsatisfactory lighting conditions.

[0189] Figure 7B An alternative capture display embodiment 320a is shown that similarly includes a target box 328 and a reference feature 326. However, unlike Fig. 7A , live action preview 324 shows the user's face, which in this case is alternatively designated for placement within target box 328, rather than being designated as containing reference feature 326. In addition, display 320a also includes detection points 321 and bounding box 323 generated by processor 310. Processor 310, controlled by instructions from the application, can detect and identify certain facial features in live action preview 324. These can be displayed to the user using detection points 321 and bounding box 323. This can help the user understand that certain conditions, such as lighting, are satisfactory for image capture, as this indicates that processor 310 is identifying these facial features with detection points 321 and confining them to bounding box 323.

[0190] Planar Alignment and Reflection Image 430

[0191] like Figure 7CAs depicted, the user may position themselves or their face and computing device 230 in front of mirror 330 such that the user's facial features and display interface 320 are reflected back to sensor 340. The user may also be instructed by processor 310 via display interface 320 through audible instructions via a speaker of computing device 230, or by a tutorial in advance, to position display interface 320 in the plane of the facial features for measurement. For example, the user may be instructed to position display interface 320 so that it faces forward and is placed under, against, or adjacent to the user's chin in a plane aligned with certain facial features to be measured. For example, display interface 320 may be placed in plane alignment with the bridge of the nose point and the supramental point. Since the final captured image is two-dimensional, plane alignment 430 helps ensure that the scaling of reference features 326 is equally applicable to facial feature measurements. In this regard, the distance between mirror 330 and both the user's facial features and the display will be approximately the same.

[0192] Detecting reference features 440

[0193] When the user is positioned in front of the mirror 330 and the display interface 320 including the reference feature 326 is roughly placed in plane alignment with the facial feature to be measured, the processor 310 checks certain conditions to help ensure adequate alignment. As previously described, one exemplary condition that can be established by the application is that the entirety of the reference feature 326 must be detected within the target box 328 in order to proceed. If the processor 310 detects that the reference feature 326 is not completely within the target box 328, the processor 310 can prohibit or delay image capture. The user can then move their face along with the display interface 320 to maintain planarity until the reference feature 326 is within the target box 328 as shown in the live action preview. This helps to optimize the alignment of the facial features and the display interface 320 relative to the mirror 330 for image capture.

[0194] Reading IMU and Tilt Alignment 450

[0195] When the processor 310 detects the entire reference feature 326 within the target box 328, the processor 310 can read the IMU 342 of the computing device for detecting the device tilt angle, as long as the computing device 230 includes the IMU 342. For example, the IMU 342 may include an accelerometer or a gyroscope. Therefore, the processor 310 can evaluate the device tilt, such as by comparing with one or more thresholds, to ensure that it is within a suitable range. For example, if it is determined that the computing device 230 and therefore the display interface 320 and the user's facial features are tilted in any direction within approximately + / -5 degrees, then the process can proceed to the capture stage 500. In other embodiments, the continuous tilt angle can be within approximately + / -10 degrees, + / -7 degrees, + / -3 degrees or + / -1 degree. If excessive tilt is detected, a warning message may be displayed or issued to correct the undesirable tilt. This is particularly useful for assisting users to help prohibit or reduce excessive tilt, especially in the front-to-back direction, which may constitute a source of measurement error if not corrected because the captured reference image will not have an appropriate aspect ratio.

[0196] Capture stage 500

[0197] Capture Start 510

[0198] When the processor 310 as controlled by the application determines alignment, the processor 310 enters the capture phase 500. This phase 500 preferably occurs automatically once the alignment parameters and any other prerequisite conditions are met. However, in some embodiments, the user can initiate capture in response to a prompt to initiate capture.

[0199] Capture 'n' images 520

[0200] When image capture is initiated, the processor 310 captures a number n of images, preferably more than one image, via the sensor 340. For example, the processor 310 via the sensor 340 may capture approximately 5 to 20 images, 10 to 20 images, or 10 to 15 images, etc. The number of images captured may be time-based. In other words, the number of images captured may be based on the number of images of a predetermined resolution that can be captured by the sensor 340 during a predetermined time interval. For example, if the image sensor 340 can capture 40 images at a predetermined resolution within 1 second, and the predetermined time interval of capture is 1 second, then the sensor 340 will capture 40 images for processing by the processor 310. The number of images may be user-defined, determined by the server 210 based on environmental conditions detected by artificial intelligence or machine learning or based on the expected accuracy target. For example, if high accuracy is required, more captured images may be required. Although it is preferred to capture multiple images for processing, one image can be expected and can be successfully used to obtain accurate measurements. However, more than one image allows an average measurement to be obtained. This can reduce errors / inconsistencies and improve accuracy. The image may be placed by processor 310 in stored data 354 of memory / data storage device 350 for post-capture processing.

[0201] Post-capture image processing stage 600

[0202] Once the image is captured, the image is processed by the processor 310 to detect or identify facial features / landmarks and measure the distances therebetween. The resulting measurements can be used to recommend an appropriate patient interface size. The processing can alternatively be performed by the server 210 receiving the transmitted captured image and / or on the user's computing device (e.g., a smart phone). The processing can also be performed by a combination of the processor 310 and the server 210. In one instance, the recommended patient interface size can be based primarily on the user's nose width. In other instances, the recommended patient interface size can be based on the user's mouth and / or nose size.

[0203] Reading an image 610 and detecting facial features 620

[0204] The processor 310, as controlled by the application, retrieves one or more captured images from the stored data 354. The processor 310 then extracts the image to identify each pixel comprising the two-dimensional captured image. The processor 310 then detects certain pre-specified facial features within the pixel formation. For example, the detection may be performed by the processor 310 using edge detection, such as Canny, Prewitt, Sobel, or Robert's edge detection. These edge detection techniques / algorithms help identify the locations of certain facial features within the pixel formation that correspond to the actual facial features of the patient presented for image capture. For example, the edge detection technique may first identify the user's face within the image, and also identify the pixel locations within the image that correspond to specific facial features, such as each eye and its border, the mouth and its mouth corners, the left and right nose wings, the bridge of the nose point, the supramental point, the glabella, and the left and right nasolabial grooves, etc. The processor 310 may then mark, flag, or store the specific pixel location or locations for each of these facial features. Alternatively, or if such detection by processor 310 / server 210 is unsuccessful, pre-specified facial features may be manually detected and marked, labeled or stored by a human operator, with access to the captured images viewed via a user interface of processor 310 / server 210 .

[0205] Measuring distances between facial features 630

[0206] Once the pixel coordinates of these facial features are identified, the application controls the processor 310 to measure the pixel distance between certain identified features. For example, the distance can generally be determined by the number of pixels of each feature and can include scaling. For example, the measurement between the left and right nose wings can be used to determine the pixel width of the nose and / or the distance between the bridge of the nose point and the supramental point can be used to determine the pixel height of the user's face. Other examples include the pixel distance between each eye, between the corners of the mouth, and between the left and right nasolabial grooves, in order to obtain additional measurement data for specific structures such as the mouth. Further distances between facial features can be measured.

[0207] Apply anthropometric correction factor 640

[0208] Once the pixel measurements of the pre-specified facial features are obtained, one or more anthropometric correction factors can be applied to the measurements. It should be understood that the correction factors can be applied before or after the scaling factors are applied, as described below. Anthropometric correction factors can correct errors that may occur in the automated process, which can be observed to always occur between patients. In other words, without the correction factors, the automated process alone may result in consistent results between patients, but the results may result in a certain number of patient interfaces with mismatched sizes. The correction factors can be empirically extracted from population tests to convert the results into closer to true measurements, thereby helping to reduce or eliminate size mismatches. Over time, the correction factors can be refined or improved in accuracy because the measurement and sizing data of each patient is transmitted from the corresponding computing device to the server 210, where such data can be further processed to improve the correction factors. Anthropometric correction factors can also vary between forms of patient interfaces. For example, the correction factor for a particular patient seeking an FFM may be different from the correction factor when seeking a nasal mask. Such correction factors can be derived from tracking mask purchases, such as by monitoring mask returns and determining the size difference between replacement masks and returned masks.

[0209] Measuring reference features 650

[0210] In order to apply the facial feature measurements to the patient interface size adjustment, whether corrected or uncorrected by the anthropometric correction factor, the measurements can be scaled from pixel units to other values ​​that accurately reflect the distance between the patient's facial features as presented for image capture. The reference feature can be used to obtain one or more scaling values. Therefore, the processor 310 similarly determines the size of the reference feature, which can include pixel width and / or pixel height (x and y) measurements (e.g., number of pixels) of the entire reference feature. More detailed measurements of the pixel dimensions of many squares / points including the QR code reference feature 326 and / or the pixel areas occupied by the reference feature and its components can also be determined. Therefore, each square or point of the QR code reference feature 326 can be measured in pixels to determine the scaling factor based on the pixel measurement of each point, and then averaged between all the squares or points measured, which can improve the accuracy of the scaling factor compared to a single measurement of the full size of the QR code reference feature 326. However, it should be understood that no matter what measurement is taken for the reference feature, these measurements can be used to scale the pixel measurements of the reference feature to the corresponding known dimensions of the reference feature.

[0211] Calculate the scaling factor 660

[0212] Once the measurements of the reference features are acquired by the processor 310, the scaling factor is calculated by the processor 310 as controlled by the application. The pixel measurements of the reference features are related to the known corresponding dimensions of the reference features, such as the reference features 326 for image capture displayed by the display interface 320, to obtain a conversion or scaling factor. Such scaling factors can be in the form of length / pixel or area / pixel^2. In other words, the known dimension(s) can be divided by the corresponding pixel measurement(s) (e.g., one or more counts).

[0213] Apply scaling factor 670

[0214] The processor 310 then applies a scaling factor to the facial feature measurements (number of pixels) to convert the measurements from pixel units to other units to reflect the distance between the actual facial features of the patient for which the mask sizing is applicable. This may generally involve multiplying the scaling factor by the number of pixels of the distance or distances of the facial features associated with the mask sizing.

[0215] These measurement steps and calculation steps for facial features and reference features are repeated for each captured image until each image in the set has scaled and / or corrected facial feature measurements.

[0216] Average the facial feature measurements 680

[0217] Processor 310 may then optionally average the corrected and scaled measurements of the set of images to obtain a final measurement of the patient's facial anatomy. Such a measurement may reflect the distances between the patient's facial features.

[0218] Comparison and output stage 700

[0219] In the comparison and output stage 700, the results from the post-capture image processing stage 600 may be output (displayed) directly to an interested person or compared to one or more data records to obtain an automatic recommendation for patient interface size.

[0220] Showing average results 710

[0221] Once all measurements are determined, processor 310 may display the results (eg, averages) to the user via display interface 320. In one embodiment, this may end the automated process. The user / patient may record the measurements for further use by the user.

[0222] Average forwarding result 720

[0223] Alternatively, the final measurements may be forwarded automatically or at the user's command from computing device 230 to server 210 via communications network 220. Server 210 or an individual at the server may perform further processing and analysis to determine an appropriate patient interface and patient interface size.

[0224] Compare results 730, select and display recommended size 732

[0225] In further embodiments, the final facial feature measurements reflecting the distances between the actual facial features of the patient are compared by the processor 310 to patient interface size data, such as in a data record. The data record may be part of an application for automatic facial feature measurement and patient interface sizing. The data record may include, for example, a lookup table accessible by the processor 310 that may include patient interface sizes corresponding to a range of facial feature distances / values. Multiple tables may be included in the data record, many of which may correspond to specific forms of patient interfaces and / or specific models of patient interfaces provided by manufacturers.

[0226] The processor 310 compares the user's measurements to determine an appropriate size or "best fit," such as by identifying one or more ranges within which the measurements fall, and then selecting an interface size, such as from a set of standard sizes (e.g., small, medium, or large, etc.) associated with the identified one or more ranges. The processor 310 may then recommend the identified patient interface size in the form of a display presented on the display interface 320. The processor 310 may even automatically forward the recommendation for the user's record via email, text message, or instant messaging. The user may further have the option provided to it via the processor 310 to order a patient interface of the recommended size. The order may be transmitted to the server 210, where additional steps for order delivery may be performed.

[0227] Alternatives and additional application features

[0228] The following describes additional and / or alternative features that can be implemented with the above-described exemplary methods, systems, and apparatus. The following description is not intended to be exhaustive, but merely an example of the many variations that can be implemented consistent with the technology of the present invention.

[0229] Alternative computing devices

[0230] In the above example, the method is at least partially performed by a mobile device as a computing device 230. However, facial feature measurement and / or interface resizing can be performed, for example, at least partially with a desktop or laptop computer 232. In this example, the image capture and image post-processing stages 500, 600 will be similar to those described above. However, the method can be different in the pre-capture stage 400. In this stage, instead of displaying the reference feature 326 on the display interface 320, the display interface 320 and the user's facial features are positioned in front of the mirror 330 so that the reference feature 326 is captured in the image, which can be printed on a piece of paper with a known scale and held by the user. Therefore, the webcam 233 can display the user's live action preview and the reference feature on the display interface (which can be a monitor) via the processor 310. A target box similar to that described above will also be displayed, in which case the user can reposition themselves and the reference feature so that the reference feature or the user's face is positioned within the target box.

[0231] Alternative reference features

[0232] Although the exemplary reference feature 326 displayed by the display interface 320 described above may include a QR code or other processor-detectable reference feature, other reference features of known size displayed by the display interface 320 or positioned near the user may be utilized. When the reference feature is not a display feature captured by the mirror 330, the reference feature may be a known physical feature suggested by the application and located near the user. For example, a paper, coin, credit card, or cardboard cutout with a QR code superimposed thereon may be used as a reference feature. Although such a close-positioned reference feature may be used in the method of implementing the mirror 330, such a reference feature may be particularly useful in the desktop computing device method described directly above. In one example, a user may pick up a credit card of known size (standard), so that the processor may detect the card as a reference feature in order to determine the scaling factor. In another embodiment, a paper of known size, such as an A4-sized paper, may be kept in a horizontal or vertical orientation. The processor may detect the corner and / or aspect ratio of the paper via the sensor 233 in order to determine when it is aligned with the alignment box as a prerequisite for image capture and / or determine the scaling factor from one or more images including a paper reference feature of known size. Another implementation using a coin as a reference feature will be described in more detail below.

[0233] coin

[0234] Figure 8The present invention includes a flowchart showing a method 800 that can be used in place of process 600 to implement the post-capture image processing stage in an implementation where coins are used as reference features. In such an implementation, the pre-capture stage 400 is not required. As described above, prior to the image capture stage 500, such as Fig.13A As shown, the coin 1300 is placed on the user's forehead 1310, approximately above the nose. Under normal skin conditions, the coin will adhere to the user's forehead 1310 long enough to complete the image capture phase 500. The method 800 is then performed to implement the post-capture image processing phase, and finally the comparison and output phase 700 is performed as described above.

[0235] Method 800 begins at step 810, which is the first step in looping over all images captured during the image capture phase. Step 810 reads the next captured image. Step 820 detects facial features (eyes and mouth) in the captured image. Fig. 9 Step 820 is described in more detail. Step 830 detects the reference feature (coin) in the captured image. Fig.10 Step 830 is described in more detail. Step 840 uses the reference features to calculate the scaling factor. Fig.11 Step 840 is described in more detail. The next step 850 measures the facial features detected in the captured image at step 820. Step 850 may alternatively be performed before steps 830 and 840. Fig.12 Step 850 is described in more detail. Step 860 then multiplies the facial feature measurements made at step 850 by the scaling factor calculated at step 840. Step 870 checks whether the end of the captured image group has been reached. If not ("N"), then the method 800 returns to step 810 to read the next captured image. Otherwise ("Y"), a final step 880 averages the scaled facial feature measurements of all captured images to produce a final scaled facial feature measurement ready for use in the comparison and output stage 700 described above.

[0236] Fig. 9A flow chart of method 900 is included that shows a facial feature detection step 820 that can be used to implement method 800 according to one implementation. Method 900 begins at step 902, which detects faces in an image. For example, an OpenCV cascade classifier with an object size threshold can be used to find faces within the image. Each pair of eyes and mouth within each detected face is found using a corresponding cascade classifier. The result of this step 902 is a list of rectangles or "boxes" that define the boundaries of each detected face within the image. For each detected face, a list of boxes defining the location and extent of the eye(s) and mouth(s) found within the face is returned by step 902. Step 902 also returns, for each detected face, a box for each coin found within the face. These boxes can be used by step 830 to detect the correct reference features within the detected face.

[0237] The following steps 905 to 960 are performed for the largest face box detected at step 902. Steps 905 to 960 filter the features detected within the face in order to select only the "best" matches and remove any duplicate / incorrect matches.

[0238] Step 905 checks whether less than two eyes or no mouth were found by face detection step 902. If yes ("Y"), then method 900 ends at step 960. If no ("N"), then at step 902, step 910 checks whether exactly two eyes and one mouth were found. If yes ("Y"), then step 915 stores two eye boxes and a mouth box, and method 900 then ends at step 960. If no ("N"), then at step 902, step 920 checks whether two eyes were found. If yes ("Y"), then step 925 stores two eye boxes, and method 900 proceeds to step 940 described below. If no ("N"), then at step 902, step 930 checks whether one mouth was found. If yes, then step 935 stores a mouth box, and method 900 proceeds to step 940. If no ("N"), then at step 902, step 940 checks whether more than one mouth was found. If yes ("Y"), then step 945 stores the widest mouth box of the plurality of mouth boxes, and method 900 then ends at step 960. Otherwise ("N"), step 950 checks whether a mouth was not found at step 902. If yes ("Y"), then step 955 stores an empty mouth box (with corners (0,0) and (0,0)), and method 900 then ends at step 960. Otherwise ("N"), method 900 ends at step 960.

[0239] In other cascade classifier implementations of step 820, other criteria may be used to filter the boxes found within the largest face box at step 902 in order to return the most likely eye boxes and mouth boxes.

[0240] Fig.10 A flow chart showing a method 1000 that can be used to implement the reference feature detection step 820 of method 800 according to an implementation of method 800, wherein method 900 is used to implement step 820. Steps 1010 to 1090 are performed on the largest face box detected at step 902 of method 900 to identify the "best" detected coin within the face. At step 902, step 1010 checks if no coin was detected within the face. If yes ("Y"), then method 1000 ends at step 1090. Otherwise ("N"), step 1020 checks if exactly one coin was detected at step 902. If yes ("Y"), then step 1030 stores the coin box and method 1000 ends at step 1090. Otherwise ("N"), step 1040 loads a "template image", i.e., a reference image of the reference feature. At step 902, steps 1050 to 1070 are then performed for each of the multiple coins detected in the face. Step 1050 calculates the match measure between the current coin and the reference image. In one implementation, step 1050 counts the number of matching points between the current coin and the reference image. Step 1060 then stores the match measure found at step 1050. Step 1070 checks whether all coins have been matched. If not ("N"), then the method 1000 returns to step 1050. If yes ("Y"), then step 1080 finds the maximum match measure stored at step 1060 and stores the coin box with the maximum match measure. The method 1000 then ends at step 1090.

[0241] Fig.11A flow chart showing a method 1100 that can be used to implement the calculate scaling factor step 840 of method 800 according to one implementation is included. Steps 1105 to 1175 are performed on the "best" coin detected at step 830. A first step 1105 transforms the image contained within the coin box to separate the hue (H) and saturation (S) channels. Step 1110 then calculates a histogram for each of the H and S channels of the box. Then, step 1115 finds the peaks of the H and S channel histograms. Steps 1120 to 1150 are then performed for each pair of peaks in the H and S channel histograms. Step 1120 thresholds the H and S channel images using a threshold value equal to the current corresponding peak position. Step 1130 then detects the image in the thresholded (binary) H and S images. In one implementation, step 1130 uses a Canny edge detection filter on the binary image. Step 1135 then combines the two edge images using a binary AND operation so that the only surviving edge pixels are those detected in both the H and S binary images. The next step 1140 fits an ellipse to the combined edge image. Step 1140 may use any convenient method (e.g., linear regression) to fit an ellipse to the combined edge image. At step 1140, an ellipse is fitted to the combined edge image because a circle rotated in three dimensions through any angle behaves as an ellipse. Step 1145 calculates the coefficient of determination (written as R) of the ellipse fitted at step 1140. 2 ). If the determination coefficient is too low or the number of edge pixels in the combined edge image is too low, then step 1150 removes the ellipse.

[0242] After the last pair of H, S histogram peaks has been processed, step 1155 selects the ellipse with the highest coefficient of determination. Step 1160 then compares the width (horizontal diameter) and height (vertical diameter) of the selected ellipse. If the width is greater than the height ("Y"), then step 1165 sets the measured diameter of the coin to the width of the ellipse (as a number of pixels). Otherwise ("N"), then step 1170 sets the measured diameter of the coin to the height of the ellipse (as a number of pixels). Finally, step 1175 calculates the scaling factor as the true diameter of the coin (a known quantity, such as millimeters) divided by the measured diameter of the coin from step 1170 or step 1165.

[0243] This calculation is based on the fact that when a disk is rotated by any angle, its projected profile appears as an ellipse, and the length of the major axis of the ellipse is the diameter of the disk, regardless of the amount or direction of rotation. This fact demonstrates that circular reference features such as coins are suitable for calculating scaling factors for image measurements.

[0244] Fig.121. A flowchart of method 1200 is included that illustrates a method 1200 that can be used to implement the facial feature measurement step 850 of method 800 according to one implementation, wherein the facial feature measurement is facial height. Method 1200 begins at step 1210, which draws an eye line, i.e., a line between the centers of the two eye boxes selected at step 820. The eye line is illustrated as Fig. 13B 1360, and two eye boxes are shown as 1330 and 1340. Step 1220 determines the midpoint of the eye line. Step 1230 then draws a center line from the midpoint found at step 1220 in a direction vertically downward from the eye line found at step 1210. The center line drawn at step 1230 is at Fig. 13B 1370. Step 1240 then measures the facial height as the length of the midline to the point where the midline intersects the bottom of the mouth box selected at step 820.

[0245] Due to a small rotation of the face about a horizontal axis, the face height measured at step 1240 may be different from the user's actual face height. The ellipse found at step 840 can provide a correction for this small horizontal rotation. The ratio of the true face height (in pixels) to the measured face height from step 1240 is the same as the ratio of the true height of the coin (in pixels) from step 1160 to the measured height of the ellipse, if that height is less than the measured width (indicating that horizontal rotation has occurred). Therefore, step 1250 checks whether the measured height of the ellipse from step 1160 is less than the measured width, also from step 1160. If so ("Y"), then the horizontal rotation can be corrected at step 1260 by multiplying the measured face height by the ratio of the true height of the coin (in pixels) to the measured height of the ellipse from step 1160. The true height of the coin in pixels is simply the length of the major axis of the ellipse. An example of an ellipse for a detected coin is shown in Fig. 13B 1320. In ellipse 1320, the width is less than the height, so there is no need to correct the measured height of the face for horizontal rotation.

[0246] At step 820, an alternative implementation of method 800 uses an active shape model (ASM) 0 to detect facial features. Typically, an active shape model uses identified landmarks within a set of training images to develop a deformable model for the relative positions (shapes) of the landmarks and a statistical model of the pixel values ​​provided by the images at those positions. The deformable shape model is defined as:

[0247]

[0248] where x is the vector of landmark positions (shapes) predicted by the model, is the average shape, P is a set of eigenvectors also determined by the training set, and b is a weight vector (pose) to allow for shape deformation. The statistical model of the pixel values ​​is determined by sampling the range of pixels perpendicular to each landmark in each image in the training set. After sampling, a vector of average pixel values ​​and a corresponding covariance matrix are determined for each landmark. The pixel model for each landmark includes a vector of average pixel values ​​and a corresponding covariance matrix.

[0249] The process of searching for landmarks in the captured image using ASM is iterative. For the first iteration, the average shape is used as the shape estimate x and is superimposed on the captured image in an initial position that may not be too far from the landmarks in the captured image, for example, centered within the captured image. A range of pixel values ​​is sampled at each landmark position in the captured image as defined by the current shape estimate x. A new estimated landmark position is then selected by determining the range of pixels having the minimum Mahalanobis distance (covariance-weighted Euclidean distance) to the average pixel value in the pixel model. The scale, translation, and rotation of the shape estimate x are then updated to best match the new estimated landmark position. The model is then deformed using any remaining error by updating the current pose estimate b. The shape estimate x is then updated using the equation. 1. Repeat these steps until a high proportion (e.g., at least 90%) of the estimated landmark positions have converged within a small radius (e.g., 2 pixels) of the previously estimated position; at this point, the changes in shape scale, translation, rotation, and pose are considered to be minimal. Convergence can be accelerated by repeating the iterative process on a three-level Gaussian pyramid, where each level L contains a vector consisting of 2 L A decimated copy of the captured image. Changes in landmark positions at the coarse level produce large relative movements, and once shape convergence occurs, the iterative process moves up one level, producing finer shape and pose changes. Fig.14 The locations of landmarks identified through the iterative process in a typical captured image 1400 are shown.

[0250] In an implementation of method 800 where an ASM is used to implement step 820, the reference feature detection step 830 may initially detect one or more "coin boxes" using the cascade classifier described above with respect to step 902, and subsequently select the "best" detected coin using the method 1000 described above.

[0251] In an alternative implementation of method 800, facial feature measurement step 850 measures the pixel distance between two selected landmarks suitable for use in resizing the mask. Fig.14 In one example shown for adjusting the size of a nasal mask (see Figure 1B), the selected landmarks are the left and right nose apex 1410 and 1420 (see Figure 3B ), which yields a facial feature measurement of nose width.

[0252] cornea

[0253] In another implementation, the reference feature may be a feature of the eye or eyes, such as the user's iris or cornea. Iris diameter is very similar in all ethnic groups and does not grow after adolescence. 95% of adults have a corneal diameter range (horizontally visible iris diameter) between 11.0 and 12.5 mm [2]. The advantage of using eye features is that it eliminates the need for extraneous reference features such as coins, credit cards, QR codes, etc. to be present in the captured image. In this implementation, method 800 may be implemented as follows. Boundary points of one or each cornea may be detected as part of an ASM implementation of facial feature detection step 820, where the boundary points of each cornea are landmarks in the ASM. Thus, a separate reference feature detection step 830 is not required. Step 840 may calculate the diameter of one or each cornea as the distance between the boundary points of the cornea or each cornea, and calculate a scaling factor as a ratio of an average corneal diameter of 11.75 mm to a measured corneal diameter (in pixels) of one or the other eye, or an average of the two measured corneal diameters. In a variation of step 840, the average corneal diameter is selected based on knowledge of the user's age, since the variation in corneal diameter across people of a given age is less than the variation across the general adult population.

[0254] Assistant Mode

[0255] It is described above that the mirror 330 is implemented to help the user capture images. However, the application for measurement and size adjustment can allow alternatives. For example, the processor 310 controlled by the application can provide the user with the option of using the mirror 330 or a third party to hold the sensor 340 and aim it directly at the expected mask user. Therefore, the expected mask user can have his / her facial features and reference features captured by a relative, friend or fixed bracket holding the sensor 340. Therefore, the application can include a "partner" or "assistant" mode, in which the display interface 320 can present a live camera preview and a target box, such as a camera live action preview 324 and a target box 328. The application can then guide the assistant how to capture a suitable image for mask size adjustment. For example, in this mode, the processor 310 can optionally operate to allow the assistant to select a front camera or a rear camera that can be integrated into a smart phone 234 or a tablet computer 236. Therefore, the assistant can look at the display interface 320 and point the rear camera to the patient's facial features and the reference features that can be held by the patient. A "partner" or "assistant" mode may also provide the user with the option of operating the front camera. Optionally, a stand may be used to hold the mobile device while the user moves relative to the camera 340 until the reference feature held by the user is properly aligned within the target box displayed on the display interface 320.

[0256] Lighting and filters

[0257] It is also described above that if the lighting conditions are not satisfactory, the processor 310 controlled by the application can notify the user that the conditions are not satisfactory and provide instructions to move to a brighter or darker lighting position. However, recognizing that it may not always be possible to meet this condition, the application can control the processor 310 to apply filters to the sensor / camera 340 to allow operation under a wide range of lighting conditions. If the environment is too bright, a specific filter can be automatically applied so that the user's facial features can be easily detected. Conversely, in the case of too dark lighting conditions, another filter can be implemented to detect the desired facial features or the light source of the computing device can be used to illuminate the patient's facial features. This can be performed during the pre-capture phase 400.

[0258] Alternative sensors

[0259] The methods described herein are particularly useful for measurement and resizing based on two-dimensional images, since cameras for two-dimensional images are ubiquitous. However, the disclosed technology contemplates other types of sensors that are capable of determining scaling without the use of reference features such as a laser and detector combination. For example, such a sensor can detect the distance of a target, which can be used to calculate scaling. Other sensors can be used to obtain a three-dimensional rendering. For example, a stereo camera can be used to obtain a three-dimensional image. A process similar to that described above can be performed on the three-dimensional image. In such an embodiment, the reference feature can be a three-dimensional object with known x, y, and z dimensions. Other sensors that can be used are a combination of a strobed light source and a detector. The reflected light from the strobed light source can be timed so that the distance is calculated in a processor.

[0260] Storage media for applications

[0261] In the exemplary method detailed above, an application for automatic facial feature measurement and patient interface sizing is downloaded from a third-party server, such as an application store server, to the internal memory of the computing device. However, in one embodiment, the application may be downloaded to the internal memory / data storage device 350 of the computing device from the server 210 via a network or via some other external computer-readable storage medium, such as a flash drive, SD card, or optical disk. Alternatively, in another embodiment, the application may be stored in an external computer-readable storage medium and directly linked to the processor 310. In further embodiments, the application, including its corresponding processor instructions and data, may be stored on a medium external to the computing device 230, such as the server 210, and downloaded by the computing device 230 via a web browser (e.g., Google or Internet ) and / or web browser extensions (such as Adobe ) to access it.

[0262] Tracking and updates

[0263] The application may include additional functionality, such as tracking the user's weight over time and corresponding facial changes as the user's weight fluctuates, which may be integrated into the patient interface recommendation analysis. For example, weight, age, and / or height may be parameters considered in the data log for selecting an appropriate mask size. In addition, the application may record the specific patient interface that the user / patient ordered and the date the order occurred so that replacement and / or maintenance reminders may be provided to the user via the application and / or for general health monitoring.

[0264] In addition, the application may be updated from time to time, which may be provided to the user via an application store or server 210. These updates may provide the user with new reference features and known size information for the reference features, updated mask adjustment size tables and new mask type information, and other data processing information, such as updated anthropometric correction factors, which may be continuously refined as more and more end users use the application and send their information back through the server 210. Advantageously, the ability to remotely update reference features may enable more accurate measurements to be obtained, or improve the responsiveness of the application as subsequent reference features are developed. Remote updates may be seamless background processes requiring little or no user interaction, such as when the application is loaded and updates are available. Remote updates of the application mean that subsequent versions of the application may also take advantage of more powerful, better, or additional hardware components provided with the latest model of the computing device 230.

[0265] Other image sources

[0266] In the exemplary method described above, the image is captured by a sensor 340, such as a camera located on the same computing device 230 as the processor 310, which executes the application 360 for facial feature measurement and patient interface sizing. However, in other implementations, the processor 310 may receive the image from another source via a network 220 to which the computing device 230 is connected. Examples of other sources include an MMS message, an email, a web page, or other image server, such as the server 210. In other implementations in which the server 210 executes the application 360 for facial feature measurement and patient interface sizing, the server 210 may receive the image from another source via a network 220 to which the server 210 is connected.

[0267] Other notes

[0268] A portion of the disclosure of this patent document contains material which is subject to copyright protection. The copyright owner has no objection to the facsimile reproduction by anyone of the patent document or the patent disclosure, as it appears in the Patent Office patent file or records, but otherwise reserves all copyright rights whatsoever.

[0269] Unless the context clearly dictates otherwise, when a range of values ​​is provided, it should be understood that each intervening value to one tenth of the unit of the lower limit between the upper and lower limits of the range and any other stated or intervening value of the stated range is included in the present technology. The upper and lower limits of these intervening ranges may be independently included in the intervening ranges and are also included in the present technology, and any limit in the stated range may be specifically excluded. When the stated range includes one or both of the limits, ranges excluding one or both of those included limits are also included in the present technology.

[0270] Furthermore, where one or more values ​​are stated herein as being implemented as part of the present technology, it should be understood that such values ​​may be approximate unless otherwise indicated and that such values ​​may be used with any suitable significant figures to the extent possible as may be permitted or required by actual technical implementations.

[0271] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the technology belongs. Although any methods and materials similar or equivalent to those described herein may also be used in practicing or testing the technology, a limited number of exemplary methods and materials are described herein.

[0272] When a particular material is determined to be used for construction of a component, obvious alternative materials having similar properties may be used as substitutes. In addition, unless otherwise specified, any and all components described herein are understood to be capable of being manufactured, and thus may be manufactured together or separately.

[0273] It must be noted that as used herein and in the appended claims, the singular forms "a," "an," and "the" include their plural equivalents unless the context clearly dictates otherwise.

[0274] As used herein, the terms "about," "substantially," and "substantially" are intended to mean that departures from the absolute are included within the scope of the term so modified.

[0275] All publications mentioned herein are incorporated herein by reference in their entirety, so as to disclose and describe methods and / or materials as the subject of these publications. Publications discussed herein are provided to provide only the disclosures before the filing date of the present application. Anything herein should not be construed as admitting that the present technology cannot be authorized before these publications by virtue of priority invention. In addition, the publication date provided may be different from the actual publication date that may need to be independently confirmed.

[0276] The terms "include" and "comprising" should be interpreted as referring to elements, components or steps indicating that the referenced elements, components or steps may be present or utilized in a non-exclusive manner, or in combination with other elements, components or steps not explicitly referenced.

[0277] The subject headings used in the detailed description are included only for the reader's ease of reference and should not be used to limit the subject matter found throughout the disclosure or claims. The subject headings should not be used to interpret the scope of the claims or claim limitations.

[0278] Although the technology herein has been described with reference to specific examples, it should be understood that these examples are merely illustrations of the principles and applications of the present technology. In some cases, terms and symbols may mean specific details that are not required for practicing the technology. For example, although the terms "first" and "second" may be used, they are not intended to indicate any order unless otherwise indicated, but may be used to distinguish different elements. In addition, although the processing steps in the method may be described or illustrated in order, such order is not required. Those skilled in the art will recognize that such order may be modified and / or various aspects thereof may be performed simultaneously or even synchronously.

[0279] It is therefore to be understood that numerous modifications may be made to the illustrative examples and that other arrangements may be devised without departing from the spirit and scope of the present technology.

[0280] Reference Mark List

[0281] Patient 10

[0282] Bed Partner 20

[0283] Capture 40

[0284] Air circuit 50

[0285] Humidifier 60

[0286] Patient Interface 100

[0287] Inflatable chamber 120

[0288] Structure 130

[0289] Vent 140

[0290] Forehead support 150

[0291] Seal forming structure 160

[0292] Connection port 170

[0293] System 200

[0294] Server 210

[0295] Network 220

[0296] Network 222

[0297] Wireless Network 224

[0298] Wireless Link 226

[0299] Computing device 230

[0300] Laptop 232

[0301] Webcam 233

[0302] Smartphone 234

[0303] Tablet computer 236

[0304] Architecture 300

[0305] Processor 310

[0306] Display interface 320

[0307] Display 320a

[0308] Checkpoint 321

[0309] Capture Display 322

[0310] Bounding Box 323

[0311] Live Action Preview 324

[0312] Reference Feature 326

[0313] Status Indicator 327

[0314] Target box 328

[0315] Reference feature image 329 Mirror 330 User control / input interface 331 Sensor 340 IMU 342 Memory / data storage device 350 Directive 352 Stored data 354 Application 360 Pre-capture stage 400 Step 410 Step 420 Step 430 Step 440 Step 450 Image Capture Stage 500 Step 510 Step 520 Post-image processing stage 600 Step 610 Step 620 Step 630 Step 640 Step 650 Step 660 Step 670 Step 680 Output stage 700 Step 710 Step 720

[0316] Step 730

[0317] Step 732

[0318] Method 800

[0319] Step 810

[0320] Step 820

[0321] Step 830

[0322] Step 840

[0323] Step 850

[0324] Step 860

[0325] Step 870

[0326] Step 880

[0327] Method 900

[0328] Step 902

[0329] Step 905

[0330] Step 910

[0331] Step 915

[0332] Step 920

[0333] Step 925

[0334] Step 930

[0335] Step 935

[0336] Step 940

[0337] Step 945

[0338] Step 950

[0339] Step 955

[0340] Step 960

[0341] Method 1000

[0342] Step 1010

[0343] Step 1020

[0344] Step 1030

[0345] Step 1040

[0346] Step 1050

[0347] Step 1060

[0348] Step 1070

[0349] Step 1080

[0350] Step 1090

[0351] Method 1100

[0352] Step 1105

[0353] Step 1110

[0354] Step 1115

[0355] Step 1120

[0356] Step 1130

[0357] Step 1135

[0358] Step 1140

[0359] Step 1145

[0360] Step 1150

[0361] Step 1155

[0362] Step 1160

[0363] Step 1165

[0364] Step 1170

[0365] Step 1175

[0366] Method 1200

[0367] Step 1210

[0368] Step 1220

[0369] Step 1230

[0370] Step 1240

[0371] Step 1250

[0372] Step 1260

[0373] Coins 1300 Forehead 1310

[0374] Ellipse 1320

[0375] Line 1360

[0376] Capture image 1400

[0377] Right nose apex 1410

[0378] Right nose apex 1420

[0379] References cited

[0380] Cootes, TF, Taylor, CJ, Cooper, DH and Graham, J. (1995) Active Shape Models-Their Training and Application. Computer Vision and Image Understanding, 61(1), 38–59.

[0381] Florian Rufer, MD, Anke Schroder, MD, and Carl Erb, MD. White-to-White Corneal Diameter: Normal Values ​​in Healthy Humans Obtained With the Orbscan II Topography System. Cornea, 24(3), 259-261, April 2005.

Claims

1. An automated display method of a user display device for initiating selection of a patient interface according to patient size, the method comprising: receiving image data captured by an image sensor coupled to the user display device, the captured image data including one or more facial features of an intended user of the patient interface; controlling display of the received image data in a camera live action preview on a display of the user display device; On the display of the user display device, in the camera live action preview, display of (a) predetermined reference features, (b) a target box, and (c) one or more status indicators is controlled. 2 . The method of claim 1 , wherein the predetermined reference features are configured to assist the one or more processors in determining when predetermined alignment conditions are met during a pre-capture phase of image processing.

3. The method of claim 1, wherein the predetermined reference features overlay a motion preview of the one or more facial features. 4 . The method of claim 1 , wherein the one or more processors are configured to detect a capture condition for an image capture, the capture condition comprising a reference feature that is completely within the target box. The method of claim 1 , wherein the live-action preview further comprises a movable reference feature image.

6. The method of claim 5, wherein the captured image data includes the one or more facial features of the intended user of the user interface associated with a displayed version of the predetermined reference features displayed on a display interface of the display device.

Images