Method and system for selecting a mask - Patents.com

Abstract

A system for selecting a patient mask for use with a respiratory therapy device, the mask being suitable for providing respiratory therapy to the patient, the system comprising: a processor configured to receive data representing at least one digital image of the patient's face; identify a predetermined reference facial feature appearing in the image, the predetermined reference facial feature being the patient's eye; determine a measurement of the patient's eye in the image; assign a predetermined dimension to the measurement and determine a scaling factor for the image that is a ratio of the measurement to the predetermined dimension; identify another facial feature in the image; determine a measurement of the other facial feature in the image; and calculate a dimension of the other facial feature by using the scaling factor and the measurement of the other facial feature; and a memory for storing mask sizing data associated with the patient mask; the processor is further configured to compare the calculated dimension of the other facial feature with stored mask sizing data associated with the patient mask, and select a mask for the patient depending on the comparison.

Description

[Technical field]

[0001] FIELD OF THEINVENTION The present disclosure relates to methods and systems for selecting a patient mask for use with a respiratory therapy device. [Background technology]

[0002] background The administration of continuous positive airway pressure (CPAP) therapy is common to treat obstructive sleep apnea. CPAP therapy is administered to patients by using a CPAP breathing system that delivers therapy to the patient via a face mask. A variety of mask types are available to patients, including full-face masks, nasal face masks, and under-the-nose masks. Masks are usually available in a variety of sizes to fit different shapes and sizes of faces. Proper fitting of the mask is important to avoid leaks in the CPAP system that can reduce the effectiveness of the therapy. A poorly fitted mask can also be uncomfortable for the patient and result in a negative and painful therapy experience. Similar considerations are also taken into account when providing other pressure therapies (e.g., bi-level pressure therapy) via a mask.

[0003] Masks are often fitted by a medical professional during the prescription of therapy. Often, the patient must visit an equipment provider or physician or sleep laboratory. The fitting process can be a trial and error process and can take a long period of time. Recently, masks can be selected remotely by the patient, for example, via an online ordering store, rather than physically purchasing the mask in an environment where the mask can be professionally fitted. Summary of the Invention [Means for solving the problem]

[0004] Summary of the Invention In a first aspect, the disclosure provides a method of selecting a patient mask for use with a respiratory therapy device, the mask being suitable for providing respiratory therapy to a patient, the method comprising: receiving data representing at least one digital image of the patient's face; identifying predetermined reference facial features appearing in the image, the predetermined reference facial features being the patient's eyes; determining a measurement of the patient's eyes in the image; allocating a predetermined dimension to the measurement and determining a scaling factor for the image (which is the ratio of the measurement to the predetermined dimension); identifying another facial feature in the image; determining a measurement of the other facial feature in the image; calculating a dimension of the other facial feature by using the scaling factor and the measurement of the other facial feature; comparing the calculated dimension of the other facial feature to mask sizing data associated with the patient mask; and selecting a mask for the patient depending on the comparison.

[0005] The measurement of the patient's eye can be a width measurement. The measurement of the patient's eye can be a height measurement. Selecting the mask may include identifying the mask.

[0006] Identifying the patient's eyes in the image may be performed by identifying at least two predetermined facial landmarks in the image associated with the eyes. The at least two predetermined facial landmarks in the image may be the corners of the eyes. The predetermined facial landmarks may be the medial and lateral canthi of the eyes. The eye measurement may be palpebral fissure width.

[0007] The other facial feature may be identified by identifying at least two facial landmarks associated with the other facial feature. The other facial feature may be used to size the mask.

[0008] Determining the measurement of the facial feature may be performed by calculating the number of pixels of the image between at least two facial landmarks in the image associated with the facial feature.

[0009] The step of determining the measurement of a reference feature in the image may be performed by identifying the patient's two eyes in the image and calculating the measurement for each eye and calculating the average measurement for the two eyes.

[0010] The facial landmarks may be anthropometric features of the patient's face identified in the image. The method includes: determining at least one attribute of the digital image; comparing the at least one attribute to a predetermined attribute criterion; and determining whether the at least one attribute satisfies the predetermined attribute criterion; and selecting a patient mask is dependent on the at least one attribute satisfying the predetermined attribute criterion. The at least one attribute may include at least one of the following: at least one of a pitch angle, a yaw angle, or a roll angle of the user's face in the image; a focal length of the image; a depth of the patient's face in the image; and at least one predetermined landmark identified in the image.

[0011] At least one attribute may be a pitch angle, the predetermined angle being between 0 and +6 degrees relative to the plane of the image.

[0012] The method may further include providing feedback related to whether the at least one attribute satisfies the predetermined attribute criteria.

[0013] In some embodiments, the step of calculating a dimension of the other facial feature may be performed for a plurality of images to generate a plurality of calculated dimensions, and the method further includes calculating an average dimension of the other facial feature across the plurality of images; and using the average dimension to compare to the mask sizing data. The average dimension may be calculated across a predetermined number of images.

[0014] Some embodiments include: determining at least one attribute of the digital image; comparing the at least one attribute to predetermined attribute criteria; and determining whether the at least one attribute satisfies the predetermined attribute criteria; and an average size is calculated for images that satisfy the predetermined attribute criteria.

[0015] Some embodiments further include: presenting at least one user question to a user; receiving at least one user response to the at least one user question; and determining a mask category for the patient depending on the received user response.

[0016] The different facial features may be selected from a plurality of facial features depending on the mask category. The mask sizing data associated with the patient mask may relate to a mask of the determined mask category.

[0017] Masks may be defined in mask categories, where different mask categories have different relationships between mask sizing data and facial feature dimensions.

[0018] Another facial feature may be selected from a plurality of facial features, the selection being based on a specified mask category.

[0019] In another aspect, the disclosure provides a method of selecting a patient mask for use with a respiratory therapy device, the mask being suitable for providing respiratory therapy to a patient, the method including the steps of: presenting at least one user question to a user; receiving at least one user response to the at least one user question; determining a mask category associated with the user dependent on the received user response; receiving a digital image of the patient's face; identifying in the image predetermined reference features of the patient's face appearing in the image, assigning dimensions to the reference features in the image, and determining a scaling factor for the image based on the reference features; identifying in the image at least one preselected feature of the patient's face appearing in the image, the at least one preselected feature being selected dependent on the determined mask type category, and calculating dimensions associated with the at least one preselected feature by using a measurement scale; and comparing the calculated dimensions of the preselected features to mask sizing data associated with the patient mask, and selecting the patient mask dependent on the comparison.

[0020] The calculated dimensions of the pre-selected feature may be compared to mask sizing data associated with patient masks of the determined mask type category. Some embodiments determine whether the pre-selected feature appears in the image and provide feedback to the user dependent on whether the pre-selected feature appears in the image.

[0021] In another aspect, the present disclosure provides a method of selecting a patient mask for use with a respiratory therapy device, the mask being suitable for providing respiratory therapy to a patient, the method including: receiving a digital image of a patient's face; determining attributes of the digital image; comparing the attributes to predetermined attribute criteria; and providing feedback to a user related to whether the attributes satisfy the predetermined attribute criteria; identifying, in the image, predetermined reference features of the patient's face appearing in the image, assigning dimensions to the reference features in the image, and determining a measurement scale of the image by using the reference features; identifying, in the image, at least one preselected feature of the patient's face appearing in the image, and calculating dimensions associated with the at least one preselected feature by using the measurement scale; and comparing the calculated dimensions of the preselected feature to mask sizing data associated with the patient mask; and selecting a patient mask dependent on the comparison.

[0022] In another aspect, the present disclosure provides a system for selecting a patient mask for use with a respiratory therapy device, the mask being suitable for providing respiratory therapy to the patient, the system including a processor configured to: receive data representing at least one digital image of the patient's face; identify a predetermined reference facial feature appearing in the image, the predetermined reference facial feature being the patient's eye; determine a measurement of the patient's eye in the image; assign a predetermined dimension to the measurement and determine a scaling factor for the image, the scaling factor being a ratio of the measurement to the predetermined dimension; identify another facial feature in the image; determine a measurement of the other facial feature in the image; and calculate a dimension of the other facial feature by using the scaling factor and the measurement of the other facial feature; and a memory for storing mask sizing data associated with the patient mask; the processor is further configured to: compare the calculated dimension of the other facial feature with stored mask sizing data associated with the patient mask, and select a mask for the patient depending on the comparison.

[0023] The system may include a display for displaying the selected mask to the patient.The system may include an image capture device for capturing digital image data representative of the patient's face.

[0024] In another aspect, the present disclosure provides a software application configured to run on a client device, the software application configured to perform a method of any one of the preceding aspects.

[0025] In another aspect, the disclosure provides a mobile communications device configured to select a patient mask for use with a respiratory therapy device, the mask being suitable for providing respiratory therapy to the patient, the mobile communications device including: an image capture device for capturing digital image data; receiving data from the image capture device representing at least one digital image of the patient's face; identifying a predetermined reference facial feature appearing in the image, the predetermined reference facial feature being the patient's eye; determining a measurement of the patient's eye in the image; assigning a predetermined dimension to the measurement and determining a scaling factor for the image, the scaling factor being a ratio of the measurement to the predetermined dimension; identifying another facial feature in the image; determining a measurement of the another facial feature in the image; calculating a dimension of the another facial feature by using the scaling factor and the measurement of the another facial feature; and a memory for storing mask sizing data associated with the patient mask, the processor further configured to: compare the calculated dimension of the another facial feature with stored mask sizing data associated with the patient mask, and select at least one mask for the patient depending on the comparison; the mobile communications device further includes a user interface for displaying data related to the at least one selected mask.

[0026] In another aspect, the present disclosure provides a method for selecting a patient interface for a patient for use with a respiratory therapy device, the patient interface being suitable for providing respiratory therapy to the patient, the method including the steps of: receiving data representing at least one digital image of the patient's face; identifying predetermined reference facial features appearing in the image, the predetermined reference facial features being the patient's eyes; determining a measurement of the patient's eyes in the image; allocating the predetermined dimension to the measurement and determining a scaling factor for the image, the scaling factor being a ratio of the measurement and the predetermined dimension; identifying another facial feature in the image; calculating the dimension of the other facial feature by using the scaling factor; and using the dimension to select a patient interface for the patient.

[0027] In another aspect, the present disclosure provides a system for selecting a patient mask for use with a respiratory therapy device, the mask being suitable for providing respiratory therapy to the patient, the system including a processor configured to: receive data representing at least one digital image of the patient's face; identify a predetermined reference facial feature appearing in the image, the predetermined reference facial feature being the patient's eye; determine a measurement of the patient's eye in the image; assign the predetermined dimension to the measurement and determine a scaling factor for the image, the scaling factor being a ratio of the measurement and the predetermined dimension; identify another facial feature in the image; calculate a dimension of the other facial feature by using the scaling factor; and use the dimension to select a patient interface for the patient.

[0028] BRIEF DESCRIPTION OF THE DRAWINGS The following description is given by way of non-limiting example only and refers to the accompanying drawings, in which: [Brief description of the drawings]

[0029] [Figure 1] FIG. 1 is a schematic diagram of a respiratory therapy device including a blower for generating a flow of breathable gas, conduits, and a patient interface for delivering the flow of breathable gas to a patient. [Figure 2Ai]1 is an illustration of a full face mask positioned on a face and showing the contact area of ​​the full face mask on the face. [Figure 2Aii] 1 is an illustration of a full face mask positioned on a face and showing the contact area of ​​the full face mask on the face. [Figure 2Bi] 1 is a diagram of a nasal mask positioned on a face and showing the contact area of ​​the nasal mask on the face. [Figure 2Bii] 1 is a diagram of a nasal mask positioned on a face and showing the contact area of ​​the nasal mask on the face. [Figure 3i] 1 is an illustration of an under-the-nose nasal mask positioned on a face and showing the contact points of the under-the-nose nasal mask on the face. [Figure 3ii] 1 is an illustration of an under-the-nose nasal mask positioned on a face and showing the contact points of the under-the-nose nasal mask on the face. [Figure 4] 1 is a schematic diagram of a mobile communication device. [Diagram 5] 1 depicts a basic architecture showing the interaction between a server and a mobile communication device. [Figure 6] FIG. 1 is a diagram showing facial features related to the eyes. [Figure 7] 1 is a flow chart illustrating steps performed in one embodiment. [Figure 8] 13 illustrates alignment of the patient's face with the camera when capturing images for the mask sizing application. [Figure 9] 1 is an illustration of an image of a patient's face displayed on the screen of a mobile communication device during image capture. [Figure 10] 1 is an illustration of an image of a patient's face identifying anthropometric features of the patient's face; [Figure 11A] 1 is an illustration of an image of a patient's face identifying pupil distance; [Figure 11B] 1 is an illustration of an image of a patient's face identifying pupil distance; [Figure 12A] 1 is an illustration of an image of a patient's face identifying various facial landmarks. [Figure 12B]1 is an illustration of an image of a patient's face identifying various facial landmarks. [Figure 13] 13 is an exemplary display of a mask recommendation to a patient. [Figure 14] The head rotation axes including pitch, yaw and roll are shown. [Figure 15] 2 is a flow diagram showing the steps taken to analyze an image to determine whether it satisfies various predetermined criteria. [Figure 16] 16, 16A and 16B show image capture of the patient's face and visual feedback provided to the patient. [Figure 17] 17, 17A and 17B show image capture of the patient's face and visual feedback provided to the patient. [Figure 18] 18, 18A, and 18B show image capture of the patient's face and visual feedback provided to the patient. [Figure 19] 1 is a flow diagram illustrating steps performed according to one embodiment. [Figure 20] 1 is an illustration of an example question displayed on a mobile communication device. [Figure 21] Illustrates the recommended mask as it is displayed to the patient. [Figure 22] 1 shows exemplary masked data scores for various survey questions. [Figure 23] 1 shows the patient's score after completing the questionnaire. [Figure 24] 1 illustrates exemplary critical feature dimensions relevant to fitting a full face mask. [Diagram 25] 1 illustrates exemplary critical feature dimensions relevant to fitting a nasal mask. [Figure 26] 1 illustrates exemplary critical feature dimensions associated with fitting an under-the-nose nasal mask. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0030] Detailed Description A method and system for selecting a patient's mask for use with a respiratory therapy device will now be described with reference to Figures 1-26. A system for selecting a mask is configured to select a patient's mask to be used with a respiratory therapy device. The mask is automatically selected by capturing an image of the patient's face and determining dimensions of various features of the patient's face by using a reference scale. The facial features may be defined between facial landmarks. These dimensions are compared to mask sizing data associated with various masks and mask sizes to automatically identify a suitable mask for the patient.

[0031] Exemplary embodiments are now described in the following text, which includes reference numerals corresponding to features illustrated in the accompanying drawings.

[0032] 1 is a schematic diagram of a respiratory therapy device 20. Respiratory therapy device 20 may be used to provide CPAP (Continuous Positive Airway Pressure) therapy or bi-level pressurized therapy. Respiratory therapy device 20 includes a humidification compartment 22 and a removable humidification chamber 24 that is inserted into and contained by compartment 22.

[0033] The humidification chamber 24 is inserted vertically when the compartment 22 is in an upright position. The compartment 22 has a top opening through which the chamber 24 is introduced into the compartment 22. The top opening may have a lid so that the humidification chamber 24 within the humidification compartment 22 may be accessed for removal for cleaning or filling. However, this is optional and other arrangements may be contemplated. For example, in other embodiments, it is possible that the humidification chamber 24 is inserted horizontally into the humidification compartment 22. Additionally / alternatively, the respiratory therapy device may include a receptacle that includes a heater plate. The humidification chamber 24 is slidable into and out of the receptacle such that a conductive base of the chamber is brought into contact with the heater plate.

[0034] The humidification chamber 24 is fillable with a quantity of water 26 and has or is coupled to a heater base 28. A heater plate 29 is powered to generate heat that is transferred (via the heat transfer plate 29) to the heater base 28 of the chamber 24 to heat the water 26 within the humidification chamber 24 during use.

[0035] The respiratory therapy device 20 has a blower 30 that draws in ambient air and / or other therapeutic gases through an inlet and produces a gas flow 34 at an outlet of the blower 30. Figure 1 shows an arrangement in which the outlet of the blower 30 is directly fluidly connected to a chamber inlet 37 via a connecting conduit 38 and a chamber outlet 36. The chamber inlet 37 and the chamber outlet 36 may have a sealed connection when the humidification chamber 24 is in the operating position.

[0036] The gas stream 34 passes through the humidification chamber 24 where the humidity of the gas stream 34 is increased and exits the humidification chamber through a gas outlet 40. The gas stream is delivered to the patient through a conduit 44, mask, cannula, or similar patient interface 46.

[0037] 1, the chamber outlet 40 is sealed to or fitted with the compartment inlet 41 by a sealed connection. In this embodiment, a lid to the compartment may or may not be provided.

[0038] In the arrangement of FIG. 1, gas flow 34 passes through humidification chamber 24 where the humidity of gas flow 34 is increased and exits via chamber outlet 40. Chamber outlet 40 is sealingly connected to or sealingly mated with compartment inlet 41. It will be appreciated that in alternative embodiments, chamber outlet 40 and compartment inlet 41 need not be sealingly connected by a connector or otherwise sealingly mated therewith. The gas flow is delivered to patient interface 46 via conduit 44. The patient interface may be a mask. The patient interface may include one of the following: a nasal mask, an oral / nose mask, an oral mask, a full face mask, an under-the-nose mask, or other suitable patient interface.

[0039] One or more sensors (not shown in FIG. 1) may be located within the respiratory therapy device 20. The sensors are used to monitor various internal parameters of the respiratory therapy device 20.

[0040] The sensors (not shown) are connected to a control system (not shown) that includes a control unit. The sensors communicate with the control system. The control unit is typically located on a PCB. In one form, the control unit may be a processor or microprocessor. The control system may receive signals from the sensors and convert these signals into measurement data, such as pressure and flow data. In some forms, the control unit may be configured to control and modify the operation of various components of the respiratory therapy device to help ensure that certain parameters (e.g., air pressure, humidity, power output, blower speed, etc.) fall within a desired range or meet a desired range, threshold, or value. Typically, the desired range, threshold, or value is predetermined and programmed into the control unit of the control system. Additional sensors (e.g., an O2 concentration sensor or humidity sensor) may be included in the respiratory therapy device. Another sensor may also include a pulse oximeter to sense the patient's blood oxygen level. The pulse oximeter is preferably mounted on the patient. The pulse oximeter may be connected to the controller by a wired or wireless connection.

[0041] The blower 30 may control the flow of air and / or other gases within the respiratory therapy device. The control system and control unit may be configured to control the state of the blower 30 via sending control signals to the blower 30. The control signals control the speed and duration of operation of the blower 30.

[0042] The control system is programmed with multiple operating states of the respiratory therapy device. Control software for each operating state is stored in memory within the control system. The control system executes the control software by sending control signals to the blower 30 and various other components of the respiratory therapy device to control the operation of the respiratory therapy device and to produce the required operating state.

[0043] The operating states of the respiratory therapy device may include respiratory therapy states and non-respiratory therapy states. Examples of respiratory therapy states include: CPAP (continuous positive airway pressure), commonly used to treat obstructive sleep apnea, where the patient is provided with a pressurized airflow that is usually pressurized to 4-20 cmH20; NIV (non-invasive ventilation), a two-level pressurized therapy used to treat obstructive respiratory diseases such as chronic obstructive pulmonary disease (COPD - including emphysema, refractory asthma and chronic bronchitis); high flow rate; and two-level. Examples of non-respiratory therapy states include: an off state, where the blower is off and thus does not provide any airflow through the respiratory therapy device; an idle state, where the blower is on and provides airflow through the respiratory therapy device but does not provide therapy; and a dry mode, where the blower may be on and repeats a predetermined speed pattern but does not provide therapy. In dry mode, the heater wires in the tubes may be activated to a predetermined level (e.g., 100% power) and the blower may be activated to a pre-set flow rate or motor speed and run for a predetermined period of time (e.g., 30-90 minutes). Dry mode dries the conduits of any liquid or liquid condensate.

[0044] Different airflow conditions within the respiratory therapy device are required for different operating states. The control system provides control signals to the blower 30 to control the blower operating parameters (including activation and speed parameters) to provide the required airflow conditions within the respiratory therapy device.

[0045] A software program is stored in the memory of the control system that defines the operating conditions required for various operating states of the respiratory therapy device. During operation at a particular operating condition, the control system receives signals (e.g., pressure and flow data) from various sensors and components of the respiratory therapy device in the communication module 62 that define the conditions within the respiratory therapy device. The control system 60 (particularly the processor) is configured to compare the conditions within the respiratory therapy device to the predetermined operating conditions of the operating state, and is further configured to control and modify the operation of the various components of the respiratory therapy device to help ensure that the particular conditions (e.g., air pressure and humidity, power output, blower speed, etc.) fall within desired ranges or meet desired thresholds or values ​​associated with the required operating state. The desired ranges, thresholds or values ​​are predefined and programmed into the software program.

[0046] In some embodiments, the respiratory therapy device includes a transceiver for transmitting and receiving radio signals or other communication signals. The transceiver may be a Bluetooth module or a WiFi module or other wireless communication module. The transceiver may be a cellular communication module for communication over a cellular network (e.g., 4G, 5G). In one example, the transceiver may be a modem integrated into the device. The transceiver enables the device to communicate with one or more remote computing devices (e.g., a server). The device is configured for bidirectional communication (i.e., to send and receive data) to one or more remote computing devices (e.g., a server). For example, device usage data may be transmitted from the device to a remote computing device. In another example, therapy settings for the device may be received from one or more remote computing devices. In another example, the respiratory therapy device may include multiple transceivers (e.g., a Wifi module, a Bluetooth module, a modem for cellular communication or other forms of communication).

[0047] In some embodiments, the transceiver may be in communication with a mobile communication device. The patient interface 46 is typically a mask configured for connection to the patient's face. The mask may be held in place on the patient's face by using a headband that stretches around the patient's head. Other suitable means for holding the mask in place (such as adhesive or suction) may also be used. The mask is an essential part of the respiratory system and preferably provides a comfortable (leak-free) delivery of gas to the patient. CPAP masks have bias flow holes to allow inhaled gas to escape the mask. A variety of mask types are available to patients, including full-face masks, nasal face masks, and under-the-nose nasal face masks. Masks are typically available in a variety of sizes to fit faces of various shapes and sizes. Proper fitting of the mask is important to avoid leaks in the CPAP system that may reduce the effectiveness of therapy or reduce the respiratory support delivered through the mask. A poorly fitted mask may also be uncomfortable for the patient and may result in a negative or painful therapy experience, for example by causing pressure sores on sensitive areas of the face. Selecting the correct mask for the patient is crucial to delivering reliable and continuous therapy.

[0048] A number of factors are important when detecting a patient's mask: The first consideration is to select the correct mask category for the patient. Patients breathe in different ways, some patients breathe through their nose, some patients breathe through their mouth, and some patients breathe through a combination of nose and mouth. Optimal respiratory therapy or respiratory support can be provided to the patient by prescribing a mask type suitable for the way the patient breathes. The primary mask categories are: full face masks, nasal masks, and under-the-nose masks. Other types of masks include oral masks (seals around the mouth only), hybrid masks (masks that have nasal pillows to seal around the mouth and with the nostrils), full face mask variations (seals around the mouth and under the nose but not the nasal pillows), and masks that seal at least partially with the mouth and / or at least partially with the nostrils. Each mask functions to create a seal with either the mouth or nose or both to maintain effective delivery of pressure-based therapy (e.g., CPAP). Consideration of which mask a patient should use is influenced by which airway the patient primarily breathes through. This airway is where pressure-based therapy should be delivered to keep the tissue of the main airway open and prevent its collapse. The mask selected seals against the airway and essentially fluidly extends the airway to the therapy device that will assist breathing; for example, the patient will most effectively receive respiratory support if they breathe primarily through their nose (if a nasal mask, under-the-nose mask, or nasal pillows are used to seal and provide pressure to the airway).

[0049] Examples of various mask categories are shown in Figures 2 and 3. Figure 2 shows each mask category on a patient's face and separately shows the contact area of ​​each mask category on the patient's face.

[0050] FIG. 2A shows a full face mask 210A that covers the patient's nose and mouth. The full face mask 210A is held to the patient's face by using headgear. The headgear includes a strap 220A that extends around the patient's chin and / or cheeks and neck, and a second strap 230A that extends around the top of the patient's head. The full face mask seals around the entire mouth and nose area and across the bridge of the nose. As shown in FIG. 2A(ii), seal 240A extends under the patient's mouth, around either side of the nose, and across the bridge of the nose. The soft seal of the full face mask can conform / fit to the variable surfaces around the nose and mouth to create an effective seal to maintain pressure as therapy is delivered.

[0051] FIG. 2B shows a nasal face mask. A nasal face mask is the same as a nasal mask. These terms may be used interchangeably. A nasal face mask covers only the nose and not the mouth. The nasal face mask 210B is held to the patient's face by using a strap 220B that extends around the patient's chin and / or cheeks and neck, and a second strap 230B that extends around the top of the patient's head. The nasal face mask seals around the nose area and across the bridge of the nose. As shown in FIG. 2B(ii), a seal 240B extends around the patient's nose. The seal 240B seals under the patient's nose, under the nostrils and over the mouth, as well as around the sides of the nose and over the bridge of the nose. The soft seal of the nasal face mask can conform / fit to the variable surfaces around the nose to create an effective seal to maintain pressure as the therapy is delivered.

[0052] FIG. 3 shows an under-nose nasal mask. The under-nose nasal mask seals only the nostrils. This is a less invasive way to create a nasal seal than using a nasal mask. The under-nose nasal mask 310C is held to the patient's face by using a strap 320C that extends around the back of the patient's head and a second strap 330C that extends over the top of the patient's head. The under-nose nasal mask seals only around the nose area. As shown in FIG. 3(ii), the seal 340C extends around the patient's nostrils. The soft seal of the under-nose nasal mask can conform / fit to the variable surfaces around the nose to create an effective seal to maintain pressure as the therapy is delivered. The seal is created over a portion of the underside of the patient's nose. The seal 340C can seal around both sides of the nose or around the sides of the nose (e.g., in the area of ​​the armpit crease or around the patient's armpits). The soft seal of the under-the-nose nasal mask can conform / fit to the variable surfaces around the nose to create an effective seal to maintain pressure as therapy is delivered.

[0053] Subnasal full face masks cover the mouth and seal below the nose. Subnasal full face mask sizing uses the Subnasal Nasal Mask Sizing Guide, which uses the Subnasal Nasal Mask Sizing Parameters in combination with the Mouth Width.

[0054] Within each mask category, masks may come in a variety of sizes (e.g., XS, S, M, L). Mask size is typically defined by the seal size (i.e., the size of the mask seal that contacts the face). Generally, patients with larger heads require a larger seal size to provide an optimal or practical seal. Headgear size is also a consideration of effectiveness and comfort, and headgear may also come in a variety of sizes depending on the patient's head size. Some mask categories may also include an XL mask size.

[0055] Another consideration may be made when selecting a mask for a patient, which relates to the patient's sleep habits. A typical prescription for respiratory therapy requires that the patient wear the mask throughout the night while sleeping. Factors including patient movement during the therapy session (e.g., whether the patient is a restless sleeper and also whether the patient wears glasses in bed) are also factors that should be considered when selecting a mask for a patient to optimize the effectiveness of the therapy and the patient's ongoing compliance with the therapy program. Other considerations include safety: a poorly fitted mask may lead the patient to tinker with the fit, settings, etc. Leaks may also be noisy and therefore disruptive to sleep (of the patient and partner). Other considerations may also be taken into account. Some OSA patients may have other health issues.

[0056] When selecting a mask for a patient, the objectives include minimizing leakage between the mask and the face to optimize therapy and also to avoid patient discomfort by avoiding excess pressure around the face-mask contact area. A poor fitting mask or set of masks that does not match the patient's breathing type can affect the effectiveness of therapy, patient comfort, and patient compliance with therapy.

[0057] Typically, masks are fitted by a clinician during patient evaluation. Mask fitting is usually performed by the patient themselves, who can try different mask types and sizes to select the most appropriate mask type and size for the patient under expert guidance. Clinicians are technical experts and therefore have experience in mask fitting for patients.

[0058] Masks are consumable items that have a limited life span at optimal use, and patients typically need to replace their masks every few months. There has been a desire for remote ordering of masks by patients. Additionally, some patients prefer to select a mask without visiting a clinician.

[0059] Recently, mask suppliers have begun to offer patients remote mask selection and remote mask ordering options. These options allow patients to view a catalog of masks, select a mask from the catalog, and order the mask remotely (e.g., over the Internet). One challenge with allowing patients to select a mask is that the mask selected by the patient may not be optimal in terms of mask category or mask fit, since the fitting procedure is not undertaken by a technical expert. As discussed above, a poorly fitting mask or group of masks that does not match the patient's breathing style and / or other sleep factors (e.g., the patient's preferred sleeping position (e.g., side sleepers)) may result in suboptimal therapy and discomfort for the patient. These factors may reduce therapy results and may result in poor patient therapy compliance.

[0060] Automated mask sizing software applications have been developed that collect patient data and recommend a mask to the patient. These may provide improved results compared to independent patient selection of a mask. However, one of the challenges of automated mask selection is the capture of accurate patient facial measurement data to enable the software application to identify a mask that fits the patient. These software applications often require significant processing. Software applications for recommending masks to patients often provide unreliable measurement data or rely on patient expertise or input to retrieve measurements. These factors may result in the recommendation of a suboptimal mask to the patient.

[0061] Another challenge is to make the process easy and fast to use, in addition to providing accurate measurements and sizing. Because patients may not be familiar with the technology or may have limited mobility, there is a need for a simple and intuitive sizing process.

[0062] In one embodiment, a method and system are provided for selecting a patient mask for use with a respiratory therapy device or system. The mask is suitable for delivering respiratory therapy or respiratory assistance to the patient. The method includes receiving data representing at least one digital image of the patient's face. The method identifies a predetermined reference facial feature appearing in the image, the predetermined reference facial feature being the patient's eye. The method determines a measurement of the patient's eye in the image and assigns a predetermined dimension to the measurement. The method determines a scaling factor for the image, the scaling factor being a ratio of the measurement to the predetermined dimension. The method identifies another facial feature in the image, determines a measurement of the other facial feature in the image, and calculates a dimension of the other facial feature by using the scaling factor and the measurement of the other facial feature. The method compares the calculated dimension of the other facial feature to mask sizing data associated with the patient mask, and selects a mask for the patient depending on the comparison.

[0063] Some embodiments provide an accurate measurement system that allows non-technical professionals to accurately and reliably capture the information required for the system to recommend a well-fitting mask. The method can be performed by using non-professional equipment. Some embodiments capture images of the patient's face that allow accurate and reliable sizing to be derived by using a reference scale. The method and system provide a convenient method for mask sizing because a user (e.g., an OSA patient) can perform the method at home without the need to visit a clinician and without the need for any specialized equipment. Furthermore, the sizing method is convenient because it can be performed on the user's mobile device (e.g., a smartphone or tablet). The method and system for mask sizing is also advantageous because there is no need for a separate reference object that needs to be held in front of the patient's face to perform mask sizing.

[0064] In the exemplary embodiment described below, the reference facial feature used to scale the image of the patient's face is the eye. Figure 6 shows a human eye and facial periphery. The eye includes two corners: a first corner 620 located on the face at the innermost point of the eye closest to the center of the face; and a second corner 625 located on the face at the outermost point of the eye furthest from the center of the face. The distance between the corners of the eye is the interpupillary distance.

[0065] These angles may be defined by the two canthi of the eye. The facial landmark associated with the innermost point of the eye is the medial canthus 620. The facial landmark associated with the outermost point of the eye is the lateral canthus 625.

[0066] Eye width is a useful feature to use as a facial reference feature since that dimension has been found to have the least variance among adults (usually aged 16 years or older).

[0067] In one example, eye width is the distance between the corners of the eyes. In other embodiments, the eye width is the distance between the inner canthus 620 and the outer canthus 625.

[0068] In another example, eye width can be defined as the distance of the white area of ​​the eye, where angles 620, 625 are defined as the points of contrast between the white of the eye and the face.

[0069] In another example, eye width is the horizontal distance between the medial canthus 620 and the lateral canthus 625. This distance is the horizontal palpebral fissure 630. The horizontal palpebral fissure is a useful facial feature to use as a reference feature. This feature has been found to have the least variance among individuals aged 16 and above. The horizontal palpebral fissure is fairly consistent between males and females, and is also fairly consistent for various ethnicities. In another example, eye height can be used as a reference feature. Eye height can be defined as the distance between the upper eyelid 650 and the lower eyelid 660 when the eyes are in an open position. Eye height can be the maximum distance between the upper eyelid and the lower eyelid when the eyes are in an open position. This height can be defined as the vertical palpebral fissure 640.

[0070] The interpupillary distance can be detected in an image or video of the patient's face. Because the canthus is a facial landmark rather than a part of the eyeball, such as the iris or pupil, these landmarks are not obscured by the patient's eyelids. Because the canthus is a facial landmark, the interpupillary distance can be captured in an image even when the eye is closed, partially closed, or even during blinking. These landmarks can be detected more easily than the iris and some parts of the eyeball. Detection of some parts of the eyeball, such as the iris or pupil, may also be difficult due to reflections from light sources or due to shadows cast from the eyelids or eyebrows. Some parts of the eyeball may also be obscured by the eyelids. Because the interpupillary distance is a larger length than some other parts that may be used as reference features (e.g., the iris or pupil), any percentage measurement error is likely to be lower than the error of smaller reference features. Similarly, the eye height can be detected in an image or video of the patient's face.

[0071] Another benefit of using eye width or eye height as a reference feature is that measurements can be taken for both eyes of a patient in an image, thereby allowing an average measurement to be calculated. This averaging can also reduce error in the measurements.

[0072] Some embodiments of the present invention provide a method and system for selecting a patient mask for use with a respiratory therapy device. The mask is suitable for delivering respiratory therapy to the patient. The system receives a patient facial image and uses the facial image to select a patient mask. The system extracts dimensions of key features of the patient's face from the image. The system selects a patient interface that will fit various dimensions of the patient's face.

[0073] A facial image is a digital image that includes the patient's face. The method may be implemented on a user device. A software application may be loaded onto the user device (e.g., a mobile phone, tablet, desktop or other computing device). The software application may run exclusively on the user device or may be connected to a server across a communications network.

[0074] In the first exemplary embodiment just described, the method is implemented by a software application running on the mobile communication device. "Terminology": Mobile communication device, mobile communication device, user device and mobile device are used interchangeably.

[0075] A diagrammatic representation of a mobile communication device is shown in Figure 4. The mobile communication device 400 includes an image capture device 405. In the example of Figure 4, the image capture device is a digital camera. The mobile communication device 400 includes a memory 420. The memory 420 is a local memory within the communication device 400. The memory 420 is suitable for storing software applications, algorithms and data for execution on the mobile communication device. Data types include mask data including mask category data and mask sizing data, reference scale and dimension information for facial features and landmarks, image recognition software applications suitable for identifying facial features and landmarks in an image, questions for presentation to a user, and so on.

[0076] The mobile communication device 400 includes a processor 410 for executing software applications stored in the memory 420. The mobile communication device includes a display 430. The display is suitable for presenting information to a user (e.g., in the form of text or images) and also for displaying images captured by the camera 405. The user input device 425 receives input from a user. The user input device may be a touch screen or a keypad suitable for user input. In some embodiments, the user input device 425 may be combined with the display 430 as a touch screen. Other examples of user input devices include a microphone. The microphone receives voice commands or other verbal instructions from the patient.

[0077] The transceiver 415 provides communication connections across a communication network. The transceiver 415 may be a wireless transceiver. The transceiver 415 may support short-range wireless communication (e.g., Bluetooth and / or WiFi). The transceiver 415 also supports cellular communication. Alternatively, multiple transceivers may be implemented, each configured to support a particular communication method (i.e., communication protocol), such as, for example, WiFi, Bluetooth, cellular communication, etc.

[0078] In the examples that follow, the mobile communication device 400 is a mobile phone, but could be a tablet, laptop or other mobile communication device having the components and capabilities described with respect to Figure 4. In some illustrated examples, the mobile communication device is a smartphone.

[0079] The communication paths between the mobile communication device 400 and various servers are shown in Figure 5. In Figure 5, the mobile communication device 400 communicates with a server 515 over a communication network 510. The server 515 accesses and / or communicates with a database 520. The mobile communication device 400 exchanges data with the server 515 and the database 520. The communication device 400 may request data from the server 515 and / or the database 520. The communication device 400 may provide data to the server 515 and / or the database 520. The server 515 and / or the database 520 may provide data to the mobile communication device 400 in response to a request from the mobile communication device and / or may selectively push data to the mobile communication device 400.

[0080] The network server typically provides updates to the mobile communication device 400. The updates may relate to data updates for lookup tables and other databases stored in the memory 420. The updates may relate to the patient interface fitting application, providing changes to the software application to modify or improve the operation of the patient interface fitting application.

[0081] The method of patient interface fitting may be performed on the mobile device 420 or across a distributed computer system. When executed on the mobile communication device, all processing, image capture, data storage and recommendations are performed on the mobile communication device. The application may operate offline without a communication connection to an external server. When the method is performed by using a distributed computing system, the functions performed during the method may be performed on different devices or at different locations. Data may be stored in different locations and retrieved or provided across a communication network. In some examples, the application may be executed entirely on a remote server by using data stored in a remote database in a cloud configuration.

[0082] Data related to the mask selection software application may include: questions in a patient questionnaire presented to the patient during the mask selection process; database data related to responses to questionnaire questions for various mask categories; data related to sizing information relating facial feature dimensions to mask sizes; and general information about the device or mask (e.g., mask instructions, cleaning instructions, FAQs, and safety information). Details of some specific databases used in various embodiments are provided below. The diagram in Figure 5 is for illustrative purposes only, and other implementations may include communication connections between multiple servers and databases.

[0083] Next, the steps performed by the mask selection software application running on the mobile communication device will be described with reference to FIG. 7. In this description, the terms: mask selection software application; mask sizing application; software application; and application are used interchangeably. The mask selection software application is a software program that can be stored in memory 420 and executed by processor 410. The software program is a computer executable program for execution by using processor 410 of mobile communication device 400. The computer program may include a series of instructions that are executed by processor 410 and may include algorithms. The program is executed locally at mobile communication device 400 using data acquired. In the following description, various modules (e.g. facial detection module, face detection module, face mesh module, application, algorithms) may specifically form part of the mask selection software application or may reside as separate computer programs stored in memory 420 that are called by the mask selection software application during execution as required.

[0084] At 710, a mask selection software application is opened on the mobile communication device 400. The mask selection software application is opened for the purpose of recommending a respiratory therapy mask to a patient. The mask selection software application is a software program that can be stored in memory 420 and executed by the processor 410.

[0085] Once the mask selection software application is selected by the patient, the mask selection software application is started at 710. The mask selection software application accesses the camera 405 to capture a digital image of the patient's face by scanning at 715. Preferably, a forward-facing camera on the same side as the display screen of the device is accessed by the mask selection software application. This orientation is typically recognized as the orientation for capturing images in "selfie" mode, so that the patient can see the image on the display screen during image capture. The mask selection software application may provide guidance to the patient (e.g., in the form of text instructions or example images on the display screen 430) to help the patient capture a suitable image. In another example, a rear-facing camera is used for image capture. This may facilitate use of the mask sizing app by the clinician sizing the patient. This allows the patient to have someone assist them in capturing the facial image.

[0086] Since the mask selection software application is configured to be operated independently by the patient, an image of the patient's face may be acquired by holding the mobile communications device away from the patient and pointing the camera at the patient's face (as shown in FIG. 8). Preferably, the image captured by the camera is displayed to the user on a display screen 430 as shown in FIG. 9. Visual guidance may be provided to assist the patient in capturing the image, for example in the form of a frame 910. Further guidance may be presented on the screen, which may include text instructing the user to position their face within the frame.

[0087] During image capture, the application captures a stream of digital image frames. The rate at which frames are captured may vary between applications or devices. The rate at which frames are captured may relate to a clock within the mobile device and may depend on the type of mobile device. In some embodiments, only a single image frame is captured. In such systems, the application may prompt the patient to capture an image (e.g., by providing a button on the screen to take an image). In other embodiments, multiple frames are captured as part of a video in a frame sequence. Individual or multiple frames for analysis may be extracted from the multiple frames. In an exemplary system, the multiple frames are captured automatically. A video image frame or frames are captured at 720 and processed to generate a digital image file of the patient's face. This file may be in any suitable file format (e.g., JPEG). Alternatively, there is no capture per se: i.e., processing may be performed on the image frame itself taken from the image buffer. In such an example, no specific image file (such as jpeg) is generated.

[0088] The mask selection software application includes a facial detection module. The facial detection module is a software program configured to analyze an image file and detect predefined facial landmarks within the image. At 725, the mask selection software application runs the facial detection module on the image. The mask selection software identifies the facial landmarks. In some implementations, an actual JPG is not generated, but rather the software uses a matrix or array of data (e.g., pixel values) and stores this in temporary memory. Preferably, no permanent record of the image is stored or transmitted since the processing is done locally. Images may be cached, processed, and then deleted. This respects the user's privacy and provides the user with confidence that their facial data will not be transmitted.

[0089] In some exemplary embodiments, the face detection module is a machine learning module for face detection and facial landmark detection. The face detection module is configured to identify and track facial landmarks. Preferably, the face detection module operates in real-time and analyzes images generated by the mobile device's camera as they are captured.

[0090] An exemplary face detection module may include a face detection module and a face mesh module. The face detection module enables real-time face detection and tracking of faces. The face mesh module provides machine learning techniques to detect facial features and landmarks on a user's face. The machine learning techniques continuously update their library and use stored data on multiple sampled faces to correct anomalies in the captured images. The face mesh module provides the location of the facial landmarks and provides a coordinate position for each landmark. The landmark positions are provided as a coordinate system. For example, the coordinate system may be a Cartesian or polar coordinate system. The zero or reference point of the coordinate system is preferably located on the patient's face (e.g., at the center of the nose). Alternatively, the reference point (i.e., a point in space (e.g., coordinates) used by the module in determining the location of the facial landmarks and providing location information) may be located away from the face. The face detection module and face mesh module together enable tracking of landmarks and features. They may be two separate programs or may be incorporated into a single program or algorithm. Alternatively, the face detection module and the face mesh module may be separate computer programs (i.e., may be stored in the memory of the mobile communication device), and the processor 410 is configured to execute the programs in this alternative configuration.

[0091] Some example embodiments may be configured to select a predetermined subset of the total facial landmarks detected by the face detection module and to calculate only the dimensions of the features defined by these landmarks. The particular subset of the total facial landmarks may be selected based on the current operation of the mask selection software application, patient input, mask category, or other selection criteria.

[0092] FIG. 10 is an illustration of a patient's face identifying various facial landmarks. Facial landmarks are points on the face. These facial landmarks are anthropometric landmarks of the face, including, but not limited to, the following: a) Inner canthus b) The lateral canthus (i.e., the ectro canthus).

[0093] c) Between the eyebrows d) Bridge of nose e) Nostril point f) Nasal tip lobule g) Nasal tip h) Left axilla (axillary lobule) i) Right axilla (axillary lobule) j) Under the nose k) Left labial commissure (i.e., left corner of the mouth) l) Right labial commissure (i.e., right corner of the mouth) m) sublabial n) Mandibular point o) Jaw p)Eye spot A facial feature is defined by facial landmarks. For example, a facial feature may be located between facial landmarks. A dimension of a facial feature may be defined as the distance between facial landmarks. For example, the facial feature of nose width may be defined between the left axillary lobule and the right axillary lobule (landmarks h and i in FIG. 10). The nose width may be calculated as the distance on the face between the left axillary lobule and the right axillary lobule. The nose width may be calculated once the coordinates of the left axillary lobule and the right axillary lobule are known.

[0094] At 725, the application identifies predetermined facial landmarks in the image captured by the patient device. The application applies a coordinate system onto the digital image of the patient's face. In one exemplary embodiment, the coordinate system is a three-dimensional coordinate system (x,y,z). In one implementation, the center of the nose is set as coordinate (0,0,0), and the coordinates of all landmarks are determined relative to the (0,0,0) point.

[0095] As shown in Figure 11A, the application identifies the medial canthus 1110 and lateral canthus 1120 (i.e., the two corners of the patient's eye) in the image of the patient's face. The x, y, z coordinates of the medial and lateral canthus (lateral canthus (x1, y1, z1) and medial canthus (x2, y2, z2)) are identified.

[0096] Next, a measurement of the interpupillary distance 1130 reference feature is calculated within the image, as shown in Figure 1 IB. In this exemplary embodiment, the interpupillary distance measurement is calculated using only the x and y coordinates, and the z coordinate is ignored. In other embodiments, the z coordinate may also be used in calculating this measurement.

[0097] In this exemplary embodiment, the pupil distance measurement is calculated between the canthi by using the following formula:

[0098]

number

[0099] This measurement is the length of the feature in the image. The unit of measurement may be image pixels. Other units for the measurement (e.g. image vectors) may be used. Calculations based on only two dimensions (x and y coordinates) may be useful as they save computation.

[0100] Another embodiment calculates the interpupillary distance measurement by using the x coordinate of only the canthus. In these exemplary embodiments, the interpupillary distance measurement is calculated by using the formula |x1-x2| or |x2-x1|. In some embodiments, it may be useful to use more than one of the x, y and z coordinates to account for any non-standard positioning of facial features.

[0101] The application may calculate the width of one eye in the image in step 730 as described above. In another embodiment, the application identifies the corners of both eyes of the patient's face appearing in the image. A width measurement is calculated for each eye and averaged to obtain an average interpupillary width for the patient in the image. The use of an average width across both eyes may reduce errors.

[0102] At 735, a scaling factor for the image is calculated. Memory 420 stores reference dimensions associated with the eyes. As discussed above, interpupillary width is a useful reference feature since it shows the least variance across adults. This dimension is the size of the facial feature of the patient. Some exemplary embodiments use a reference dimension for interpupillary width that is 28 mm. The reference dimension may relate to the average interpupillary width of the human eye (i.e., horizontal palpebral fissure). A different reference dimension for eye height may be used (e.g., 10 mm), which corresponds to the average eye height (i.e., vertical palpebral fissure). In the sizing method shown and described, interpupillary width is used. Some other embodiments may select an alternative reference dimension for interpupillary width (e.g., 29 mm).

[0103] The application calculates a scaling factor for the image by using the interpupillary distance measurement in the image and an interpupillary distance dimension of 28 mm. The scaling factor is the ratio of the width measurement in the image to the width dimension. As discussed above, the width measurement may be in pixels or some other suitable units.

[0104] 12A, at 740, facial landmarks are identified in the image by a facial detection module and the coordinates (x, y, z) of each facial landmark in the image are determined. A processor of the mobile device is configured to receive the image coordinates for each identified facial landmark. The anthropometric landmarks of interest may be a preselected subset of the total anthropometric landmarks identified in the image.

[0105] 12B, measurements of the preselected facial features are calculated by identifying two anthropometric landmarks associated with each preselected facial feature and determining the length between the landmarks in the image. This measurement can be the difference in just the absolute value of the x coordinate (e.g., x1-x2) or just the absolute value of the y coordinate (y1-y2). The horizontal or x dimension can be obtained by determining the difference between the x coordinates, and the vertical or y dimension can be obtained by determining the difference between the y coordinates (as previously described). Alternatively, the measurement between the landmarks can be calculated using the formula

[0106]

number

[0107] Some example embodiments may calculate the measurements by using two or three dimensions. Again, the measurements may be calculated in pixels or any other suitable units of measurement.

[0108] In FIG. 12B, the arrows indicate various facial feature measurements that can be calculated. The z dimension can be used to calculate, for example, nasal depth (e.g., z distance between the subnose and tip of the nose). The nasal feature measurements can be calculated in pixels or in some other measure (e.g., image vectors). The z dimension can only relate to a specific mask category (e.g., the subnose mask shown in FIG. 3). The z depth measurement |z2-z1| or |z1-z2| can be calculated in the image and converted to the patient's facial dimensions by using the same scaling factor derived from the interpupillary distance as previously described.

[0109] In 745, the facial measurements in the image (i.e., in a number of pixels) are converted to facial dimensions by using a scaling factor of the image calculated with respect to the interpupillary distance dimension. For example, by using 28 mm as the interpupillary distance dimension:

[0110]

number

[0111] Optionally, each of the measurements may be multiplied by a scaling factor, which may be a predetermined suitable scalar, in some embodiments, the scaling factor may compensate for the fisheye effect of the camera lens and / or other distortion factors.

[0112] Feature identification and size calculations may be calculated from a single image. In another embodiment, multiple images, each a separate image frame, may be captured by a camera and processed. In each image, a size may be calculated for each feature, and the final calculated size of the feature on the patient's face is the average size across the multiple images to reduce error.

[0113] The face detection module may be pre-programmed to capture a minimum number of frames to calculate an overall average dimension. In one exemplary embodiment, at least 30 frames are captured and / or processed. In another example, at least 100 frames are captured and / or processed. The face detection module may be pre-programmed to require that "data be captured for a minimum length of time (e.g., 10 seconds of video (i.e., 10 seconds of x,y,z data for facial landmarks) be captured and processed"). The measurements are then averaged across the captured frames.

[0114] To manage memory storage space, frames or patient images may not be stored in memory (i.e., none persist). Frames are stored for a time to process and then deleted. Temporary memory can be ROM, RAM, and optionally some temporary cache memory.

[0115] Processing may occur in real-time on the mobile communications device. In one exemplary embodiment, the processor processes frame-by-frame in real-time on the mobile communications device. In an alternative embodiment, multiple frames are stored and then processed in batches, e.g., frames from a period of recording or from a predefined number of frames are stored and processed on the phone. Additionally / alternatively, the captured video / images are transmitted and processed on a cloud server. Another alternative is that each frame is captured and transmitted to the cloud for processing.

[0116] As mentioned above, the face detection module may include a machine learning (ML) module. The machine learning module is configured to apply one or more deep neural network models. In one example, two ML models are used. A first face detection module operates on images (or frames of video) for real-time face detection and face tracking. A second face mesh module detects facial features and facial landmarks and provides locations of the facial landmarks. The face mesh model may operate on the identified locations to predict and / or approximate the surface geometry by regression.

[0117] The face detection module uses two ML models to identify facial features and landmarks. The identified facial features can be displayed on the screen. These facial features can be used as part of processing the recorded images (or processing each frame of a recording). Landmarks can be identified and tracked in real time as the patient may move. The ML models use known facial geometry and facial landmarks to predict the location of landmarks in the images.

[0118] Once calculated at 745, the dimensions are compared to mask data stored in a database to identify a suitable mask for the patient. A mask size corresponding to the facial feature dimensions is recommended to the patient at 750. An example of a recommended mask displayed to the patient is shown in FIG. 13. In the example of FIG. 13, the recommended mask is a full face mask (medium size). The application may provide links to purchasing options for the patient. For example, the application may provide links that allow for the purchase of a selected mask and size from a mask retailer or dealer that offers such masks.

[0119] Some methods verify that the camera is correctly positioned to capture an image of the patient's face. The angle between the camera and the patient's face is calculated. For example, if the method is implemented on a mobile communication device (e.g., a handset), the angle may be calculated by using sensors in the phone that also includes the camera. In one example, the sensors may include one or more accelerometers and one or more gyroscopes.

[0120] In some embodiments, the images are analyzed to determine whether attributes of the images satisfy a predefined criterion. If attributes of the images do not satisfy the predefined criterion, measurements from those images are not used to calculate the dimensions of the patient's face. These images may be discarded. This is a filtering step that ignores images where the measurements may be inaccurate, leading to an inaccurate calculation of the dimensions of the patient's face. The predefined criterion is predefined by filtering criteria. The step of analyzing the images to determine whether they satisfy a predefined criterion may be performed after the images have been processed.

[0121] An example of an image attribute is the angle of the patient's head relative to the camera in the image. Other examples of image attributes include: the distance between the camera and the patient's head; the lighting level; the position of the head within the display; and whether all required features are included in the image.

[0122] Figure 14 shows the three axes of rotation of a patient's head. Pitch 1410 is the angle of head tilt up and down. Yaw 1420 is the angle of left and right rotation. Roll 1430 is the angle of side-to-side rotation. The pitch, yaw and roll angles are measured relative to the angle of the camera. The accuracy of calculating the size of features in an image can be affected by variations in the pitch, yaw and roll angles of the image. Images with different angles of pitch, yaw or roll may produce different measurements of some features, and the distance between landmarks of those features may change, and landmarks may appear closer to each other or farther apart than they actually are.

[0123] Figure 15 illustrates steps that may be implemented by the application to determine whether attributes of an image satisfy predetermined criteria. If the attributes of the image satisfy the predetermined criteria, then the image may be used to calculate the patient's facial dimensions. Typically, the steps of Figure 15 are performed in real-time as image frames are captured in step 720 of Figure 7.

[0124] At 1510, an image is captured by the camera and processed (step 1510 is equivalent to step 720 of FIG. 7). At 1520, the application determines the pitch, yaw and roll angles of the patient's head in the image and any other required attributes. In one exemplary embodiment, these attributes are determined in real time.

[0125] Various methods may be used to determine the pitch, yaw and roll angles. In one exemplary method, the application generates a matrix of face geometry. The matrix defines the x, y and z values ​​of points on the face in Euclidean space. The mask sizing application determines the pitch, yaw and roll from the relative changes in the x, y and z Euclidean values ​​as the user's face moves and changes angle. As the user's face moves and changes angle, the coordinates of a landmark or point may be compared to the coordinates of that landmark when the face measured the pitch, yaw and roll at (0,0,0), or the previous angle, or a calibration reference point, to derive new values ​​of pitch, yaw and roll at the changed angle. Pitch, yaw and roll may be measured in +ve and -ve values ​​around various axes that intersect at a common origin. All of the x, y and z points used to measure pitch, yaw and roll are measured relative to a common origin (0,0,0), which may be located, for example, at the bridge or tip of the nose.

[0126] At 1530, the pitch, yaw and roll angles are compared to predefined thresholds stored in memory. These thresholds define the tolerance levels for an acceptable image. The predefined thresholds may be different for pitch, yaw and roll. In one embodiment, the predefined threshold for the pitch angle is 10 degrees in either the +ve or -ve direction. If the pitch angle is greater than 10 degrees in either the +ve or -ve direction, measurements from the image are not used to calculate the patient's facial dimensions.

[0127] A predetermined threshold also applies to yaw and roll. In one example, the predetermined threshold for roll and yaw is greater than 2 degrees in either the +ve or -ve direction.

[0128] The predetermined threshold may vary among embodiments. In one embodiment, the pitch threshold is between 10 degrees in the +ve or -ve direction. In some exemplary embodiments, the pitch threshold is 6 degrees in the +ve or -ve direction. Other thresholds may be used in other embodiments. In some embodiments, the threshold may apply to pitch, yaw, and roll. In other embodiments, the threshold may apply to one or more of pitch, yaw, and roll. There is usually a balance to consider when selecting a tolerance value by selecting a value that is small enough to obtain accurate measurements and dimensional values, but is not so restrictive that it makes it difficult for the patient to capture images that meet the predetermined criteria.

[0129] If the image meets the predetermined threshold criteria at 1530, the patient's facial measurements or dimensions calculated from the image may be used during mask selection at 1540. If the image does not meet the predetermined threshold criteria at 1530, the image is not used in the mask selection process towards the recommendation at step 750 of FIG.

[0130] The filtering process of determining whether an image satisfies a predetermined criterion may occur at various stages, and the timing of calculating the predetermined criterion may be selected based on the processing power of the device, frame rate, or other factors.

[0131] In one embodiment, facial feature dimensions are calculated regardless of whether attributes of the image satisfy predetermined threshold criteria. In such an embodiment, steps 725-745 of FIG. 7 are performed regardless of whether attributes of the image satisfy the predetermined criteria. The present application discards dimensions calculated from images that do not satisfy the predetermined criteria, so that these dimensions are not used in selecting a mask for the patient. In other applications, image attributes are calculated and compared against threshold criteria during image processing immediately after image capture. Images whose attributes do not satisfy the required criteria are discarded after step 720 of FIG. 7, so that dimensions are not calculated using these images.

[0132] By discarding images in real time immediately after image capture in step 720, memory storage and processing loads are reduced. Each frame is evaluated as it is extracted from the video stream or image frame buffer. Alternatively, the system may store all or a predetermined number of frames and then evaluate filtering criteria, such as the image attributes described above. By discarding images with attributes that do not satisfy the predetermined criteria, frames that may give incorrect or inaccurate pupil distance measurements or that may give distorted facial features are not considered in calculating the measurements.

[0133] In some embodiments, the application provides feedback to the patient to ascertain whether attributes of the image or images being captured by the patient meet predefined criteria. The feedback may be visual feedback. The feedback may be a visual indicator. The feedback may be text. Providing feedback to the patient allows the patient to respond to the feedback in real-time to capture images that meet requirements. This may help improve the user experience.

[0134] The feedback can be haptic feedback. The haptic feedback can include vibrations or specific vibration patterns to indicate commands to the user. For example, two short vibrations can mean tilt up and a single short vibration can mean tilt down. Similar haptic feedback of the distance of the phone from the face is provided: for example, three vibrations could mean moving the camera closer to the head and four vibrations could mean moving the camera further away from the head.

[0135] The feedback may be audio feedback. The audio feedback may provide voice commands or sounds to provide instructions to the patient to change the relative orientation or position of the camera with respect to their head. Audio feedback instructions are particularly useful to assist patients with visual impairments.

[0136] Some embodiments include a combination of feedback (e.g., a combination of tactile, visual, and audio feedback). Some embodiments may include a combination of tactile and visual feedback, a combination of tactile and audio feedback, a combination of audio and visual feedback, or a combination of tactile, visual, and audio feedback.

[0137] FIG. 16 shows an example of the orientation of the patient's head 1620 relative to the mobile communication device 1610 during image capture. In the example of FIG. 16, this in-image pitch requirement is met. FIG. 16B is a side view showing the pitch angle of the patient's head relative to the camera. Similar images may be provided to show the yaw and roll angles. The mobile communication device's camera 1640 is on a front surface 1650 of the mobile communication device that includes a display for displaying images captured by the camera. As discussed above, this arrangement allows the patient to see an image of their face during the image capture process. The camera line level is represented as 1630. The plane of the camera (and therefore the plane of the image) is represented as 1670 in FIG. 16B. The associated angle of the patient's head is shown as 1660. In the example of FIG. 16, the patient's head is directly facing the camera, and therefore the angle of the patient's head relative to the camera plane 1670 is near zero. This produces a pitch angle of zero or close to zero. In the example of FIG. 16, the image captured by the camera satisfies the predetermined threshold criterion because the pitch angle is within the threshold. The application provides feedback to the patient confirming that the captured image meets the criteria. This feedback is provided to the patient by presenting a green outline indicator 1680 on the display of the mobile communication device 1610. This color indicator provides an indication to the user that they are using the device correctly and that their face is straight. Text feedback 1690 "Please keep your face inside the box" may also be provided on the screen of the mobile communication device.

[0138] FIG. 17 shows another example of the orientation of the patient's head 1720 relative to the mobile communication device 1710 during image capture. In the example of FIG. 17, the pitch requirement in the image is not met. FIG. 17B is a side view showing the pitch angle of the patient's head relative to the camera. The camera line level is represented as 1730. The plane of the camera (and therefore the plane of the image) is represented in FIG. 17B as 1770. The angle of the patient's head is shown as 1760. In the example of FIG. 17, the patient's head is tilted forward relative to the camera plane 1770. This tilt of the head relative to the camera creates a negative non-zero pitch angle. The patient's head is not facing the camera, and an elevation view of the patient's face appears in the image. In the example of FIG. 17, the pitch angle does not meet the predefined threshold criteria because it is outside the threshold. The application provides feedback to the patient confirming that the captured image does not meet the criteria. This feedback is provided to the patient by presenting a red outline indicator 1780 on the display of the mobile communication device 1710. In the example of Figure 17, further feedback in the form of text on the device's screen is provided to the patient to assist them in capturing a suitable image. Text feedback command 1790 instructs the patient to "hold the phone at eye level."

[0139] FIG. 18 shows another example of the orientation of the patient's head 1820 relative to the mobile communication device 1810 during image capture. In the example of FIG. 18, the pitch requirement is not met in the image. FIG. 18B is a side view showing the pitch angle of the patient's head relative to the camera. The camera line level is represented as 1830. The plane of the camera (and therefore the plane of the image) is represented in FIG. 18B as 1870. The angle of the patient's head is shown as 1860. In the example of FIG. 18, the patient's head is tilted backwards relative to the camera plane 1870. This tilt of the head relative to the camera produces a positive non-zero pitch angle. The patient's head is not facing the camera, and an underside view of the patient's face appears in the image. In the example of FIG. 18, the pitch angle does not meet the predefined threshold criteria because it is outside the threshold. The application provides feedback to the patient confirming that the captured image does not meet the criteria. This feedback is provided to the patient by presenting a red outline indicator 1880 on the display of the mobile communication device 1810. In the example of Figure 18, further feedback to assist in capturing a suitable image is provided to the patient in the form of text on the screen of the device. Text feedback command 1890 instructs the patient to "Hold the phone at eye level."

[0140] 16, 17 and 18 provide illustrations of various pitch angles of the patient's head within the image. Similar calculations can be performed for yaw and roll angles, and the application can provide similar patient feedback of those angles to reposition the phone relative to the face, if necessary.

[0141] Images are processed in real-time during the patient's use of the camera, and patient feedback is provided in real-time. Thus, the system provides guidance to the patient on using the application to help the patient capture usable images for determining facial dimensions. This patient feedback assists non-professional users in capturing images that can be used to obtain precise measurements that can be used to calculate precise dimensions used for mask sizing.

[0142] In another embodiment, one of the attributes of the image frame is the distance between the patient's face and the camera. This attribute is used as a filtering criterion to determine if the image frame is used to calculate the dimensions of the facial features. Preferably, the phone should be held at a predefined distance from the user's face. In one example, the set distance is the focal length or focal distance of the camera. In another example, the set distance is based on a reference feature (i.e., interpupillary distance). The reference feature (which is interpupillary distance) is assigned a reference dimension such as 28mm. The distance between the user's face and the camera (and therefore the phone) can be calculated by using the reference feature dimensions and other searchable measurements (such as the focal length of the camera). Such information can be stored in the device metadata or in images captured by the device. Additionally, the measurements of the reference features can be calculated by the application as they appear in images captured by the device. This measurement can be in pixels. The following formula can then be used to find the distance of the face from the camera by taking the ratio of the above measurements:

[0143]

number

[0144] In one example, the predetermined distance may be a set distance with a tolerance (e.g., 30cm + / - 5cm). Alternatively, the predetermined distance may be defined as a range, e.g., 15cm to 45cm. Visual feedback is provided to the patient to indicate whether the relative position of the camera and the user's face is within the predetermined distance or range.

[0145] As shown in Figures 16A, 17A, 18A, visual feedback is provided in the form of an indicator displayed on the screen as a circle around an image of the patient's face. The indicator (circle around the face) is a first color (e.g. red) if the phone is not held at a predetermined distance or does not meet other required attributes. If the phone is held at a set distance, the indicator (circle) is green indicating that the predetermined attributes are met. This is advantageous as it provides the user with an easy to understand and visual indicator to correctly position the mobile communication device. Furthermore, the visual indicator is advantageous as it provides real-time feedback to correctly position the patient's head and the mobile communication device. Optionally, real-time audio feedback and / or real-time tactile feedback may also be provided. Audio feedback and tactile feedback may optionally be provided in combination with visual feedback presented on the screen of the mobile communication device.

[0146] Another exemplary embodiment collects subjective data from the patient in addition to the patient's facial image data. Some embodiments include a question that is presented to the patient. In one exemplary embodiment, the question is stored in memory. In some exemplary embodiments, the question is presented on a display of the mobile communication device. The patient is prompted to respond to the question by providing a response. In one exemplary embodiment, the response is received via the user input device 425. The question may be a "yes / no" question or a question with predefined response options that is presented to the patient.

[0147] The application presents questions to the patient as part of the mask selection process. The questions are presented in addition to the image capture process described above. The questions are a separate part of the process of data collection during mask selection or a separate part of the process of data processing.

[0148] Patient responses in the form of subjective data as described above are used in selecting a mask category for the patient. The responses from the patient are used to help the application identify which mask is most suitable for the patient. The patient responses may be used in combination with dimensional data calculated from an image of the patient's face to recommend a mask to the patient.

[0149] Next, an embodiment including a patient question will be described with respect to Figure 19. In the following embodiment, the question is presented to the patient when the mask sizing application is activated. The question is presented and a response is received before the application initiates the camera for the image capture process.

[0150] Questions are provided to assist the mask selection software application in recommending an appropriate mask or an appropriate group of masks or mask category for the patient. In the following example, questions are presented to the patient to select a suitable mask type or mask category for the patient. The mask categories include full face mask, nasal mask, sub-nasal mask, and under-the-nose mask. As discussed above, each mask category fits in a different way on the patient's face. Each mask category may address different features of the patient's face.

[0151] At 1910, the mask selection software application is accessed by the patient on the mobile communication device. At 1915, a question is presented to the patient. In one exemplary embodiment, the question is presented on the screen of the mobile communication device. The questions may be presented individually or collectively. FIG. 20 is an illustration of a question presented on the screen of a mobile communication device. The question is presented as text 2010 and asks the patient, "Do you breathe through your mouth?" The user is presented with the response options "Yes" 2020 or "No" 2030. Preferably, the display is a touch screen display and the patient may provide a response by touching the appropriate response text on the display. The response is received by the application at 1920.

[0152] In other embodiments, an audible question is presented to the patient. Voice recognition software on the phone can be used to receive an audio response from the patient. An example of suitable software is Apple's Siri application or Android's Voice Access application. This application can be used to present the question to the patient. Patient responses can be provided by the user via a touch screen, via virtual buttons, or in an audible manner (where the patient can speak their response).

[0153] Multiple questions may be presented in succession, and in one exemplary embodiment all questions are "yes / no" questions, although in some embodiments additional predefined responses may be presented or the patient may be able to provide a separate open text response.

[0154] Different sets of questions may be presented to different patients. In one example, the application poses an initial question at 1915 to determine whether the patient has previously used a Positive Airway Pressure (PAP) device. Different sets or sequences of questions are presented to the patient depending on whether the patient has previously used a PAP device.

[0155] At 1915, patients are asked the following questions: Have you used a PAP device or mask before? The user is presented with the response options “Yes” and “No.” The user response is received at 1920.

[0156] At 1925, the application identifies the patient response and determines which question should be asked next. The following sequence of questions is an example of a sequence of questions that may be presented to the patient depending on whether they answer "yes" or "no" to the question "Have you used a PAP device or mask before?". Questions may be presented sequentially: displaying a single question at a time and waiting for the patient's response before displaying the next question to the patient. Alternatively, questions may be displayed simultaneously or in groups.

[0157] In some exemplary embodiments, if the patient answers "no" to the question "Have you used a PAP device before?", the application presents the patient with the following question:

[0158] [Table 1]

[0159] In some exemplary embodiments, if the patient answers "yes" to the question "Have you used a PAP device before?", the application presents the patient with a different set of questions:

[0160] [Table 2]

[0161] The questions listed above are a combination of yes / no and multiple choice questions. The questions may also include an option to answer "I don't know." This allows a better score to be calculated for patients who do not know the answer to the question, and prevents patients from guessing "yes" or "no" answers. Another embodiment may include a variety of questions. Another embodiment includes an option for the patient to provide a free text response. Another example has no initial question that determines the presentation of subsequent questions. Another example allows the user to update, skip, or change the content of a question, or add another question, as the user progresses through the question-based questionnaire.

[0162] The set of questions may be predefined and fixed, or in another embodiment, the set of questions may depend on the responses provided by the patient, and the application determines which question to pose next based on previous responses.

[0163] Upon receiving a response at 1920, the application determines at 1925 whether another question is required. If yes, another question is presented to the patient at 1915. If not, the patient response is analyzed at 1930. Optionally, the application may not present a single question if the user (e.g., patient) answers yes to the question "Have you used a PAP device before?" If the user answers yes, the application may present a question such as "Please select the mask category you use / have used before." The application may then present available mask categories (e.g., full face mask, nasal mask, under nose mask, etc.).

[0164] In one example, described in more detail below with reference to Figures 22 and 23, each response received by the application is provided with a score and a weighting. An overall score for the patient is calculated. Mask categories are provided with the particular scores, and a mask category recommendation is generated at 1935. In other embodiments, a list of two or more mask categories may be recommended (e.g., in order of suitability). The mask category recommendation may be displayed on the mobile communication device at 1940. Other information may be displayed with the mask recommendation. Examples of other information include an image of a mask and information about the mask (e.g., mask category, or appropriateness of the mask). Figure 21 provides an example of a display identifying that a full face mask is recommended for the patient. The display identifies that the full face mask provides 90% consistency based on the answers provided by the patient.

[0165] FIG. 22 shows an example of a score sheet associated with a series of questions presented to a patient. The questionnaire includes seven questions that are presented to the patient. In the example of FIG. 22, each question has a "yes / no" answer. Patient responses are collected and mapped against three different mask categories (i.e., full face mask, under-the-nose nasal mask, nasal mask). Additional categories and associated mappings of answers may also be included.

[0166] The table shown in FIG. 22 is used to calculate a suitability score for each mask for a particular patient based on the answers to the questions for that particular patient. This step is performed in step 1930 of FIG. 19. Since each mask has different characteristics, each question may have different appropriateness / weighting for different masks. The weighting is represented by different scores assigned to the "yes / no" responses of different masks (as shown in FIG. 22). For example, the nasal mask is suitable for patients who breathe through their nose, so it provides a high score of 5 for a "no" answer to the question asking "Does the patient breathe through their mouth?". The specific scores are generated based on various clinical and other studies, and can be tweaked and recalibrated in the future.

[0167] Some questions may be neutral with respect to the particular mask, in which case the score given for that question will be the same regardless of the patient's response indicating that the question has little relevance / relevance for that particular mask. An example question is question 5: "Do you have trouble dealing with things or wearing your current mask headgear?" The patient will score a "4" regardless of whether the input response is "yes" or "no" for the undernose category as this question has little relevance for that particular category.

[0168] An example of a patient response to the question of Figure 22 will now be described with reference to Figure 23 to show how the mask selection software application uses the patient response to select a mask category for the patient. The patient responses are shown in the table below:

[0169] [Table 3]

[0170] The answer to each question generates a score for each mask category that depends on the suitability of that mask for the response provided by the patient. For example, question 1: Do you breathe through your mouth while you sleep? (Do you wake up in the morning with a dry mouth?). The patient enters the answer "Yes". The answer "Yes" scores a 5 in all mask categories, which is a high score indicating that all face mask categories are suitable for mouth-breathing patients. The answer "Yes" scores a 2 only in the Sub-Nose and Nasal Mask categories, indicating that these masks are less suitable for mouth-breathing patients.

[0171] In this example, Question 6: Do you know your PAP pressure? Is your PAP pressure higher than 10 cmH20? The patient answers "No." This response scores a 4 in each of the mask categories, indicating that "none of the masks is better than the other masks for patients who do not know their PAP pressure." This is an example of a neutral response.

[0172] A patient's mask score based on the responses provided is calculated for each mask category. In the example shown in FIG. 23, the patient's highest scoring mask category is full face mask. The lowest scoring mask category is subnasal mask. These scores indicate that the patient's most preferred mask category is full face mask. After the patient's mask category has been determined, as discussed above, the mask categories may be displayed to the patient at 1940. FIG. 21 shows an example of a display screen presenting the mask categories to the patient.

[0173] In one exemplary embodiment, the questionnaire is presented to the patient in a first stage of the mask selection process. After a response is received by the application, the application enters a second stage of the mask selection process at 1945, capturing an image of the patient's face and calculating the patient's facial dimensions. The second stage of the mask selection process follows many of the steps described above in connection with FIG. 7. Typically, the first stage of the mask selection process, which presents the questions to the patient, involves selecting the most suitable mask category. The second stage of the mask selection process involves sizing the mask for the patient and selecting the most appropriate size mask within the preferred mask category.

[0174] Once image capture and analysis is complete in step 1945 of FIG. 19, the application uses the questionnaire data and image data to select and recommend a mask to the patient in 1950.

[0175] Various mask categories contact the face at various points on the face, as shown in Figures 2 and 3 and described above. Consequently, various facial dimensions are important when fitting masks of various categories. In some cases, some facial dimensions may be more dominant than others during fitting, or some dimensions may not be required.

[0176] The patient responses are used to identify which mask categories will be included in the mask sizing. The following paragraphs provide examples of facial dimensions that may be important for various mask categories. After determining the patient's most suitable mask category, some exemplary embodiments of the present application calculate the dimensions of facial features that are important for the determined mask category and use these dimensions to select the size of a mask within the determined category.

[0177] FIG. 24 shows a seal 2420 between the mask and the face for a full face mask. For a full face mask, exemplary critical feature dimensions for sizing are shown in FIG. 24. The first critical dimension is the dimension 2430 from the bridge of the nose to the lower lip. With reference to FIG. 10, this is the dimension from landmark (d) bridge of the nose to landmark (m) bottom of the lip. The second critical dimension is the width of the mouth 2450. With reference to FIG. 10, this is the dimension between landmark (k) left lip commissure and landmark (l) right lip commissure. The third critical dimension is the width of the nose 2440. With reference to FIG. 10, this is the dimension between landmark (h) left axilla and landmark (i) right axilla.

[0178] 19, if the application determines at 1935 based on the patient's response to the patient questionnaire at 1920 during image analysis at 1945 that the patient requires a full face mask, the application retrieves the coordinates of the following six exemplary landmarks relevant to sizing the full face mask: (d) bridge of nose; (m) lower lip; (k) left lip commissure; (l) right lip commissure; (h) left axilla and (i) right axilla. The dimensions of the following features defined by the landmarks (i.e. bridge of nose to lower lip; mouth width; and nose width) are then calculated. These dimensions are then compared to mask sizing data including dimensions or thresholds to determine which dimension mask is suitable for the patient. The mask sizing data may be stored in the memory 420 of the mobile communication device 400. By storing the mask sizing data on the mobile communication device, the application can recommend a mask to the patient without requiring a network connection.

[0179] In some embodiments, the face detection module determines the coordinates of all facial landmarks in the image, and the application searches those coordinates to identify landmarks associated with a particular mask category and calculate measurements of key facial features in the image and the dimensions of those key facial features.

[0180] Next, the sizing process of the nasal-facial mask is described with reference to FIG. 25. For the nasal-facial mask, the key facial features are nasal height 2530 and nasal width 2540. The nasal height facial feature is defined between facial landmark (d) nasal bridge and landmark (j) subnasal. The nasal width facial feature is defined between left axillary lobule and right axillary lobule (landmarks h and i in FIG. 10). Referring again to FIG. 19, once the application determines that the patient requires a nasal-facial mask at 1935 based on the response to the patient questionnaire at 1920, during image analysis, the application retrieves the coordinates of the following four exemplary landmarks relevant to sizing the nasal-facial mask: (d) nasal bridge; (j) subnasal; (h) left axillary lobule and (i) right axillary lobule. The dimensions of the features defined by the landmarks (i.e., nasal height and nasal width) are compared to mask sizing data that includes dimensions or thresholds to determine which dimensions of the nasal-facial mask are suitable for the patient.

[0181] The table below provides exemplary sizing data for a nasal-facial mask. Recommended mask sizes are provided for various nose heights and widths. In one exemplary embodiment, the data is stored as a look-up table in memory 420 and the application references the sizing data to select a mask size for the patient.

[0182] [Table 4]

[0183] The mask sizing data in the table is for sizing nasal-facial masks. The look-up table provides known results for various possible combinations of key feature dimensions. For example, for a nasal mask, if the patient's nose height is calculated to be 4.4-5.2 cm and nose width is calculated to be greater than 4.1 cm, the most suitable size is large (L).

[0184] A similar lookup table is provided for each mask category. For example, to size a full face mask with n critical dimensions, an nD lookup table (a lookup table or function with n input parameters and which produces a known result based on the various possible combinations of the input parameters and their various ranges) would be used. Different masks may have different sizing charts, lookup tables or sizing functions. The lookup tables are stored in memory.

[0185] The sizing process for an under-the-nose nasal mask will now be described with reference to Figure 26. For an under-the-nose nasal mask, the important facial features are nose width 2620 and nose length 2630 (i.e., nose depth) since the seal sits under and wraps around the underside of the nose.

[0186] Nasal width is defined as the dimension between the left axillary lobule (feature h in FIG. 10) and the right axillary lobule (feature i in FIG. 10). Nasal length is determined based on the distance from, for example, the nose tip (feature g in FIG. 10) to the subnasal (feature j in FIG. 10). Referring again to FIG. 19, once the application determines that the patient requires a subnasal mask at 1935 based on the response to the patient questionnaire at 1920, during image analysis, the application retrieves the coordinates of the following four exemplary landmarks relevant to sizing a subnasal nasal facial mask: left axillary lobule (h); right axillary lobule (i); nose tip (g); and subnasal (j). The dimensions of the features defined by the landmarks (nose length and nose height) are compared to mask sizing data including dimensions or thresholds to determine which dimension mask of the subnasal nasal mask is suitable for the patient. The dimensions can be calculated by using all three (x, y, z) coordinates of the four landmarks or by using only y, z.

[0187] As mentioned above, in some exemplary embodiments, the patient's selection of mask category from responses to a questionnaire is used to determine what dimensions are needed for mask sizing. The questionnaire is first presented and the patient's responses are used to determine the mask category. Once the category is identified, the specific landmarks needed for that mask category are identified in the application. All landmarks may be collected, but calculations of distances between specific landmarks are made by the application based on the identified mask category.

[0188] Other methods for determining the mask category for a patient may be used, for example, the application may be pre-configured with the patient's particular mask category or the application may rely on the patient selecting a mask category.

[0189] In the above embodiment, the application and various databases were stored locally on the mobile communication device. In addition, all processing during mask selection occurs on the mobile communication device. This arrangement avoids the need for any network connection during the mask selection process. Local processing and data retrieval may also reduce the time it takes to perform the mask selection process. One advantage is that the query and images can be processed locally, and only the calculated mask size needs to be transmitted, for example, when ordering a product. This reduces transmitted data and reduces data costs.

[0190] However, alternative embodiments execute the mask sizing application by using a distributed data storage and processing architecture. In such embodiments, a database (e.g., a mask sizing database, or a questionnaire database) may be located remotely from the mobile communication device and accessed over the communication network during execution of the mask selection application. Processing (e.g., facial landmark identification) may occur within a remote server, and the mobile communication device may transmit the captured images for processing over the communication network. In other examples, processing of the questionnaire responses may occur remotely. Such embodiments leverage external processing power and data storage facilities.

[0191] In the embodiment described above, the application ran on a mobile communications device. In another embodiment, the application or portions of the application may run on a respiratory therapy device.

[0192] The described example provides an automated way of recommending a mask category and a mask size within a particular category for a mask to be selected for a patient. Some embodiments are configured to allow a non-professional user to capture data by using non-professional equipment to enable selection of a suitable mask for use with a respiratory therapy device. Sizing decisions can occur by using a single camera that allows the application to run on a smartphone or other mobile communication device. Some embodiments do not require the use of any other phone features / sensors (e.g., accelerometers).

[0193] Some embodiments provide an application that allows for remote mask selection and sizing. This allows for remote patient set-up and reduces the need for the patient to enter a specialist's office for mask fitting and set-up. The application may also provide general mask information and may provide additional user instructions, cleaning instructions, and troubleshooting instructions.

[0194] The application uses the palpebral fissure width as a reference measurement within the image of the patient's face. The palpebral fissure can be detected within the facial image by using facial feature detection software, and is less likely to be obscured by the patient's eyelids compared to ocular features (e.g., iris or pupil). The larger width of the eye compared to smaller facial or ocular features such as the iris allows the application to capture accurate measurements even when the patient does not hold their head still or when the device being used is not capable of capturing higher resolution images. The use of the palpebral fissure as a reference measurement also allows the application to measure a single interpupillary distance or two interpupillary distance measurements to be measured and averaged. The corners of the eyes can also be detected from the contrast between the white of the eye and the skin.

[0195] Some embodiments take into account the tilt of the patient's head and filter the measurements, which may cause errors due to excessive tilt (i.e., pitch). Similar filtering may be used for roll and yaw. The described embodiments are also advantageous because tilt does not use the inertial measurement unit (e.g., accelerometer or gyroscope) of the mobile communication device and may reduce the processing load and time on the mobile communication device's processor. This also means that less state-of-the-art devices, which may not have an inertial measurement unit, can still be used to implement the described examples.

[0196] Sizing measurements can be taken even when the phone distance from the face varies. There is a preferred distance to ensure that the facial features of interest are captured with high enough resolution to obtain accurate measurements. There are visual guides to help the user navigate and use the sizing app. Sizing can be done in many different environments (e.g. outdoor light, indoor light). Sizing can be done regardless of user orientation (i.e. the user can be lying down or sitting or standing). This provides a more robust sizing app for sizing patient interfaces.

[0197] Some exemplary embodiments are configured to capture images from only a single angle, and the patient is not required to take profile images or multiple images from different angles.

[0198] Some exemplary embodiments provide real-time processing of image / video frames, which reduces the processing load and does not require large caching / memory requirements. Some exemplary embodiments do not require large memory or caching, and frames / images are not stored but are processed or discarded as they are received.

[0199] The above example illustrates "selecting." In some exemplary embodiments, the selection involves identifying a mask.

[0200] It should be understood that if any prior art publication is referred to herein, such reference does not constitute an admission that the publication forms part of the common general knowledge in that art in Australia or any other country.

[0201] In the claims that follow, as well as in the preceding description, unless the context otherwise requires to indicate language or necessary implication, the term "comprises" or variations thereof are used in an inclusive sense: that is, it is used to specify the presence of stated features and not to exclude the presence or addition of other features in various embodiments of the invention.

[0202] It should be understood that the foregoing description refers only to certain exemplary embodiments of the invention, and that variations and modifications to the certain exemplary embodiments of the invention are possible without departing from the spirit and scope of the invention, the scope of which is determined from the following claims.

Claims

1. 1. A method for selecting a patient mask for use with a respiratory therapy device, the mask being suitable for providing respiratory therapy to the patient, comprising: receiving data representing at least one digital image of a patient's face; identifying predetermined reference facial features appearing in said image, said predetermined reference facial features being said patient's eyes; determining measurements of the eye of the patient in the image; assigning a predetermined dimension to the measurement and determining a scaling factor for the image, the scaling factor being a ratio of the measurement to the predetermined dimension; identifying another facial feature in the image; determining a measurement of the other facial feature in the image; calculating a dimension of the other facial feature by using the scaling factor and the measurement of the other facial feature; and comparing the calculated dimension of the other facial feature with mask sizing data associated with a mask for the patient, and selecting a mask for the patient depending on the comparison.

2. The method of claim 1 , wherein the measurement of the eye of the patient is interpupillary distance.

3. The method described in claim 1, wherein the predetermined reference facial features appearing in the image are defined by at least two predetermined facial landmarks.

4. 10. The method of claim 1, wherein the step of identifying the patient's eyes in the image is performed by identifying at least two predetermined facial landmarks that are anthropometric features of the patient's face in the image associated with the eyes.

5. The method of claim 3 , wherein the at least two predetermined facial landmarks in the image are the corners of the eyes.

6. The method of claim 3 , wherein the at least two predetermined facial landmarks are the medial canthus and the lateral canthus of the eye.

7. 10. The method of claim 1, wherein the measurement of the eye is palpebral fissure width.

8. The method of claim 1 , wherein the other facial feature is identified by identifying at least two facial landmarks associated with the other facial feature.

9. The method of claim 1 , wherein the step of determining a measure of a facial feature is performed by calculating the number of pixels of the image between at least two facial landmarks in the image associated with the facial feature.

10. 2. The method of claim 1, wherein the step of determining the measurement of the reference feature in the image is performed by identifying two eyes of the patient in the image, calculating a measurement for each eye, and calculating an average measurement for the two eyes.

11. The method of claim 1 , wherein facial landmarks are anthropometric features of the patient's face identified in the image.

12. determining at least one attribute of said digital image; comparing the at least one attribute with predetermined attribute criteria; and determining whether the at least one attribute satisfies the predetermined attribute criteria; 2. The method of claim 1, wherein the step of selecting a mask for the patient is dependent on the at least one attribute satisfying the predetermined attribute criteria.

13. The at least one attribute is: an angle of the user's face in the image, the angle being at least one of a pitch angle, a yaw angle, or a roll angle; the focal length of said image; the depth of the patient's face within the image; and The method of claim 12 , including at least one of at least one predetermined landmark identified within the image.

14. 14. The method of claim 13, wherein the at least one attribute is the pitch angle and the predetermined attribute criterion is an angle between 0 and +6 degrees relative to the plane of the image.

15. 13. The method of claim 12, further comprising providing feedback related to whether the at least one attribute satisfies the predetermined attribute criteria.

16. 2. The method of claim 1, wherein the step of calculating the dimension of the other facial feature is performed on a plurality of images to generate a plurality of calculated dimensions, the method further comprising: calculating an average dimension of the other facial feature across at least a predefined number of the plurality of images; and using the average dimension to compare with the mask sizing data.

17. determining at least one attribute of said digital image; comparing the at least one attribute with predetermined attribute criteria; and determining whether the at least one attribute satisfies the predetermined attribute criteria; The method of claim 16, wherein the average size is calculated over images that satisfy the predetermined attribute criteria.

18. The method of claim 1 , further comprising the step of determining a mask category for the patient.

19. presenting at least one user question to the user; receiving at least one user response to the at least one user question; and The method of claim 1 , further comprising determining a mask category for the patient dependent on the received user response.

20. the other facial feature is selected from a plurality of facial features depending on the mask category; the mask sizing data associated with the patient mask relates to a mask of the determined mask category; 20. The method of claim 18 or claim 19, wherein the masks are defined in mask categories, where different mask categories have different relationships between mask sizing data and facial feature dimensions.

21. 1. A system for selecting a patient mask for use with a respiratory therapy device, the mask being suitable for providing respiratory therapy to the patient, the system comprising: receiving data representing at least one digital image of a patient's face; identifying predetermined reference facial features appearing in the image, the predetermined reference facial features being the patient's eyes; determining a measurement of the eye of the patient in the image; assigning a predetermined dimension to the measurement and determining a scaling factor for the image that is the ratio of the measurement to the predetermined dimension; Identifying another facial feature in the image; determining a measurement of the other facial feature in the image; and a processor configured to calculate a dimension of the other facial feature by using the scaling factor and the measurement of the other facial feature; and a memory for storing mask sizing data associated with a patient mask; and the processor further: The system is configured to compare the calculated dimension of the other facial feature with the stored mask sizing data associated with a patient mask, and select a mask for the patient depending on the comparison.

22. The system described in claim 21, wherein the system includes a mobile communication device, the mobile communication device further including an image capture device for capturing digital image data and a user interface for displaying data associated with at least one selected mask.

23. The system described in claim 21, wherein the measurement of the patient's eye in the image is the distance between at least two predefined facial landmarks.

24. 23. The system of claim 21 or 22, wherein the measurement of the eye of the patient is interpupillary distance.

25. The system described in claim 21, wherein the predetermined reference facial features appearing in the image are defined by at least two predetermined facial landmarks.

26. The system described in claim 25, wherein the measurement of the patient's eye in the image is the distance between the at least two predetermined facial landmarks.

Images