Determining degree of opening

Abstract

An oral care system includes an oral care device and a processor. The processor is configured to obtain an image of a user using the oral care device, and process the image to identify a portion of the oral care device in the image, and determine a relative measurement of the user from the image. An absolute measurement of the user may then be determined using the identified portion of the oral care device and the relative measurement.

Description

Technical Field

[0001] The present invention relates to the field of oral health. In particular, the present invention relates to the field of detecting trismus and temporomandibular joint dysfunction. Background Art

[0002] Trismus (commonly known as lockjaw) and temporomandibular joint (TMJ) dysfunction (TMJD) result in reduced jaw opening or limited movement. Trismus and TMJD interfere with eating, speaking, and maintaining proper oral hygiene. When untreated or undiagnosed, trismus and TMJD can become chronic and painful and affect people’s daily lives and health by causing migraines, back, neck and shoulder discomfort, and hearing impairment / loss.

[0003] Examination and treatment of trismus and TMJD often requires access to the mouth, which may be limited or, in some cases, not possible. Trismus may be caused by joint problems, infection, trauma, cancer treatment, or post-dental care. For example, temporomandibular joint disease may lead to trismus. In the United States, approximately 12% of the population (approximately 35,000,000 people) are affected by TMJD at any given time.

[0004] Another significant cause of trismus is head and neck cancer, where the prevalence of trismus is estimated to be as high as 38% of cases. For patients with this type of cancer, trismus management focuses primarily on preventing the progression of trismus and restoring jaw function. Currently, there is no non-invasive mechanism for tracking the progression of trismus or tracking the restoration of jaw function to allow health professionals to observe the patient's response to treatment.

[0005] A common method of diagnosing the severity of trismus is to measure the subject's mouth opening by means of a tool such as a Boley gauge or ruler. Normal mouth opening (i.e., no trismus) is expected to be above 40 mm. Mild trismus is typically classified as a mouth opening between 30 mm and 40 mm, moderate trismus is typically classified as a mouth opening between 15 mm and 30 mm, and severe trismus is typically classified as a mouth opening less than 15 mm.

[0006] Alternatively, a three-finger screening test can be used to estimate the severity of trismus. In the three-finger screening test, the subject attempts to open the mouth as wide as possible to place their fingers between the upper and lower teeth. Being able to accommodate three or more fingers generally indicates no trismus, two to three fingers indicates mild trismus, one to two fingers indicates moderate trismus, and less than one finger indicates severe trismus.

[0007] Both methods are considered invasive, require a second person to take the measurements, and do not provide health professionals with the tools they need to monitor a patient's progress after a procedure such as wisdom tooth removal.

[0008] US5097820A discloses a device that can be used to diagnose and treat limited mouth opening. However, such devices are often invasive and very expensive. WO2017015342A1 discloses a trismus rehabilitation device similar to the device disclosed in US5097820A, which has an additional sensor for measuring the pressure applied at different points of the jaw to estimate the mouth opening.

[0009] The main limitation of these solutions is that their purpose is limited to single-site measurements, usually performed by dental professionals in a practice or clinic, and therefore are not used frequently by subjects. This reduces their usefulness in large-scale monitoring of TMJD or trismus. Additionally, it is difficult to digitize these measurements in a seamless workflow to enable long-term monitoring of disease progression.

[0010] Therefore, there is a need for an improved device for detecting the presence of trismus or any mouth opening / closing disorder, and continuously tracking its progression, particularly in a home environment.

[0011] US2020 / 359777 describes a dental device tracking method, comprising acquiring at least one first image, including an image of at least one body part of a user, using an imager of the dental device. Summary of the invention

[0012] The invention is defined by the claims.

[0013] According to an example of one aspect of the present invention, there is provided an oral care system, comprising:

[0014] Oral care equipment; and

[0015] A processor configured to:

[0016] obtaining an image of a user using an oral care device;

[0017] Process the image to:

[0018] identifying parts of an oral care device in an image;

[0019] determining relative measurements of the user based on the image; and

[0020] determining an absolute measurement of the user based on the identified portion of the oral care device and the relative measurement;

[0021] The processor is further configured to:

[0022] obtaining a known length of the portion of the oral care device;

[0023] processing the image to determine a relative length of the portion of the image (708);

[0024] determining a calibration factor based on the relative length of the portion and the known length of the portion; and

[0025] Apply calibration factors to relative measurements to determine absolute measurements.

[0026] Oral care devices are not used to treat mouth opening disorders (such as trismus). For example, oral care devices can be used to maintain oral hygiene or as therapeutic devices for treating other oral health disorders other than mouth opening disorders.

[0027] When using the oral care device on the palate side of the front-most teeth, the user's jaw is usually forced to drop, so that the user opens the mouth as wide as possible. Therefore, an image of the user can be obtained in this case, which provides an image of the user opening the mouth as wide as possible.

[0028] However, the image itself does not provide a true measurement of maximum mouth opening. Therefore, a known length / dimension in the same image is required to measure the maximum mouth opening. It has been recognized that a portion of an oral care device can provide a known length and always appear in the image of a user using the oral care device. In other words, the portion enables dynamic image rescaling or length calibration.

[0029] The length of the portion may be known (e.g. from a cloud database including a computer-aided design (CAD) drawing with dimensions), or may be obtained (e.g. using a QR code or barcode that appears in the image). Additional sensors may also be used to measure the true length of the portion (e.g. a depth sensor may be used to generate a 3D rendering relative to the image, thereby being able to provide the true length of the portion).

[0030] When the oral care device is used on the most anterior teeth, it can provide high accuracy in length calibration. The proximity of the oral care device and its parts to the teeth significantly reduces potential errors in the determination of the maximum mouth opening value. For example, when a calibration length is used in an image, the calibration length and the length to be measured (in this case, the maximum mouth opening) may not be at the same depth relative to the image sensor.

[0031] Thus, an image of an oral care device used on the palatal side of the anterior-most teeth provides an image including a convenient portion of the oral care device (which can be used to calibrate the maximum mouth opening value) used by the user when the mouth is opened as wide as possible in a position that provides high calibration accuracy. In addition, all of this is achieved while the user is using the oral care device for other primary purposes (e.g., oral hygiene, such as brushing or jet rinsing the teeth), thereby not adding additional activities to the user's daily routine.

[0032] The identified portion of the oral care device is preferably a rigid portion of the oral care device, the shape of which does not substantially change when the oral care device is in use.

[0033] Relative measurements can be relative mouth opening values. Absolute measurements can be mouth opening values.

[0034] The mouth opening value is the absolute mouth opening value representing the distance between the upper teeth and the lower teeth.

[0035] The relative measurement may be a relative measurement of an oral feature of the user. For example, the oral feature may represent misaligned teeth or receding gums.

[0036] The image can be an image of a user performing a specific action with the oral care device. The specific action is a specific use of the oral care device. For example, a user may use the oral care device on the palatal side of their front-most upper teeth or on the lingual side of their front-most lower teeth. In another example, absolute measurements can be taken regularly (e.g., in real time: 20 to 60 times per second). These values ​​can then be compared to each other to determine the maximum and / or minimum measurements (when necessary). For more static measurements (e.g., facial feature measurements), a single absolute measurement may be sufficient.

[0037] The absolute measurement may be the absolute mouth opening. The relative measurement may be a measurement of a user feature in the image. Thus, the absolute measurement may be an absolute measurement of the feature.

[0038] The system may further include a memory system storing the known lengths of the portions, wherein the processor obtains the known lengths of the portions from the memory system.

[0039] The oral care device may include a sensor system configured to determine an orientation of the oral care device, and wherein the processor is further configured to determine the calibration factor based further on the orientation of the oral care device.

[0040] The oral care device may not be exactly perpendicular to the image sensor, so the length calibration element may appear distorted in the image. This is often caused by a portion of the oral care device being used on the palatal side of the teeth (i.e., in the mouth), while another portion of the oral care device is held by the user and is outside the mouth. Therefore, knowledge of the orientation of the oral care device (e.g., relative to the gravity vector) can be used to improve the calibration factor.

[0041] Similarly, images can be acquired at non-normal angles and angle correction can be applied to the images. This can be done by comparing the toothbrush angle obtained from the sensor with the toothbrush angle seen in the image.

[0042] The orientation may include at least one angle measurement. The angle measurement may be a measurement relative to gravity and / or relative to the teeth. For example, the angle measurement may include a pitch angle measurement, a roll angle measurement, and / or a yaw angle measurement of the oral care device. In some cases, a change in the pitch angle measurement may be sufficient to determine the mouth opening value.

[0043] The processor may be further configured to apply a feature recognition algorithm to the image to identify portions of the oral care device in the image and measure relative lengths of the identified portions in the image.

[0044] The processor may also be configured to apply a feature recognition algorithm to the image to identify the user's mouth in the image, and measure the user's relative maximum mouth opening based on the mouth identified in the image.

[0045] The portion of the oral care device may include at least two markings placed on the oral care device, wherein the distance between the markings is known. A marking of a known size (eg diameter or length) may also be used.

[0046] The oral care device may be a toothbrush, an oral irrigator, or a toothbrush with an oral irrigator.

[0047] The oral care device may include one or more cameras, wherein at least one of the cameras is configured to obtain an image of a user using the oral care device.

[0048] On a toothbrush, the camera can be positioned to face in the same general direction as the bristles, such that an image of the user using the oral care device is obtained when the user is brushing the palatal side of the front-most upper teeth. Similarly, the camera can be positioned to face in the opposite direction of the bristles, such that an image of the user using the oral care device is obtained when the user is brushing the lingual side of the front-most lower teeth.

[0049] The processor may be further configured to identify a physiological marker of the user in the image, wherein determining the absolute measurement of the user is further based on the physiological marker.

[0050] Various physiological markers of the user can be used to calibrate the length of the user's measurements. The user's measurements can include measurements of user features of unknown dimensions (e.g., unknown length or height). The user's physiological markers can have known dimensions, wherein the known dimensions can be used to determine absolute measurements based on relative measurements. This can be achieved by using a calibration factor between the known dimensions of the physiological markers and the relative dimensions of the physiological markers measured from the image (e.g., in pixels). The known dimensions can be pre-measured or estimated dimensions (e.g., based on measurements of dimensions of other users with similar physiological features). Physiological markers can include the distance between the user's eyes, the distance from the eyes to the mouth, the width of the mouth, the size of the nose, etc. One or more of the user's multiple physiological markers can be used (e.g., markers for forensic facial reconstruction).

[0051] The relative measurement may be relative mouth opening, and the absolute measurement may be absolute mouth opening.

[0052] The present invention also provides a method for determining an absolute measurement of a user, the method comprising:

[0053] obtaining an image of a user using an oral care device;

[0054] obtaining a known length of a portion of an oral care device;

[0055] Process the image to:

[0056] identifying parts of an oral care device in an image;

[0057] determining a relative length of the portion of the image (708);

[0058] determining a calibration factor based on the relative length of the portion and the known length of the portion;

[0059] determining relative measurements of the user from the image; and

[0060] The calibration factor is applied to the relative measurement to determine an absolute measurement for the user based on the identified portion of the oral care device and the relative measurement.

[0061] The method may also include determining an orientation of the oral care device, wherein determining the calibration factor is further based on the orientation of the oral care device.

[0062] The present invention also provides a computer program product comprising computer program code, which, when executed on a computing device having a processing system, causes the processing system to execute all the steps of the aforementioned method.

[0063] These and other aspects of the invention will be apparent from and elucidated with reference to the embodiments described hereinafter. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] For a better understanding of the present invention, and in order to more clearly show how the same may be carried into effect, reference will now be made, by way of example only, to the accompanying drawings, in which:

[0065] Figure 1 An open mouth is shown;

[0066] Figure 2 A toothbrush head having a sensor system on the head of the toothbrush is shown;

[0067] Figure 3 A toothbrush head with a sensor system for use in an oral cavity is shown;

[0068] Figure 4 An oral cavity having an isosceles triangle shape is shown;

[0069] Figure 5 and Figure 6 The toothbrush is shown used on the maxillary most lateral teeth;

[0070] Figure 7 A toothbrush is shown used on the lingual side of the upper front teeth;

[0071] Figure 8 An image for determining mouth opening is shown; and

[0072] Fig. 9 A method for determining absolute mouth opening is shown. DETAILED DESCRIPTION

[0073] The present invention will be described below with reference to the accompanying drawings.

[0074] It should be understood that the detailed description and specific examples, while indicating exemplary embodiments of the apparatus, system, and method, are intended for illustrative purposes only and are not intended to limit the scope of the invention. These and other features, aspects, and advantages of the apparatus, system, and method of the invention may be better understood in conjunction with the following description, the appended claims, and the accompanying drawings. It should be understood that the drawings are schematic only and are not drawn to scale. It should also be understood that the same reference numerals represent the same or similar components throughout the drawings.

[0075] The present invention provides an oral care system, including an oral care device and a processor. The processor is configured to obtain an image of a user using the oral care device, and process the image to identify a portion of the oral care device in the image, and determine a relative measurement of the user from the image. Then, the absolute measurement of the user can be determined using the identified portion of the oral care device and the relative measurement.

[0076] The present invention proposes a system for obtaining the mouth opening degree of a user by using an oral care device. This enables, for example, a clinician to detect the mouth opening dysfunction when the user has it and to estimate its progression by continuing to use the system.

[0077] This facilitates continuous remote monitoring of TMJ dysfunction or trismus, thereby allowing health and dental professionals to evaluate and quantify the results of the treatment they provide. Another benefit is that by evaluating the requested treatment date with the date trismus was detected, health and dental insurance companies will have an additional data point to verify claims related to TMJ dysfunction or trismus.

[0078] Figure 1 An open mouth is shown. As shown, the upper teeth 102 are not evenly separated from the lower teeth 104, so the degree of mouth opening (lines 106 and 108) can be different depending on where the mouth opening is measured. Currently, mouth opening values / measurements are primarily taken at the front of the mouth (i.e., line 106) because this is the most easily obtained measurement from the outside, and therefore a clinician or other person can measure the user's mouth opening.

[0079] However, it should be understood that the mouth opening value can be obtained at any location in the mouth in a manner that can be interpreted by the clinician. For example, line 108 (near the middle of the mouth) can be used in conjunction with a specific location in the mouth. Conversely, when the location is unknown, the processor can obtain multiple measurements and find the maximum value among these measurements. The maximum measurement is usually interpreted as the mouth opening at line 106 (front of the mouth) because this is usually the location with the greatest mouth opening.

[0080] Maximum mouth opening is one of the most relevant mouth openings currently measured. Maximum mouth opening quantifies the maximum extent to which the user's mouth can open. A small maximum mouth opening can indicate the possibility of a mouth opening disorder. Similarly, minimum mouth opening (i.e., maximum mouth closure) is also a relevant mouth opening. If the user cannot completely close their mouth, it will result in a large minimum mouth opening, indicating the possibility of a mouth opening disorder.

[0081] It should be noted that Figure 1 Typical mouth openings are shown. In practice, the maximum mouth opening (and potentially the minimum mouth opening) is often more clinically relevant than the typical mouth opening. However, if accurate and frequent mouth openings can be obtained, the entire range of mouth openings may be relevant.

[0082] The inventors have recognized that the use of oral care / hygiene equipment (such as toothbrushes, dental floss equipment, spray equipment, etc.) all occurs in the user's mouth, and usually occurs frequently (such as once a day or more). In addition, considering the nature of the use of oral care equipment in the user's mouth, the user usually opens his mouth during use. Therefore, it is proposed to measure the user's mouth opening while using the oral care equipment. This enables the user to do two things at the same time (i.e., maintain oral hygiene and measure the mouth opening). Compared with the situation where the user or the usual clinician must perform the mouth opening measurement separately, this also means that the mouth opening value can be obtained more frequently.

[0083] The following examples will relate to measuring maximum mouth opening. However, it should be understood that the systems and methods described with respect to measuring maximum mouth opening can also be applied to the measurement of any general mouth opening. In certain cases where the system or method specifically relates to maximum mouth opening, alternatives to minimum mouth opening or general mouth opening can be provided.

[0084] Similarly, a toothbrush will be used in the following embodiments to illustrate the examples. However, it should be understood that other oral care devices may also be used.

[0085] In a first embodiment, the sensor is placed on the head of the toothbrush. Figure 2 A toothbrush head 202 is shown having a sensor system 206 in the head 202 of the toothbrush. The sensor system is placed at the distal end of the toothbrush head 202, close to the bristles 204 of the toothbrush head 202. In alternative embodiments, the sensor system may also be present in the handle of the oral care device.

[0086] The quantitative measure of mouth opening can depend on where along the patient's jaw the measurement is taken. This is because the mouth opens widest at the front and narrowest at the back.

[0087] Figure 3 A toothbrush head 202 with a sensor system 206 is shown for use on the mouth. In this case, the sensor system may include proximity sensors and / or distance sensors and / or acceleration sensors (inertial measurement) and / or gyroscopes. In order to estimate the level of mouth opening at any location in the oral cavity, the sensor system uses a sensor system 206 attached to the brush head along the longitudinal axis (i.e., along the longitudinal axis) of the brush head. Figure 3 The distance measurement performed by the sensor system 206 on the line segment AB shown in FIG. 1 determines the position of the brush head along a given jaw. The mouth opening at the current brush head position is determined by the distance measurement performed by the sensor system 206 on the line segment AB shown in FIG. Figure 3 The distance measurement is determined by the sensor on the line segment BC) shown.

[0088] The elevation angle of the current position can be calculated (i.e., angle ∠BAC = ∠BAD). As shown in the figure, the isosceles triangle BAD can also be used. Point D can be found using a simple geometric shape consisting of points A, B, and C.

[0089] Those skilled in the art will appreciate that the calculation of segment BD is straightforward. Given the measurements of segment BC, an approximate isosceles triangle (ABD) can be created, from which basic trigonometric functions allow BD to be derived. To this end, segment AD is calculated by simply applying the Pythagorean theorem on triangle ABC.

[0090] Although neither the maxillary nor mandibular planes are linear, local linearity can be reasonably expected by measuring with the brush head as far posterior to the occlusal plane as possible, so that the mouth opening far in the mouth can be approximated by calculating the distance represented by line segment BD.

[0091] Therefore, the elevation angle can be used to estimate the mouth opening at any other point, including the maximum opening. Given the BD and the angle ∠BAD at a particular given point, the mouth opening at other points can be estimated. It is expected that the nonlinear planes of the two jaws may limit the accuracy of the proposed method. However, the range of mouth opening is large enough (e.g. 15 mm for moderate trismus) that the estimates made using this method are expected to be within an acceptable range.

[0092] For example, Figure 4 A mouth with an isosceles triangle (AB'D') is shown. As shown, despite the irregularity of the tooth shape, the triangle surprisingly fits the shape of the mouth opening. Thus, it can be concluded that the isosceles triangle is a good descriptor of mouth opening. It follows that the angle ∠B'AD'=∠BAD can also be used to accurately express mouth opening. Thus, the mouth opening value is not limited to the distance measurement between the upper teeth 102 and the lower teeth 104, the elevation angle can also be used as a mouth opening value.

[0093] The first embodiment can be implemented with a sensor system having a dual sensor configuration, wherein the sensitive axes of the two sensors are perpendicular to each other. The orientation of the sensors is determined by Figure 3 The line segments BA and BC in are represented. Generally, the first example uses triangulation to estimate the mouth opening value.

[0094] In a second embodiment, the mouth opening value is determined using sensor data from a sensor system already present in current toothbrushes. In this embodiment, measurements are taken at unique brushing positions and orientations (e.g., behind upper or lower molars). In particular, the second embodiment enables measurements to be taken during precise and unique movements to achieve a desired mouth opening (e.g., maximum or minimum mouth opening).

[0095] While the first embodiment uses sensors added to the brush head to facilitate triangulation of mouth opening, it is recognized that sensors already present in the (electric) oral care device may also be used to assess and track the mouth opening. For example, accelerometers, inertial measurement units, rotation sensors (e.g., compasses, gyroscopes, etc.), displacement sensors, position sensors, Hall sensors, and / or printed strain gauges may already be present in the electric oral care device.

[0096] As a reminder, the description and drawings show an electric toothbrush as an example. However, any other toothbrush (e.g. a combined brushing and flossing device) or a water spray device or a flossing device may also be considered. In particular, a water spray device is extremely suitable because the head of the device with a nozzle is relatively long and narrow, which can reduce measurement problems (e.g. there is no problem of the wider device body of a toothbrush interfering with the measurement).

[0097] It has been recognised that during certain brushing movements (ie during brushing behind the rearmost molars / wisdom teeth), the user is forced to open the mouth maximally and rotate the brush through a certain angle until pressing hard against the jaw opposite the tooth being brushed.

[0098] The purpose of this embodiment is to measure this brushing motion and interpret the measurement as representative of the degree of openness of the mouth. Limited motion or reduced opening over time may be associated with a mouth opening disorder (e.g., TMJ, trismus, etc.).

[0099] While measurements taken during cleaning of both the upper and lower teeth can in principle form an interpretation of mouth opening, the inventors have realised that using measurements taken while brushing behind the most posterior molars / wisdom teeth in the maxilla may be preferred.

[0100] When brushing the upper jaw, the opening of the mouth causes the lower jaw to drop, so that the brush is initially in the same position as when brushing along the tops of the upper teeth, flat against the upper teeth. Starting from this fixed reference position, the brush must traverse the entire opening angle of the mouth before pressing against the lower jaw. Therefore, the movement measured is a direct measure of the degree of mouth opening.

[0101] Conversely, when brushing the lower jaw, the opening of the mouth again causes the lower jaw to drop. However, in this case, the jaw drop has already separated the brush from the jaw, and there is no need to move the brush. From this starting position, the brush only needs to traverse a portion of the opening angle of the oral cavity before pressing against the upper jaw. Therefore, the movement measured may not be a direct measure of the degree of mouth opening. However, it must be noted that this measurement may have some predictive value of the longitudinal measurement.

[0102] Figure 5 and Figure 6 The toothbrush 502 is shown being used on the posterior-most teeth of the upper jaw 506. To determine the degree of mouth opening during brushing of the upper jaw 506, the following measurements are suitable measures:

[0103] - When brushing from the upper jaw 506 (such as Figure 5 ) to the lower jaw 508 (as shown) Figure 6 As shown), the total movement of the sensor system 504 (e.g., including an accelerometer and / or a position sensor) is measured.

[0104] - When the toothbrush 502 is moved from the upper jaw 506 (eg Figure 5 ) to the lower jaw 508 (as shown in Figure 6 ), the rotation angle of the sensor system 504 (e.g., including a compass and / or a gyroscope) is measured. If necessary, the distance from the top of the brush head to the sensor system 504 can be taken into account to convert the rotation angle into a movement amplitude.

[0105] For many use cases (e.g., such as determining the progression of TMJD), monitoring the measured changes in brush sensor motion or rotation angle may be sufficient.

[0106] However, for some use cases (e.g. determining the severity of TMJD / trismus), it may be preferable to determine the absolute opening of the mouth. To achieve this, the magnitude of the distance traversed by the brush at the point on the brush where it exits the oral cavity may be determined. This may be derived as follows:

[0107] Brushing motion at mouth opening = sensor system 504 at Figure 6 The measured movement between point E and point F in the figure is the ratio of the mouth depth (D-mouth opening) to the distance between the sensor system 504 and the brush head (D-sensor). The mouth depth (D-mouth opening) is the distance between the front-most teeth in the mouth and the back-most teeth in the mouth.

[0108] When calculating the absolute mouth opening, the dimensions (e.g., diameter, thickness, etc.) of the brush at the point where the brush leaves the mouth (brush diameter) may be considered: for example, mouth opening = distance traveled by the brush at the mouth opening position + brush diameter.

[0109] Note that for this absolute mouth opening measurement, it is advantageous for the portion of the toothbrush that enters the mouth to be relatively narrow and of constant diameter. For example, a relatively long toothbrush can extend out of the mouth even when placed behind the last tooth. Spraying devices may be particularly suitable because they have an inherently elongated tube for spraying purposes.

[0110] It is also possible to determine the time when the brushing motion is most likely to be associated with the mouth opening. Specifically, if the sensor system 504 includes a motion or position sensor (or the like), the following pattern may represent a preferred measurement time:

[0111] - the brush is positioned to brush the top surfaces of the upper teeth;

[0112] - Move the brush back along the teeth;

[0113] - The brush stops moving (motion sensor) or the position is at or behind the rear molars / wisdom teeth (position sensor) and remains constant; and

[0114] - The measured brushing movement indicates possible mouth opening due to jaw drop.

[0115] In a particular use case, it may be determined that a particular brush head is being used specifically to brush behind the back teeth (i.e., a single-tuft brush head). The measurements in this particular use case may be given a higher weight because the user is likely focusing on brushing behind the molars and trying to open the mouth fully.

[0116] Obviously, the user does not necessarily have to do this every time they brush, as these jaw-related problems only develop slowly over time. However, if a relatively long time has passed since a measurement could be made, in an alternative method the user can be instructed to perform this specific movement by placing the device tip as described above, opening the jaw and moving the device parallel to the opening of the jaw. In this way, a good quality reference measurement can always be established. The user can be guided via an external device, such as an application on the user's smartphone.

[0117] Existing oral care devices use algorithms to determine the orientation of the oral device relative to the gravity vector. This can be used to determine the angle change when the mouth is open. Figure 6 Orientations 602 and 604 are shown in FIG. Orientation 602 is Figure 5 Point E shows the placement of sensor system 504 corresponding to orientation 602 of the toothbrush. Similarly, orientation 604 is the orientation corresponding to sensor system 504 being located at point F. Figure 6 The orientation of the toothbrush shown. Therefore, the angle θ between the two orientations can be found.

[0118] In some cases, conversion from angle measurements to mouth opening distance (requiring "jaw length") may not be necessary. In fact, the angle of mouth opening may be more accurate (because fewer measurements are required) and may contain more information than the distance measurement of mouth opening. The use of the distance measurement of mouth opening (usually in mm) may be more common as it is easiest to assess according to current practice.

[0119] Robustness to head motion may also be a concern. Mouth opening may be confused with head tilt when orientation changes are detected. One approach to addressing this issue may be to use algorithms trained to differentiate between "mouth opening" and "head tilt." Algorithms such as these are currently used to determine the position of a toothbrush in the mouth.

[0120] For example, position sensing algorithms using inertial measurement unit (IMU) sensor signals have been used previously. These algorithms use the average or median direction of the brush over time to estimate the head tilt. The head tilt is then used to obtain a better estimate of the orientation of the brush relative to the user's head. In some cases, the direction of the head tilt can be re-estimated (based on the brush direction) each time the user changes to an action that acts on another section of the mouth.

[0121] A second approach could be to use pressure sensors to determine when the brush head is pushed against the teeth and when the mouth is opened due to "brushing behind the molars". A third approach could be to add an acoustic sensor to the sensor system. Due to the resonance of the mouth, increased sound levels and changes in the sound spectrum occur. These are different based on whether the mouth is open or closed.

[0122] In a third embodiment, an image algorithm with dynamic image rescaling and calibration is used to determine the (maximum) mouth opening for a unique brushing position and orientation (brushing the lingual incisors).

[0123] Figure 7 A toothbrush 702 is shown being used on the lingual side of the upper front teeth 710. The toothbrush includes two markings 706 and 707 with a known length 708 between the markings 706. A camera 712 is used to capture images of the action.

[0124] A third embodiment provides software-implemented mouth opening measurement using a calibrated toothbrush (or part thereof) and a camera (e.g., a camera in a smartphone) without the need for a sensor on the toothbrush 702. The camera could also be part of the toothbrush handle (distal end), and during brushing, the camera could be facing the bathroom mirror and in this way image the mouth opening.

[0125] The mouth opening is measured at the unique brushing orientation / position shown, thereby providing a reliable measurement of the width of the mouth opening. It has been recognized that during a particular brushing motion (e.g., brushing behind the maxillary or mandibular incisors), the user is forced to open the mouth to the maximum and rotate the brush through a unique angle until it is pressed against the lingual maxillary or mandibular side opposite the teeth being brushed, such as Figure 7 shown.

[0126] By knowing the distance to (at least) one (reference) mark 706 of the length calibration toothbrush shaft and identifying the mark and the mouth opening (e.g. using a feature detection algorithm and / or a feature matching algorithm), the absolute mouth opening can be determined by, for example, accessing a database with known brush head or reference mark dimensions, deriving an image calibration or image scaling factor. The mark 706 can be a colored / textured dot added to the toothbrush or a mark inherent to the brush head (e.g. an ejector pin mark from an injection molding process). A single mark 706 (e.g. with a known diameter) can be used.

[0127] A database containing reference lengths 708 may be accessed, allowing images to be rescaled dynamically. Because the distance at which the images are acquired cannot be reliably controlled (i.e., to perform compensation and automatic scaling for different image magnification factors), knowing the length enables the absolute maximum mouth opening value to be reliably determined (e.g., if images are acquired over subsequent weeks).

[0128] The toothbrush 702 may be connected to a database of brush head design data (eg, diagrams and dimensions) including toothbrush (or portion thereof) dimensions (eg, in mm). A radio frequency identification (RFID) code may be used to identify the type of toothbrush attached.

[0129] Images were acquired while the toothbrush was in use to allow the degree of mouth opening to be determined. Figure 8 An image for determining mouth opening is shown. The user can brush their teeth and take an image (eg, via a smartphone camera) while brushing their incisors. Alternatively, the camera can be part of a bathroom mirror.

[0130] The processor or cloud engine then performs the following algorithm steps to Figure 8 The absolute maximum mouth opening is determined from the acquired images as shown in the processed images in:

[0131] - Decompose RGB image into RGB channels;

[0132] - Apply feature recognition functions (e.g., boundary and edge detection) to distinguish between brush head features and oral features (e.g., lips, teeth) - see Figure 8 ;

[0133] - Identify markers 706 and 707 and measure the relative distance between the two markers (measured in pixels);

[0134] - Identify the relative mouth opening 804 (measured in pixels);

[0135] - rescaling the uploaded image based on the known distance 708 (e.g. measured in mm) of the boundary segmentation features and the markings 706 on the brush head and the relative measurements;

[0136] - using the relative distances of the markers 706 and 707 identified in the image and the known distance 708, a calibration factor (e.g., distance / pixel) may be determined; and

[0137] - Apply the length calibration factor to the relative mouth opening 804 to determine the absolute maximum mouth opening.

[0138] Optionally, the sensor system 704 already present in the toothbrush 702 can be used to improve the measurement accuracy by taking into account the influence of the brushing angle on the determination of the absolute mouth opening. It should be noted that the mandibular and maxillary incisors are brushed at different pitch angles.

[0139] The camera 712 can also be placed on the toothbrush 702. This takes advantage of the fact that the user usually brushes their teeth in front of a mirror. Therefore, the camera 712 can be placed on the toothbrush so that it captures the image of the user brushing their teeth reflected in the mirror in front of the user. This also eliminates the need to use an external camera while brushing teeth.

[0140] Fig. 9 A method for determining absolute mouth opening is shown. The method includes receiving an image of a user brushing their teeth in step 902, measuring the (relative) length (in pixels) between marks (or other parts) on the toothbrush in step 904, and determining a calibration factor 906 (e.g., in pixels / mm or distance / pixel) using the measured length between the marks and the known length between the marks (e.g., extracted from a database) in step 906. In step 908, the (relative) mouth opening (in pixels) is also determined or measured from the image. Therefore, the calibration factor can be applied to the (relative) mouth opening to determine the absolute mouth opening. Note that the mouth opening can be a maximum mouth opening, a minimum mouth opening, or any general mouth opening.

[0141] Those skilled in the art will be able to easily develop a processor for executing any method described herein. Therefore, each step in the flowchart may represent a different action performed by the processor and may be performed by a corresponding module of the processor.

[0142] As described above, the system utilizes a processor to perform data processing. The processor can be implemented in a variety of ways using software and / or hardware to perform various desired functions. The processor typically employs one or more microprocessors that can be programmed with software (such as microcode) to perform the desired functions. The processor can be implemented as a combination of dedicated hardware that performs some functions and one or more programmed microprocessors and their associated circuits that perform other functions.

[0143] Examples of circuits that may be employed in various embodiments of the present disclosure include, but are not limited to, conventional microprocessors, application specific integrated circuits (ASICs), and field programmable gate arrays (FPGAs).

[0144] In various embodiments, the processor may be associated with one or more storage media, such as volatile and non-volatile computer memory, such as RAM, PROM, EPROM, and EEPROM. The storage media may be encoded with one or more programs that, when executed on one or more processors and / or controllers, perform the desired functions. The various storage media may be fixed in the processor or controller, or may be transportable so that one or more programs stored therein may be loaded into the processor.

[0145] In the process of implementing the claimed invention, those skilled in the art can understand and implement variations of the disclosed embodiments by studying the drawings, the disclosure and the appended claims. In the claims, the word "comprising" does not exclude other elements or steps, and the indefinite article "a" or "an" does not exclude a plurality.

[0146] The functions implemented by a processor may be implemented by a single processor or multiple separate processing units, which may be considered to collectively constitute a “processor.” In some cases, these processing units may be remote from each other and communicate with each other in a wired or wireless manner.

[0147] The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.

[0148] The computer program may be stored / distributed on an appropriate medium, such as an optical storage medium or solid-state medium provided with the hardware or as part of other hardware, but may also be distributed in other forms, such as via the Internet or other wired or wireless telecommunication systems.

[0149] If the term "suitable for" is used in the claims or the specification, it should be noted that the term "suitable for" is intended to be equivalent to the term "configured to". If the term "device" is used in the claims or the specification, it should be noted that the term "device" is intended to be equivalent to the term "system" and vice versa.

[0150] Any reference signs in the claims should not be construed as limiting the scope.

Claims

1. An oral care system, comprising: Oral care equipment (202, 502); as well as A processor, the processor being configured to: obtaining an image of a user using the oral care device (802); Process the image to: identifying a portion of the oral care device in the image (706, 707); determining a relative measurement of the user from the image (804); and determining an absolute measurement of the user based on the identified portion of the oral care device and the relative measurement (106); Wherein, the processor is further configured to: obtaining a known length of the portion of the oral care device; processing the image to determine a relative length of the portion in the image (708); determining a calibration factor based on the relative lengths of the portions and the known lengths of the portions; and The calibration factor is applied to the relative measurement, thereby determining the absolute measurement.

2. The system of claim 1 further comprising a memory system storing the known length of the portion, wherein the processor obtains the known length of the portion from the memory system.

3. The system according to claim 1 or 2, wherein: The oral care device includes a sensor system configured to determine an orientation (602, 604) of the oral care device, and wherein the processor is further configured to determine the calibration factor based further on the orientation of the oral care device.

4. The system according to any one of claims 1 to 3, wherein: The processor is also configured to: applying a feature recognition algorithm to the image to identify the portion of the oral care device in the image; and The relative length of the identified portion in the image is measured.

5. The system according to any one of claims 1 to 4, wherein: The processor is also configured to: applying a feature recognition algorithm to the image to identify the user's mouth in the image; and The user's relative maximum mouth opening is measured from the mouth identified in the image.

6. The system according to any one of claims 1 to 5, wherein: The portion of the oral care device includes at least two markings placed on the oral care device, wherein a distance between the markings is known.

7. A system according to any one of claims 1 to 6, wherein: The oral care device is a toothbrush, an oral irrigator or a toothbrush with an oral irrigator.

8. The system according to any one of claims 1 to 7, wherein: The oral care device comprises one or more cameras, wherein at least one of the cameras is configured to obtain the image of a user using the oral care device.

9. The system according to any one of claims 1 to 8, wherein: The processor is further configured to identify physiological markers of the user in the image, wherein determining the absolute measurement of the user is further based on the physiological markers.

10. The system according to any one of claims 1 to 9, wherein: The relative measurement is relative mouth opening, and wherein the absolute measurement is absolute mouth opening.

11. A method for determining an absolute measurement of a user, the method comprising: Obtaining an image of a user using an oral care device (802); obtaining a known length of a portion of the oral care device; Process the image to: identifying a portion of the oral care device in the image (706); determining a relative length of the portion in the image (708); determining a calibration factor based on the relative lengths of the portions and the known lengths of the portions; determining a relative measurement of the user from the image (804); as well as The calibration factor is applied to the relative measurement to determine an absolute measurement for the user based on the identified portion of the oral care device and the relative measurement (106).

12. The method of claim 11, further comprising determining an orientation (602, 604) of the oral care device, wherein: The calibration factor is determined further based on an orientation of the oral care device.

13. A computer program product comprising computer program code which, when executed on a computing device having a processing system, causes the processing system to perform all the steps of the method according to any one of claims 8 to 12.

Images