Determination of the opening degree

Abstract

An oral care system having an oral care device and a processor is provided. The processor is configured to acquire an image of a user using the oral care device, process the image to identify a portion of the oral care device within the image, and determine a relative measurement of the user from the image. Thereafter, an absolute measurement of the user can be determined using the identified portion of the oral care device and the relative measurement.

Description

Technical Field

[0001] The present invention relates to the field of oral hygiene. In particular, the present invention relates to the field of detecting trismus and temporomandibular joint disorders.

Background Art

[0002] Trismus (commonly known as lockjaw) and temporomandibular joint (TMJ) disorders (TMJD) cause a reduction in jaw opening or restriction of movement. Trismus and TMJD can interfere with eating, speaking, and maintaining proper oral hygiene. If left untreated or undiagnosed, trismus and TMJD can become chronic and painful, and can also cause migraines, discomfort in the back, neck, and shoulders, and hearing impairment / loss, potentially affecting aspects of daily life and health.

[0003] Examination and treatment of trismus and TMJD typically require access to the oral cavity, which can be limited and in some cases impossible to achieve. Trismus can be caused by joint problems, infections, trauma, cancer treatment, or dental procedures. For example, temporomandibular joint disorder can cause trismus. In the United States, approximately 12% (about 35 million people) of the population suffers from TMJD at any given time.

[0004] Another significant cause of trismus is head and neck cancer, and in these cases, the prevalence of trismus is estimated to be as high as 38%. In the case of patients suffering from this type of cancer, the management of trismus focuses on preventing the progression of trismus and restoring mandibular function.

Summary of the Invention

Problems to be Solved by the Invention

[0005] Currently, there is no non-invasive mechanism for tracking the progression of trismus or the restoration of mandibular function to enable medical professionals to see how patients respond to treatment.

[0006] A common way to diagnose the severity of trismus is to measure the subject's opening via a tool such as a boley gauge or a ruler. A normal oral opening (i.e., without trismus) is expected to exceed 40 mm. Mild trismus is usually classified based on an opening of 30 - 40 mm, moderate trismus is usually classified based on an opening of 15 - 30 mm, and severe trismus is usually classified based on an opening of less than 15 mm.

[0007] Alternatively, a three - finger screening test can be used to estimate the severity of trismus. In the three - finger screening test, the subject tries to open their mouth as wide as possible so that fingers can be accommodated between the upper and lower teeth. The ability to accommodate more than three fingers generally indicates no trismus, two to three fingers indicate mild trismus, one to two fingers indicate moderate trismus, and less than one finger indicates severe trismus.

[0008] Both approaches are considered invasive, require another person for measurement, and do not provide medical professionals with the tools necessary to monitor the patient's progress after procedures such as extraction by a stranger.

[0009] US5097820A discloses the structure of a device that can be used to diagnose and treat opening limitations. However, such devices tend to be invasive and expensive. WO2017015342A1 discloses a trismus rehabilitation device similar to the device disclosed in US5097820A and includes additional sensors that measure the pressure applied to various parts of the jaw to estimate the opening.

[0010] The main limitation of these solutions is that their purpose is usually limited to single-spot measurements performed by dental healthcare providers in a clinic or a clinic, and as a result, the frequency of use of these solutions by the target is low. This reduces the benefit in large-scale monitoring of TMJD or trismus. Furthermore, it is difficult to digitize these measurements in a seamless workflow to enable long-term monitoring of disease progression.

[0011] Therefore, there is a need for improved means for detecting the presence of trismus or any opening / closing disorder and continuously tracking its progression, especially in a home environment.

[0012] US2020 / 359777 describes a dental device tracking method that includes acquiring at least a first image including an image of at least one user body part using an imager of a dental device.

Means for Solving the Problem

[0013] The present invention is defined by the claims.

[0014] According to an example according to one aspect of the present invention, an oral care system is provided, which an oral care device, and a processor, wherein the processor acquires an image of a user using the oral care device, processes the image to identify a portion of the oral care device within the image, determine relative measurements of the user from the image, determine absolute measurements of the user based on the identified portion of the oral care device and the relative measurements, the processor further obtains a known length of a portion of the oral care device, processes the image to determine a relative length (708) of the portion within the image, determines a calibration coefficient based on the relative length of the portion and the known length of the portion, The absolute measurement value is determined by applying the calibration coefficient to the relative measurement value.

[0015] This oral care device does not treat an opening disorder (e.g., trismus). For example, the oral care device can be for maintaining oral hygiene or a treatment device for treating a second oral hygiene disorder that is not an opening disorder.

[0016] When the oral care device is used on the palatal side of the front teeth, this generally forces the user to lower the jaw, which results in the user opening the mouth as wide as possible. Thus, an image of the user in this situation can be acquired, which provides an image of the user with the mouth opened as wide as possible.

[0017] However, the image itself cannot provide the measured value of the maximum opening. Therefore, in order to measure the maximum opening, a known length / size in the same image is required. It has been understood that a part of the oral care device can provide a known length and is always displayed in the image of the user using the oral care device. In other words, this part enables dynamic image rescaling or length calibration.

[0018] The length of the part can be known (e.g., from a cloud database with a computer-aided design (CAD) drawing with dimensions) or obtained (e.g., using a QR code or barcode displayed in the image). An additional sensor can also be used to measure the actual length of that part (e.g., using a depth sensor to generate a 3D rendering of an image that can provide the actual length of that part).

[0019] This provides high accuracy in length calibration when the oral care device is used on the frontmost tooth. The proximity of the oral care device and its part to the tooth greatly reduces the potential error in determining the maximum opening value. For example, when the calibrated length is used in the image, the calibrated length may not be at the same depth as the length measured with respect to the image sensor (in this case, the maximum opening degree).

[0020] In this way, the image of the oral care device being used on the palatal side of the front teeth provides an image that includes a convenient part of the oral care device (which can be used for calibration of the maximum opening value) at a position that provides high calibration accuracy when the user opens their mouth as wide as possible. Additionally, all of these are achieved while the user is using the oral care device for other primary purposes (such as oral hygiene like toothbrushing or tooth jetting), thereby not adding new activities to the user's routine.

[0021] The identified part of the oral care device is preferably a rigid part of the oral care device that does not substantially change shape when the oral care device is used.

[0022] The relative measurement value can be a relative opening value. The absolute measurement value can be an opening value.

[0023] The opening value is an absolute opening value representing the distance between the upper teeth and the lower teeth.

[0024] The relative measurement value may be a relative measurement value of the user's oral characteristics. For example, the oral characteristics may represent tooth misalignment or gum recession.

[0025] The image may be of the user performing a specific operation with the oral care device. The specific operation is a specific use of the oral care device. For example, the user can use the oral care device on the palatal side of the upper front teeth or the lingual side of the lower front teeth. In another example, the absolute measurement can be performed periodically (e.g., in real time: 20 - 60 times per second). Then, the values can be compared to each other (if necessary) to determine the maximum and / or minimum measurement values. In the case of more static measurements (such as facial feature measurements), a single absolute measurement may be sufficient.

[0026] The absolute measurement value may be the absolute opening degree. The relative measurement value may be a feature of the user within the image. Therefore, the absolute measurement value may be the absolute measurement value of that feature.

[0027] The system may further have a memory system that stores the length of a known part, and the processor obtains the length of the known part from the memory system.

[0028] The oral care device can have a sensor system configured to determine the orientation of the oral care device, and the processor is further configured to further determine a calibration coefficient based on the orientation of the oral care device.

[0029] The oral care device may not be exactly perpendicular to the image sensor, and thus the length-calibrated element may appear distorted in the image. This usually occurs because one part of the oral care device is used inside the mouth on the palatal side of the teeth (i.e., inside the oral cavity), while another part of the oral care device is held by the user and is outside the oral cavity. Therefore, knowledge of the orientation of the oral care device (e.g., relative to the gravity vector) can be used to improve the calibration coefficient.

[0030] Similarly, the image may be taken at an irregular angle and angle correction can be applied to the image. This can be achieved by comparing the angle of the toothbrush obtained from the sensor with the angle of the toothbrush seen in the image.

[0031] The orientation has at least one angle measurement. The measurement of the angle may be relative to gravity and / or relative to the teeth. For example, the angle measurement value may have a pitch angle measurement value, a roll angle measurement value, and / or a yaw angle measurement value of the oral care device. In some cases, a change in the pitch angle measurement value may be sufficient to determine the opening value.

[0032] The processor may be further configured to apply a feature recognition algorithm to the image to identify the part of the oral care device within the image and measure the relative length of the identified part within the image.

[0033] The processor may be further configured to apply a feature recognition algorithm to the image to identify the user's oral cavity within the image and measure the user's relative maximum opening degree from the identified oral cavity within the image.

[0034] The oral care device portion may have at least two markers disposed on the oral care device, and the distance between the markers is known. One marker of a known dimension (e.g., diameter or length) can also be used.

[0035] The oral care device can be a toothbrush, an oral irrigator, or a toothbrush with an oral irrigator.

[0036] The oral care device may include one or more cameras, and at least one of the cameras is configured to acquire an image of a user using the oral care device.

[0037] In the case of a toothbrush, the camera can be arranged facing the same general direction as the bristles. As a result, an image of the user using the oral care device is obtained when the user is brushing the palatal side of the frontmost teeth of the upper jaw. Similarly, the camera can be arranged facing the opposite direction of the bristles. As a result, an image of the user using the oral care device is obtained when the user is brushing the lingual side of the bottom teeth.

[0038] The processor may be further configured to identify the user's physiological markers within the image, and determining the absolute measurement value of the user is further based on the physiological markers.

[0039] Various physiological markers of a user can be used to calibrate the length of the user's measurement. The user's measurement may have a measurement of a user characteristic of an unknown size (e.g., an unknown length or height). The user's physiological marker can have a known size, and the known size can be used to determine an absolute measurement from a relative measurement. This can be achieved by using a calibration factor between the known size of the physiological marker and the relative size (e.g., in pixel units) of the physiological marker measured from the image. The known size may be previously measured or estimated (e.g., from the size measurements of other users with similar physiological characteristics). Physiological markers can include the distance between a user's eyes, the distance from the eyes to the mouth, the width of the mouth, the size of the nose, etc. One or more of many physiological markers of a user (e.g., markers used in forensic facial reconstruction) can be used.

[0040] The relative measurement may be a relative aperture, and the absolute measurement may be an absolute aperture.

[0041] The present invention also provides a method for determining an absolute measurement of a user, the method comprising: acquiring an image of a user using an oral care device; obtaining a known length of a portion of the oral care device; processing the image to identify a portion of the oral care device within the image, determine a relative length (708) of the portion within the image, determine a calibration factor based on the relative length of the portion and the known length of the portion; determining a relative measurement of the user from the image; and determining an absolute measurement of the user based on the identified portion of the oral care device and the relative measurement by applying the calibration factor to the relative measurement.

[0042] The method may further include the step of determining the orientation of the oral care device, and the determination of the calibration coefficient is further based on the orientation of the oral care device.

[0043] The present invention also provides a computer program product including computer program code which, when executed on a computing device having a processing system, causes the processing system to execute all steps of the aforementioned method.

[0044] These and other aspects of the present invention will become apparent from the embodiments described hereinafter and will be described with reference to the embodiments.

Brief Description of the Drawings

[0045]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Modes for Carrying Out the Invention

[0046] For a better understanding of the present invention and to more clearly show the method of its implementation, reference is made to the accompanying drawings which are merely illustrative.

[0047] The present invention will be described with reference to the drawings.

[0048] It should be understood that the detailed description and specific examples, while indicating exemplary embodiments of the apparatus, system, and method, are for the purpose of illustration only and are not intended to limit the scope of the present invention. These and other features, aspects, and advantages of the apparatus, system, and method of the present invention will be more preferably understood from the following description, the appended claims, and the accompanying drawings. It should be understood that the figures are merely schematic and not drawn to scale. It should also be understood that the same reference numbers are used throughout the figures to indicate the same or similar parts.

[0049] The present invention provides an oral care system having an oral care device and a processor. The processor is configured to acquire an image of a user using the oral care device, process the image to identify a portion of the oral care device within the image, and determine relative measurements of the user from the image. Thereafter, an absolute measurement of the user can be determined using the identified portion of the oral care device and the relative measurements.

[0050] A system for obtaining the degree of mouth opening of a user by using an oral care device is proposed. This enables, for example, a clinician to detect when a user has an opening disorder and to estimate its progression through continuous use of the system.

[0051] This facilitates continuous remote monitoring of TMJ disorders or trismus and enables medical and dental professionals to evaluate and quantify the results of the treatments provided. An additional advantage is that medical insurance companies and dental insurance companies will have additional data points for verifying benefit claims related to TMJ disorders or trismus by evaluating the date of the treatment billed and the date on which trismus was detected.

[0052] Figure 1 shows an open mouth. As can be seen, the upper teeth 102 do not uniformly separate from the lower teeth 104, and thus the openings (lines 106 and 108) can be different based on the location where the opening measurement is taken. Currently, the opening value / measurement is most often taken at the front of the mouth (i.e., line 106). This is because it is the most easily accessible measurement from the outside and thus a clinician or other person can measure the user's opening.

[0053] However, it should be understood that the opening value can be obtained at any location within the oral cavity in a manner that can be interpreted by a clinician. For example, line 108 (near the center of the mouth) can be used in combination with a specific location within the mouth. In contrast, when the location is unknown, the processor can acquire multiple measurements and find the maximum value of those measurements. The maximum measurement is typically interpreted as the opening at line 106 (the front of the mouth) because this is usually the location with the maximum opening.

[0054] The maximum opening is currently one of the most important openings measured. The maximum opening quantifies how widely a user can open their mouth. A relatively small maximum opening indicates the possibility of an opening disorder. Similarly, the minimum opening (i.e., the maximum closing) is also a relevant opening. If a user cannot fully close their mouth, this results in a relatively large minimum opening, indicating the possibility of an opening disorder.

[0055] Note that Figure 1 shows a general opening. In reality, the maximum opening (and potentially the minimum opening) is typically more clinically relevant than the general opening. However, if accurate and frequent opening measurements are obtained, the full range of openings can be relevant.

[0056] The inventors have noticed that using an oral care / hygiene device (such as a toothbrush, floss device, jet device, etc.) is used entirely within the user's oral cavity and is typically used frequently (e.g., more than once a day). Further, due to the nature of using an oral care device in the user's oral cavity, the user typically keeps their mouth open during use. Thus, it has been proposed to measure the user's degree of mouth opening during use of the oral care device. This enables the user to do two things simultaneously (i.e., maintain oral hygiene and measure the degree of mouth opening). It also means that opening values can be obtained much more frequently than if the user or typically a clinician had to perform the measurement of opening separately.

[0057] The following examples relate to the measurement of maximum mouth opening. However, it should be understood that the systems and methods described in relation to the measurement of maximum mouth opening can also be applied to any general measurement of mouth opening. In specific examples where the system or method is particularly related to maximum mouth opening, alternatives can be provided for minimum mouth opening or general mouth opening.

[0058] Similarly, the following examples use a toothbrush to illustrate the example. However, it should be understood that other oral care devices can also be used.

[0059] In a first embodiment, the sensor is disposed on the head of the toothbrush. FIG. 2 shows a toothbrush head 202 with a sensor system 206 disposed on the head of the toothbrush 202. The sensor system is disposed at the distal end of the toothbrush head 202 near the bristles 204 of the toothbrush head 202. In an alternative embodiment, the sensor system can also be present on the handle of the oral care device.

[0060] The quantitative measure of opening can depend on where the measurement is taken along the patient's jaw. This is because the mouth opens widest at the front and narrowest at the back.

[0061] Figure 3 shows a toothbrush head 202 equipped with a sensor system 206 being used within the oral cavity. In this case, the sensor system can have a proximity sensor and / or a distance sensor and / or an acceleration sensor (inertial measurement) and / or a gyroscope. To estimate the opening level at any position within the oral cavity, the sensor system determines the position of the brush head along a given jaw from distance measurements taken by the sensor system 206 attached along its longitudinal axis (i.e., along segment AB shown in Figure 3). The opening degree at the current brush head position is determined from distance measurements obtained from a sensor attached along the vertical axis (i.e., along segment BC shown in Figure 3).

[0062] The elevation angle can be calculated at the current position (i.e., angle ∠BAC = ∠BAD). As can be seen, the isosceles triangle BAD can also be used. Point D can be found using simple geometry from points A, B, and C.

[0063] One skilled in the art will understand that the calculation of segment BD is straightforward. Given the measurement of segment BC, an approximate isosceles triangle (ABD) can be created, from which basic trigonometric functions enable the deduction of BD. Thus, segment AD is calculated by a simple application of the Pythagorean theorem for triangle ABC.

[0064] Although the planes of both the upper and lower jaws are not straight lines, by making the measurement when the brush head is as far back as possible in the occlusal plane, local linearity can be reasonably expected, and as a result, by calculating the distance represented by line segment BD, the distal opening degree in the mouth can be approximated.

[0065] Therefore, the aperture at any other arbitrary point including the maximum aperture can be estimated using the elevation angle. If both BD and the angle ∠BAD at its particular given point are known, it is possible to estimate the aperture at other points. It is expected that the non-linear planes of both jaws may limit the accuracy of the proposed approach. However, since the range of the opening is large enough (e.g., 15 mm in moderate trismus), the estimates made using this method are expected to be well within the acceptable range.

[0066] For example, FIG. 4 shows the mouth of an isosceles triangle (AB'D'). As can be seen, despite the irregular nature of the tooth shape, the triangle fits surprisingly well to the shape of the opening. Thus, it can be concluded that the isosceles triangle is a good descriptor of the opening. This leads to the conclusion that the angle ∠B'AD' = ∠BAD can also be accurately used to represent the opening. Thus, the opening value is not limited to the distance measurement between the upper tooth 102 and the lower tooth 104, and the elevation angle can also be used as the opening value.

[0067] The first embodiment can be realized in a sensor system having a two-sensor configuration in which the sensitive axes of both sensors are perpendicular to each other. The directions of the sensors are shown by segments BA and BC in FIG. 3. Generally, the first example uses triangulation to estimate the opening value.

[0068] In the second embodiment, the opening value is determined using sensor data from a sensor system already present in the current toothbrush. In this embodiment, measurements are taken at specific brushing positions and directions (e.g., behind the upper or lower molar teeth). In particular, the second embodiment allows measurements to be taken during an accurate and specific movement that results in the desired opening (e.g., maximum or minimum opening).

[0069] The first embodiment utilizes sensors added to the brush head to facilitate triangulation measurement of the opening, but it has been found that it is also possible to evaluate and track the opening degree using sensors already present in a (powered) oral care device. For example, accelerometers, inertial measurement units, rotational sensors (such as compasses, gyroscopes, etc.), displacement sensors, position sensors, Hall sensors, and / or printed strain gauges may already be present in a powered oral care device.

[0070] For the sake of clarity, this description and the figures show a powered toothbrush used as an example. However, any other toothbrush (e.g., a combined brushing and flossing device), water jet device, or flossing device is also conceivable. In particular, water jet devices are very suitable. This is because the head of a device equipped with a jet tube is quite slender, which can reduce measurement problems (e.g., there is no problem of the wide device body of a toothbrush interfering with the measurement).

[0071] During a specific brushing motion, namely while brushing the back side of the rearmost molars / wisdom teeth, it has been found that the user is forced to open their mouth maximally and rotate the brush at an angle until the brush is strongly pressed against the jaw on the side opposite the teeth being brushed.

[0072] The purpose of this embodiment is to measure this brush movement and interpret this measurement value as representing the opening degree. Limited movement or an opening that decreases over time may be related to an opening disorder (such as TMJ, trismus, etc.).

[0073] Measurements taken during the cleaning of the upper or lower teeth both, in principle, result in an interpretation of the opening, but the inventors have noticed that it may be preferable to use the measurement while brushing the back side of the rearmost molars / wisdom teeth in the upper jaw.

[0074] While brushing the upper jaw, the opening is caused by the jaw dropping, and the brush remains in the same position as its initial flat position relative to the upper teeth, just as when brushing along the upper surface of the upper teeth. From this fixed reference position, the brush must cross the full opening angle of the mouth before it is pushed against the lower jaw. As a result, the measured movement becomes a direct measure of the opening.

[0075] In contrast, while brushing the lower jaw, the opening is also caused by the jaw dropping. However, in this case, the jaw dropping separates the brush from the jaw without moving the brush. From this starting position, the brush is only required to cross a portion of the opening angle before it is pushed against the upper jaw. As a result, the measured movement may not be a direct measure of the opening. However, it should be noted that this measurement may have some predictive value for vertical measurements.

[0076] Figures 5 and 6 show the toothbrush 502 being used on the rearmost teeth of the upper jaw 506. To establish the degree of opening during brushing of the upper jaw 506, the following measurements are appropriate means. Measurement of the total movement of the sensor system 504 (including, for example, an accelerometer and / or a position sensor) as the brush rotates from the upper jaw 506 (shown in FIG. 5) to the lower jaw 508 (shown in FIG. 6). Measurement of the rotation angle of the sensor system 504 (including, for example, a compass and / or a gyroscope) as the toothbrush 502 rotates from the upper jaw 506 (shown in FIG. 5) to the lower jaw 508 (shown in FIG. 6). If necessary, the rotation angle can be converted to the movement amplitude by considering the distance of the sensor system 504 from the top of the brush head to the sensor system 504.

[0077] In many use cases (such as establishing the progression of TMJD), it is sufficient to monitor the measured movement of the brush sensor or the change in the rotation angle.

[0078] However, in some use cases (e.g., establishing the severity of TMJD / trismus), it may be preferable to establish the absolute opening degree. To achieve this, at the position on the brush where the brush moves away from the mouth, the magnitude of the distance that the brush traverses can be established. This can be obtained as follows.

[0079] Movement of the brush during opening = Measured movement of the sensor system 504 between points E and F in FIG. 6 × Ratio of the depth of the mouth (D - opening) to the distance between the sensor system 504 and the brush head (D - sensor). The depth of the mouth (D - opening) is the distance between the frontmost tooth and the rearmost tooth in the mouth.

[0080] When calculating the absolute opening degree, the size of the brush (e.g., diameter, thickness, etc.) (brush diameter) at the position on the brush where the brush moves away from the mouth can be considered. For example, opening degree = Distance that the brush traverses at the opening position + Brush diameter.

[0081] It should be noted that in this absolute opening degree measurement, it is advantageous that the part of the toothbrush entering the mouth is relatively narrow and has a constant diameter. For example, a relatively long toothbrush protrudes from the mouth even when placed behind the last tooth. The jet injection device is particularly suitable because it has an essentially elongated tube for injection purposes.

[0082] It is also possible to establish the most likely moment when associating the movement of the brush with the opening. Specifically, when the sensor system 504 has a motion sensor or a position sensor (or the like), the following patterns are likely to represent preferred measurement moments. The brush is at the position where it polishes the upper surface of the upper teeth. The brush moves backward along the teeth. The movement of the brush stops (motion sensor), or the position is at or behind the position of the posterior molar / wisdom tooth (position sensor) and is kept constant. The movement of the brush that potentially represents the opening due to the lowering of the jaw is measured.

[0083] In certain usage examples, it is established that a particular brush head is used, especially for polishing the back side of the molar teeth (i.e., a single tuft brush head). Higher weights can be given to the measurements taken in this particular usage example. This is because the user is most likely to concentrate on cleaning the back side of the molar teeth and attempt to fully open the mouth.

[0084] Obviously, the user does not need to perform this operation every time they brush. This is because these jaw-related problems only progress slowly over time. However, if a relatively long period has passed since the measurement became possible, in an alternative approach, the user can also be instructed to perform this particular operation by placing the tip of the device as described above, opening the jaw, and moving the device parallel to the jaw opening. In such a manner, a good reference measurement can always be established. The user can be instructed via an external device (such as a smartphone app).

[0085] Existing oral care devices use an algorithm to determine the orientation of the oral device with respect to the gravity vector. This will be used to determine the change in angle when opening the mouth. Directions 602 and 604 are shown in FIG. 6. Orientation 602 is the orientation of the toothbrush as shown in FIG. 5. Point E indicates the arrangement of the sensor system 504 corresponding to the toothbrush orientation 602. Similarly, orientation 604 is the orientation of the toothbrush as shown in FIG. 6 corresponding to the sensor system 504 being at point F. Thus, the angle Θ between the two directions can be determined.

[0086] In some cases, the conversion from angle measurement to opening distance (requiring "jaw length") may not be necessary here. In fact, the opening angle may be more accurate (because it requires fewer measurements) and may contain more information than the opening distance measurement. Opening distance measurements (generally in mm units) are commonly used. This is because it is the most easily evaluated in current practice.

[0087] The robustness against head movement is also a point worthy of attention. When detecting a change in orientation, the opening may be confused with the tilt of the head. One approach to address this is to use an algorithm trained to distinguish between "opening the mouth" and "tilting the head". Such an algorithm is currently used to determine the position of the toothbrush in the oral cavity.

[0088] For example, position detection algorithms that utilize sensor signals from an inertial measurement unit (IMU) have been used previously. These algorithms estimate the tilt of the head using the average or median direction of the brush over time. Subsequently, the tilt of the head is used to obtain a better estimate of the brush's orientation relative to the user's head. In some cases, each time the user changes the operation to another segment of the mouth, the direction of the head tilt can be re - estimated (based on the direction of the brush).

[0089] The second approach is to use a pressure sensor to determine when the brush head is pressed against the teeth and when the opening is due to "polishing the back of the molars". The third approach is to add an acoustic sensor to the sensor system. Oral resonance causes an increase in volume and a change in spectrum. These are different based on whether the mouth is open or closed.

[0090] In the third embodiment, an image algorithm with dynamic image rescaling and calibration is used to determine the (maximum) opening degree at a specific brushing position and direction (lingual incisor brushing).

[0091] Figure 7 shows that the toothbrush 702 is used on the lingual side of the upper front teeth 710. The toothbrush has two markers 706 and 707 with a known length 708 between the markers 706. A camera 712 is used to take an image of this operation.

[0092] The third embodiment provides a software-based measurement of the opening degree by using a calibrated toothbrush (or a part thereof) and a camera (e.g., on a smartphone) without requiring a sensor on the toothbrush 702. The camera can be part of the handle (distal end) of the toothbrush, while during brushing, the camera faces the bathroom mirror and can take pictures of the opening in such a manner.

[0093] The opening is measured at a specific brushing orientation / position that provides a reliable measurement of the opening width. During a specific brushing motion (e.g., brushing behind the upper incisors or lower incisors), as shown in FIG. 7, it has been found that the user is forced to rotate the brush at a specific angle until the mouth is opened maximally and the brush is strongly pressed against the lingual side of the upper or lower jaw so that the teeth are being brushed.

[0094] By knowing the distance between (at least) one (reference) marker 706 of the length-calibrated toothbrush shaft and identifying the marker and the opening (e.g., using a feature detection algorithm and / or a feature matching algorithm), for example, accessing a database with the dimensions of a known brush head or reference marker, and obtaining an image calibration or image scaling factor, the absolute opening degree can be determined. The marker 706 can be a colored / texture dot added to the toothbrush or a marker specific to the brush head (e.g., an ejector mark from an injection molding process). A single marker 706 (e.g., of a known diameter) can be used.

[0095] A database containing a reference length 708 can be accessed, which enables the image to be dynamically rescaled. The known length enables a reliable determination of the absolute maximum opening value (e.g., when an image is taken in a subsequent week). This is because a person cannot reliably control the distance at which the image is taken (i.e., correction for different image magnifications and automatic scaling are performed).

[0096] The toothbrush 702 may be connected to a database of design data (such as drawings and dimensions) of the brush head including the dimensions (e.g., in mm) of the toothbrush (or a part thereof). A radio frequency identification (RFID) code can be used to recognize the type of toothbrush to be worn.

[0097] An image can be acquired while the toothbrush is in use, enabling the determination of the opening degree. FIG. 8 shows an image used to determine the opening degree. The user can brush their teeth and take an image (e.g., via the camera of a smartphone) when the incisors are being brushed. Alternatively, the camera can be part of a bathroom mirror.

[0098] Next, the processor or cloud engine executes the following algorithm steps to determine the absolute maximum opening degree from the acquired image as shown in the processed image of FIG. 8. Decompose the RGB image into RGB channels. Apply feature recognition functions (such as edge detection and boundary detection) to distinguish the features of the brush head and oral features (e.g., lips, teeth). See FIG. 8. Identify markers 706 and 707 and measure the relative distance (measured in pixels) between the two markers. Identify the relative opening degree 804 (measured in pixels). Rescale the uploaded image based on the boundary-segmented features, the known distance 708 (e.g., measured in mm) of marker 706 on the brush head, and the relative measurement value. Using the relative distance between the identified markers 706 and 707 in the image and the known distance 708, a calibration coefficient can be determined (e.g., distance / pixel). Apply the length calibration coefficient to the relative opening degree 804 to determine the absolute maximum opening degree.

[0099] As an option, by considering the influence of the brushing angle on the determination of the absolute opening degree, the sensor system 704 already present in the toothbrush 702 can be used to improve the accuracy of the measurement. Note that the lower incisors and the upper incisors are brushed at different pitch angles.

[0100] The camera 712 can also be arranged on the toothbrush 702. This takes advantage of the fact that the user usually brushes their teeth in front of a mirror. Thus, the camera 712 can be arranged on the toothbrush, and as a result, it takes an image of the mirror in front of the user, which reflects the user while brushing their teeth. This eliminates the need to use an external camera during toothbrushing.

[0101] Figure 9 shows a method for determining the absolute opening degree. The method includes, in step 902, receiving an image of the user while brushing their teeth, in step 904, measuring the (relative) length (in pixels) between markers (or other parts) on the toothbrush, and in step 906, using the measured length between the markers and the known length between the markers, for example, extracted from a database, to determine a calibration coefficient 906 (such as pixels / mm or distance / pixels). In step 908, the (relative) opening degree is also determined or measured from the image (in pixels). Thus, the calibration coefficient can be applied to the (relative) opening degree to determine the absolute opening degree. Note that the opening degree may be the maximum opening degree, the minimum opening degree, or any general opening degree.

[0102] A person skilled in the art would be able to easily develop a processor for executing any of the methods described in this document. Thus, each step of the flowchart may represent different operations executed by the processor and can be executed by individual modules of the processor.

[0103] As described above, this system utilizes a processor for data processing. The processor can be implemented in various ways using software and / or hardware to execute the various required functions. The processor typically uses one or more microprocessors and can be programmed using software (e.g., microcode) to execute the necessary functions. The processor can be implemented as a combination of dedicated hardware for executing some functions and one or more programmed microprocessors and associated circuitry for executing other functions.

[0104] Examples of circuits that can be employed in various embodiments of the present application include, but are not limited to, conventional microprocessors, application specific integrated circuits (ASICs), and field programmable gate arrays (FPGAs).

[0105] In various implementations, the processor can be associated with one or more storage media such as volatile and non-volatile computer memories such as RAM, PROM, EPROM, and EEPROM. The storage media may be encoded with one or more programs that execute the required functions when executed by one or more processors and / or controllers. The various storage media may be fixed within the processor or controller, or may be transportable such that one or more programs stored therein can be loaded into the processor.

[0106] Modifications to the disclosed embodiments can be understood and implemented by those skilled in the art of practicing the invention claimed in the claims from a consideration of the figures, disclosure, and appended claims. In the claims, the word "comprising" does not exclude other elements or steps, and the indefinite article "a" or "an" does not exclude a plurality.

[0107] The functions implemented by a processor can be implemented by a single processor or by a plurality of individual processing units that are considered to constitute the "processor" together. In some cases, such processing units are separated from each other and can communicate with each other either wired or wirelessly.

[0108] The mere fact that certain means are recited in mutually different dependent claims does not indicate that a combination of these means cannot be used advantageously.

[0109] A computer program can be stored / distributed on a suitable medium such as an optical storage medium or a solid-state medium supplied together with or as part of other hardware, but can also be distributed in other forms, such as via the Internet or other wired or wireless communication systems.

[0110] It should be noted that, when the term "adapted to" is used in the claims or the specification, the term "adapted to" is intended to be equivalent to the term "configured to". It should also be noted that, when the term "arrangement" is used in the claims or the specification, the term "arrangement" is intended to be equivalent to the term "system", and vice versa.

[0111] Any reference signs in the claims shall not be construed as limiting the scope of the invention.

Claims

1. An oral care system, Oral care device, It has a processor, and the processor is The image of the user using the oral care device is acquired, Process the image, Identify the portion of the oral care device in the aforementioned image, The relative measurement values ​​of the user are determined from the aforementioned image. Based on the identified portion of the oral care device and the relative measurement value, the absolute measurement value of the user is determined. The aforementioned processor further, Obtaining a known length of the part of the oral care device, The image is processed to determine the relative length of the portion within the image. A calibration coefficient is determined based on the relative length of the aforementioned part and the known length of the aforementioned part. An oral care system that determines the absolute measurement value by applying the calibration coefficient to the relative measurement value.

2. The system according to claim 1, further comprising a memory system for storing a known length of the portion, wherein the processor obtains the known length of the portion from the memory system.

3. The system according to claim 1 or 2, wherein the oral care device has a sensor system for determining the orientation of the oral care device, and the processor further determines the calibration coefficient based on the orientation of the oral care device.

4. The aforementioned processor further, A feature recognition algorithm is applied to the image to identify the portion of the oral care device within the image. The system according to any one of claims 1 to 3, for measuring the relative length of the identified portion in the image.

5. The aforementioned processor further, A feature recognition algorithm is applied to the image to identify the user's mouth in the image. The system according to any one of claims 1 to 4, which measures the relative maximum mouth opening of the user from the identified oral cavity in the image.

6. The system according to any one of claims 1 to 5, wherein the part of the oral care device has at least two markers arranged in the oral care device, and the distance between the markers is known.

7. The system according to any one of claims 1 to 6, wherein the oral care device is a toothbrush, an oral irrigator, or a toothbrush equipped with an oral irrigator.

8. The system according to any one of claims 1 to 7, wherein the oral care device has one or more cameras, and at least one of the cameras acquires an image of the user using the oral care device.

9. The system according to any one of claims 1 to 8, wherein the processor further identifies the user's physiological markers in the image and determines the user's absolute measurement value, the determination being based on the physiological markers.

10. The system according to any one of claims 1 to 9, wherein the relative measurement value is a relative aperture and the absolute measurement value is an absolute aperture.

11. A computer program, when executed on a computing device having a processing system, causes the processing system to perform a method for determining an absolute measurement of a user, wherein the method is: The steps include: acquiring an image of the user using the oral care device, The steps include obtaining a known length of the portion of the oral care device, After processing the aforementioned image, Identify the portion of the oral care device in the aforementioned image, Determine the relative length of the portion in the aforementioned image, A calibration coefficient is determined based on the relative length of the aforementioned part and the known length of the aforementioned part. The steps include determining the user's relative measurements from the aforementioned image, A computer program comprising the steps of applying the calibration coefficient to the relative measurement value and determining the absolute measurement value of the user based on the identified portion of the oral care device and the relative measurement value.

12. The computer program according to claim 11, wherein the method further comprises the step of determining the orientation of the oral care device, and the determination of the calibration coefficient is further based on the orientation of the oral care device.

Images