Toothbrush system, notification method, and program

Abstract

A toothbrush system comprises: an acquisition unit (101) that acquires moving image data obtained by imaging a state in which teeth or gums are being brushed with a toothbrush; a determination unit (102) that determines, from a still image included in the moving image data, whether a deformation state of bristles or bristle bundles of the toothbrush exceeds a prescribed range; and a notification unit (103) that notifies a user on the basis of the determination result that the deformation state exceeds the prescribed range.

Description

Toothbrush System, Notification Method, and Program 【0001】 The present disclosure relates to a toothbrush system, a notification method, and a program. 【0002】 Patent Document 1 discloses a configuration that detects brushing time, brushing pressure, and the left - right direction of the brush and displays the achievement degree of brushing for each side. Patent Document 2 discloses a configuration in which a timer operates when the toothbrush is taken out from the charging stand to measure the brushing time. Patent Document 3 discloses an idea of detecting the direction around the axis of the toothbrush body in four or eight steps and estimating the brushing site from the detection result. 【0003】 Japanese Utility Model Publication No. 61 - 77827, Japanese Patent Application Laid - Open No. 5 - 329024, Japanese Patent Application Laid - Open No. 2018 - 134241 【0004】 In the above - mentioned prior art, it is difficult to assist brushing so as not to damage teeth or gums. 【0005】 The present disclosure provides a toothbrush system that can assist brushing so as not to damage teeth or gums. 【0006】 A toothbrush system according to an aspect of the present disclosure includes an acquisition unit that acquires moving image data obtained by photographing a state in which teeth or gums are being brushed with a toothbrush, a determination unit that determines whether or not a deformed state of bristles or bristle bundles of the toothbrush exceeds a predetermined range from a still image included in the moving image data, and a notification unit that notifies a user based on a determination result that the deformed state exceeds the predetermined range. 【0007】 A toothbrush system and the like according to the present disclosure can assist brushing so as not to damage teeth or gums. 【0008】Figure 1 is an external view of the main body of the toothbrush system according to the embodiment. Figure 2 is an enlarged view of the head according to the embodiment. Figure 3 is an external view when the toothbrush part according to the embodiment is attached to the main body. Figure 4 is an external view when the toothbrush part according to the embodiment is attached to the main body. Figure 5 is a schematic configuration diagram of the toothbrush system according to the embodiment. Figure 6 is a functional block diagram of the portable terminal according to the embodiment. Figure 7 is a diagram showing an example of a still image when the toothbrush part is not pressed against the teeth or gums. Figure 8 is a diagram showing an example of a still image when the toothbrush part is pressed against the teeth or gums with excessive pressure (brushing state). Figure 9 is a diagram to explain another example of the detection area. Figure 10 is a diagram showing an example of a still image when the toothbrush part is not pressed against the teeth or gums. Figure 11 is a diagram showing an example of a still image when the toothbrush part is pressed against the teeth or gums with excessive pressure (brushing state). Figure 12 is a diagram showing the flow of intraoral imaging operation in the toothbrush system. 【0009】 A toothbrush system according to a first aspect of this disclosure includes: an acquisition unit that acquires motion image data of a state in which a tooth or gum is being brushed with a toothbrush; a determination unit that determines from a still image included in the motion image data whether or not the deformation state of the bristles or bundle of bristles of the toothbrush exceeds a predetermined range; and a notification unit that notifies the user based on the determination result that the deformation state exceeds the predetermined range. 【0010】 According to this, the user can be notified if the deformation of the toothbrush bristles or bristle bundles exceeds a predetermined range. For example, the user can be notified if they are applying excessive pressure to their teeth or gums with the toothbrush. Therefore, it is possible to prevent the user from applying excessive pressure to their teeth or gums with the toothbrush and to assist in brushing in a way that does not damage the teeth or gums. 【0011】A toothbrush system according to a second aspect of the present disclosure is a toothbrush system according to a first aspect, wherein the determination unit detects the entry of a part of the bristles or bristle bundle into a detection area in the still image, which is an area located at a predetermined distance from the position of the bristles or bristle bundle of the toothbrush when it is not being brushed, and determines, based on the entry, that the deformation state exceeds the predetermined range. 【0012】 Therefore, by performing image processing, it is possible to detect when a user is applying excessive pressure to their teeth or gums with a toothbrush. In other words, since it is possible to detect when a user is applying excessive pressure to their teeth or gums with a toothbrush without installing a pressure sensor in the toothbrush, a toothbrush with reduced costs can be used. 【0013】 A toothbrush system according to a third aspect of the present disclosure is a toothbrush system according to a second aspect, wherein the determination unit determines the deformation state based on each of a plurality of still images included in the motion data, and the notification unit makes the notification based on the fact that the frequency of entry has exceeded a predetermined threshold. 【0014】 According to this, for example, it is possible to suppress notifications when it is determined that excessive pressure has been applied instantaneously. Therefore, notifications due to false detections can be suppressed. 【0015】 A toothbrush system according to a fourth aspect of the present disclosure is a toothbrush system according to a second or third aspect, wherein the determination unit sets the detection area based on the type of toothbrush. 【0016】 Therefore, based on a detection area appropriately set according to the type of toothbrush, it is possible to determine whether the deformation exceeds a predetermined range. Thus, the deformation can be appropriately determined according to the type of toothbrush. 【0017】A toothbrush system according to a fifth aspect of the present disclosure is a toothbrush system according to a first aspect, further comprising a storage unit that stores a learning model obtained by machine learning using training data that associates a test image of a state in which the tooth or gum is being brushed with the toothbrush and the pressure applied by the toothbrush to the tooth or gum in the state of the test image, and the determination unit applies the still image included in the moving image data to the learning model to identify the pressure as the deformation state, and determines that the deformation state has exceeded a predetermined range based on the identified pressure exceeding a predetermined threshold. 【0018】 Therefore, a determination process using a learning model can be used to determine whether or not the deformation state exceeds a predetermined range. 【0019】 A toothbrush system according to a sixth aspect of this disclosure is a toothbrush system according to a fifth aspect, wherein the learning model is machine-learned using training data set for each type of toothbrush. 【0020】 Therefore, based on a machine learning model that uses appropriately set training data according to the type of toothbrush, it is possible to determine whether the deformation state exceeds a predetermined range. Thus, the deformation state can be appropriately determined according to the type of toothbrush. 【0021】 A notification method according to a seventh aspect of this disclosure is a notification method performed by a toothbrush system, which acquires video data of a state in which a tooth or gum is being brushed with a toothbrush, determines from a still image included in the video data whether the deformation state of the bristles or bristle bundle of the toothbrush exceeds a predetermined range, and notifies the user based on the determination that the deformation state exceeds the predetermined range. 【0022】According to this, the user can be notified if the deformation of the toothbrush bristles or bristle bundles exceeds a predetermined range. For example, the user can be notified if they are applying excessive pressure to their teeth or gums with the toothbrush. Therefore, it is possible to prevent the user from applying excessive pressure to their teeth or gums with the toothbrush and to assist in brushing in a way that does not damage the teeth or gums. 【0023】 The program relating to the eighth aspect of this disclosure is a program that causes a computer to execute the notification method relating to the seventh aspect. 【0024】 These comprehensive or specific embodiments may be implemented as a system, method, integrated circuit, computer program, or recording medium such as a computer-readable CD-ROM, or as any combination of a system, method, integrated circuit, computer program, and recording medium. 【0025】 The embodiments will be described in detail below, with reference to the drawings as appropriate. However, unnecessarily detailed explanations may be omitted. For example, detailed explanations of already well-known matters and redundant explanations of substantially identical configurations may be omitted. This is to avoid the following explanation becoming unnecessarily verbose and to facilitate understanding for those skilled in the art. 【0026】 The inventors provide the accompanying drawings and the following description so that those skilled in the art may fully understand this disclosure, and they are not intended to limit the subject matter described in the claims. 【0027】 (Embodiment) [1. Configuration] [1-1. Configuration of the main body] Figure 1 is an external view of the main body 10 in the toothbrush system according to this embodiment. As shown in Figure 1, the main body 10 has a toothbrush-shaped housing that can be handled with one hand, and the housing includes a head portion 10a (also called the tip portion) that is placed in the user's oral cavity when taking dental scans, a handle portion 10b that the user grasps, and a neck portion 10c that connects the head portion 10a and the handle portion 10b. 【0028】Figure 2 is an enlarged view of the head unit 10a shown in Figure 1. The head unit 10a comprises an imaging optical system 12 (also called the imaging unit) and a plurality of LEDs (first to fourth LEDs) 26A to 26D. 【0029】 For example, the imaging optical system 12 is incorporated into the head portion 10a and the neck portion 10c. For example, the imaging optical system 12 includes an image sensor and a lens arranged on its optical axis. 【0030】 The image sensor is an imaging device such as a CMOS (Complementary Metal-Oxide-Semiconductor) sensor or a CCD (Charge Coupled Device) element. 【0031】 The imaging optical system 12 may also include a blue light cut filter that cuts out the blue wavelength light component contained in the light incident on the image sensor. 【0032】 The multiple LEDs 26A to 26D are illumination devices (light sources) that irradiate light onto the tooth being photographed during imaging. The multiple LEDs 26A to 26D are, for example, blue LEDs (Light Emitting Diodes) that emit blue light with a peak wavelength of 405 nm. As shown in Figure 2, in this embodiment, the multiple LEDs 26A to 26D are arranged to surround the entrance of the imaging optical system 12. At least some of the multiple LEDs 26A to 26D may be white LEDs. In addition, any light source may be used instead of LEDs. 【0033】 [1-2. Attachments] The toothbrush system according to this embodiment has one or more attachments that can be attached to or removed from the main body 10. The toothbrush system also has an oral camera mode for taking pictures of the condition inside the oral cavity and a toothbrush mode for displaying the condition inside the oral cavity while brushing teeth. 【0034】 In toothbrush mode, the toothbrush part 11a, which is one of the attachments, is attached to the main body 10. Figure 3 is an external view when the toothbrush part 11a is attached to the main body 10. As shown in Figure 3, the toothbrush part 11a is attached to the head part 10a of the main body 10. 【0035】 In Figure 3, an example of a toothbrush section 11a having a brush with multiple bristle bundles arranged in a circular pattern is shown, but toothbrush sections of other shapes may be used. Figure 4 is an external view when the toothbrush section 11b is attached to the main body section 10. As shown in Figure 4, a toothbrush section 11b having a deck brush-shaped brush may be used. 【0036】 Either the toothbrush section 11a or the toothbrush section 11b may be used, or both the toothbrush section 11a and the toothbrush section 11b may be included in multiple attachments of the toothbrush system. The toothbrush sections 11a and 11b have toothbrush bristles or bundles of bristles. 【0037】 In addition, when either of the attachments shown in Figures 3 and 4 is attached, the imaging optical system 12 and the multiple LEDs 26A to 26D are exposed, allowing for light irradiation and imaging of the oral cavity care area (e.g., teeth). Alternatively, the imaging optical system 12 and the multiple LEDs 26A to 26D may be covered with a light-transmitting material instead of being exposed. 【0038】 Note that the attachment shapes shown in Figures 3 and 4 are examples only, and attachments of any shape may be used. 【0039】 [1-3. Overall Functional Configuration] Figure 5 is a schematic diagram of the toothbrush system according to this embodiment. As shown in Figure 5, the toothbrush system according to this embodiment is generally configured to use the main body 10 to photograph the teeth and to perform image processing on the captured image. 【0040】 As shown in Figure 5, the toothbrush system includes a main unit 10, a mobile terminal 70, and a cloud server 80. The mobile terminal 70 is, for example, a wirelessly wireless smartphone or tablet. The mobile terminal 70 includes, for example, a touchscreen 72 capable of displaying dental images as an input and output device. The mobile terminal 70 functions as the user interface for the toothbrush system. 【0041】The cloud server 80 is a server that can communicate with the mobile terminal 70 via the Internet or the like, and provides an application for using the main body unit 10 to the mobile terminal 70. For example, the user downloads the application from the cloud server 80 and installs it on the mobile terminal 70. Further, the cloud server 80 acquires the dental arch image captured by the main body unit 10 via the mobile terminal 70. 【0042】 The main body unit 10 includes a central control unit 50 which is a main part for controlling the system, a plurality of LEDs 26A to 26D, an LED control unit 54, an actuator 36, an actuator 40, a lens driver 56, and a position sensor 90. 【0043】 The main body unit 10 also has a wireless communication module 58 that performs wireless communication with the mobile terminal 70, and a power supply control unit 60 that supplies power to the central control unit 50 and the like. 【0044】 The central control unit 50 of the main body unit 10 is mounted, for example, on the handle portion 10b of the main body unit 10. For example, the central control unit 50 includes a controller 62 such as a CPU (Central Processing Unit) or an MPU (Micro Processing Unit) that executes various processes described later, and a memory 64 such as a RAM (Random Access Memory) or a ROM (Read Only Memory) that stores a program for causing the controller 62 to execute various processes. Note that, in addition to the program, the memory 64 stores moving image data including the dental arch image (image data) captured by the imaging element 14 and various setting data. 【0045】 The controller 62 transmits the dental arch image output from the imaging element 14 to the mobile terminal 70 via the wireless communication module 58. The mobile terminal 70 displays the transmitted dental arch image on the touch screen 72, thereby presenting the dental arch image to the user. 【0046】The LED control unit 54 is mounted, for example, on the handle portion 10b of the main body portion 10, and performs lighting and extinguishing of the plurality of LEDs 26A to 26D based on a control signal from the controller 62. The LED control unit 54 is configured of a circuit, for example. 【0047】 The actuator 36 is mounted, for example, on the head portion 10a or the neck portion 10c of the main body portion 10, and adjusts the composition of the imaging optical system 12. Specifically, the actuator 36 moves the housing holding the image pickup device 14 and the like in the extending direction of the optical axis in order to adjust the angle of view. 【0048】 The actuator 40 is mounted, for example, on the head portion 10a or the neck portion 10c of the main body portion 10, and adjusts the focus of the imaging optical system 12. Specifically, the actuator 40 moves the lens in the extending direction of the optical axis in order to adjust the focus. 【0049】 The lens driver 56 is mounted, for example, on the handle portion 10b of the main body portion 10, and controls the actuator 36 and the actuator 40 based on a control signal from the controller 62 of the central control unit 50. The lens driver 56 is configured of a circuit, for example. 【0050】 The wireless communication module 58 is mounted, for example, on the handle portion 10b of the main body portion 10, and performs wireless communication with the portable terminal 70 based on a control signal from the controller 62. The wireless communication module 58 performs wireless communication compliant with an existing communication standard such as WiFi (registered trademark) or Bluetooth (registered trademark) with the portable terminal 70. A dental arch image is transmitted from the main body portion 10 to the portable terminal 70 via the wireless communication module 58. Also, an operation signal is transmitted from the portable terminal 70 to the main body portion 10 via the wireless communication module 58. 【0051】In this embodiment, the power control unit 60 is mounted on the handle portion 10b of the main body 10 and distributes power from the battery 66 to the central control unit 50, LED control unit 54, lens driver 56, and wireless communication module 58. The power control unit 60 is composed of, for example, a circuit. In this embodiment, the battery 66 is a rechargeable secondary battery and is wirelessly charged by an external charger 69 connected to a commercial power source via a coil 68 mounted on the main body 10. 【0052】 The position sensor 90 is a sensor for detecting the attitude and position of the main body 10, and is, for example, a multi-axis (in this case, three axes: x, y, and z) accelerometer. For example, the position sensor 90 may be a six-axis sensor having a three-axis accelerometer and a three-axis gyroscope. 【0053】 [1-4. Functional Configuration of a Mobile Terminal] Figure 6 is a functional block diagram of a mobile terminal according to an embodiment. The mobile terminal 70 includes an acquisition unit 101, a determination unit 102, a notification unit 103, and a storage unit 104. 【0054】 The acquisition unit 101 acquires video data transmitted from the main unit 10. 【0055】 The determination unit 102 determines from the still image included in the video data whether the deformation state of the toothbrush bristles or bristle bundles exceeds a predetermined range. 【0056】 The notification unit 103 notifies the user based on the determination that the deformation state of the toothbrush bristles or bristle bundles exceeds a predetermined range. Specifically, if the notification unit 103 determines that the deformation state of the toothbrush bristles or bristle bundles exceeds a predetermined range, it may notify the user by displaying a message indicating that the user is applying excessive pressure to the teeth or gums with the toothbrush, or it may not notify the user by outputting a message. If the notification unit 103 does not determine that the deformation state of the toothbrush bristles or bristle bundles exceeds a predetermined range, it may not notify the user by displaying a message indicating that the user is applying appropriate pressure to the teeth or gums with the toothbrush. 【0057】The storage unit 104 may store the processing results from the acquisition unit 101, the determination unit 102, and the notification unit 103. For example, the storage unit 104 may store video data acquired by the acquisition unit 101, or it may store the determination results from the determination unit 102. 【0058】 The functions of the acquisition unit 101, the determination unit 102, and the notification unit 103 are realized, for example, by a processor such as a CPU in the mobile terminal 70 executing a program stored in the memory of the mobile terminal 70. At least a portion of the acquisition unit 101, the determination unit 102, and the notification unit 103 may be composed of circuits. 【0059】 Here, the detailed processing by the determination unit 102 will be explained with reference to a diagram. 【0060】 Figure 7 shows an example of a still image of a toothbrush not being pressed against the teeth or gums. Figure 8 shows an example of a still image of a toothbrush being pressed against the teeth or gums with excessive pressure. Note that Figure 7 is an example of a state without brushing, and Figure 8 is an example of a state with brushing. 【0061】 As shown in Figure 7, in the still image 201A, the detection region 400 is set as an area located at a predetermined distance from the position of the bristles or bristle bundles of the toothbrush part 11a. The detection region 400 is an area used by the determination unit 102 to determine whether or not the deformation state of the toothbrush bristles or bristle bundles exceeds a predetermined range. The detection region 400 is represented by a dashed line boundary and is the area furthest from the toothbrush bristles or bristle bundles. 【0062】When a user applies excessive pressure to their teeth or gums with a toothbrush, the bristles or tufts of bristles of the toothbrush part 11a enter the detection area 400, as shown in the still image 201B of Figure 8. Therefore, the determination unit 102 detects the entry of a portion of the toothbrush bristles or tufts of bristles into the detection area 400 in the still image 201B by performing image processing. Based on the detection that a portion of the toothbrush bristles or tufts of bristles has entered the detection area 400, the determination unit 102 may determine that the deformation state of the bristles or tufts exceeds a predetermined range. In other words, if the determination unit 102 detects that a portion of the toothbrush bristles or tufts of bristles has entered the detection area 400, it may determine that the deformation state of the bristles or tufts exceeds a predetermined range, and if not, it may determine that the deformation state of the bristles or tufts does not exceed a predetermined range. 【0063】 Therefore, the determination unit 102 can detect if the user is applying excessive pressure to the teeth or gums with the toothbrush by performing image processing. In other words, even without equipping the toothbrush with a pressure sensor, it is possible to detect if the user is applying excessive pressure to the teeth or gums with the toothbrush by performing image processing on a still image taken during brushing, thus enabling the use of a more cost-effective toothbrush. 【0064】 The determination unit 102 may determine the deformation state of the hair or hair bundle based on each of the multiple still images included in the moving image data. In this case, the notification unit 103 may notify the user that the user is applying excessive pressure to the teeth or gums with the toothbrush, based on the frequency of hair or hair bundles entering the detection area 400 being above a predetermined threshold. The notification unit 103 does not need to notify the user if the frequency of hair or hair bundles entering the detection area 400 is below the predetermined threshold. 【0065】 Figure 9 is a diagram illustrating another example of the detection region. 【0066】As shown in Figure 9, two detection regions 400 and 401 may be set in the still image 201C. Detection region 400 is the same as the detection region described in Figures 7 and 8. Detection region 401 is a region with a smaller diameter than detection region 400. The determination unit 102 may determine that a higher pressure is being applied if it detects that a portion of the toothbrush bristles or bristle bundle has entered detection region 401 than if it detects that a portion of the toothbrush bristles or bristle bundle has entered only detection region 400. In this way, by setting two detection regions 400 and 401 with different diameters (sizes), the pressure applied to the teeth or gums can be detected in multiple stages, and the effect of pressure on the teeth or gums can be evaluated with greater accuracy. 【0067】 The determination unit 102 may set the detection area based on the type of toothbrush head. The type of toothbrush head may be classified by the hardness of the bristles or bristle bundles, by the length of the bristles or bristle bundles, by the arrangement shape of the bristles or bristle bundles, or by whether it is for adults or children. The detection area set based on the type of toothbrush head may be set to a size appropriate for that type, or to a shape appropriate for that type. In other words, the detection area may be set to different sizes and shapes depending on the type. For example, the harder or shorter the bristles or bristle bundles, the less likely they are to deform even when pressure is applied, so the detection boundary line of the detection area may be set at a position where the distance from the bristles or bristle bundles is short. 【0068】 Figure 10 shows another example of a still image where the toothbrush head is not pressed against the teeth or gums. Figure 11 shows another example of a still image where the toothbrush head is pressed against the teeth or gums with excessive pressure. Figures 10 and 11 show examples using the toothbrush head 11b. Note that Figure 10 is an example of a state without brushing, and Figure 11 is an example of a state with brushing. 【0069】As shown in Figure 10, in the still image 201D, the detection region 410 is set as an area located at a predetermined distance from the position of the bristles or bristle bundles of the toothbrush part 11b. The detection region 410 is the area used by the determination unit 102 to determine whether the deformation state of the toothbrush bristles or bristle bundles exceeds a predetermined range. The detection region 410 is represented by a dashed line boundary and is the area furthest from the toothbrush bristles or bristle bundles. 【0070】 When a user applies excessive pressure to their teeth or gums with a toothbrush, the bristles or tufts of bristles of the toothbrush part 11b enter the detection area 410, as shown in the still image 201E of Figure 11. Therefore, the determination unit 102 can detect the entry of a portion of the toothbrush bristles or tufts of bristles into the detection area 410 in the still image by performing image processing. 【0071】 [2. Operation] Next, the intraoral imaging operation in the toothbrush system will be explained. Figure 12 is a diagram showing the flow of the intraoral imaging operation in the toothbrush system. Note that the process shown in Figure 12 is a real-time process, and is performed each time one or more frames of image data included in the video data are obtained. 【0072】 Motion data is generated by capturing footage of the user brushing their teeth or gums in their mouth with the toothbrush of the main unit 10 (S101). In other words, the motion data is motion data of the user brushing their teeth or gums with a toothbrush. 【0073】 Next, the main unit 10 transmits the captured video data to the mobile terminal 70 (S102). As a result, the mobile terminal 70 acquires the video data from the main unit 10. 【0074】 Here, the video data is not limited to video images, but may also be still image data containing one or more still images. Furthermore, sensor data may be transmitted for each frame of the video data, or for each still image data, or for every multiple frames. 【0075】Furthermore, the transmission of video and image data may be done in real time, or it may be transmitted all at once after a series of images have been taken (for example, images of all teeth in the oral cavity). 【0076】 The mobile terminal 70 determines from the still image contained in the received video data whether the deformation of the toothbrush bristles or bristle bundles exceeds a predetermined range (S103). 【0077】 If the mobile terminal 70 determines that the deformation of the toothbrush bristles or bristle bundle exceeds a predetermined range (Yes in S103), it notifies the user (S104). 【0078】 If the mobile terminal 70 determines that the deformation state of the toothbrush bristles or bristle bundle does not exceed a predetermined range (No in S103), it may return to the process in step S103 and perform the process in step S103 for the still image following the still image that was the subject of the determination in the previous step S103. 【0079】 [3. Effects, etc.] As described above, the toothbrush system according to this embodiment comprises an acquisition unit 101, a determination unit 102, and a notification unit 103. The acquisition unit 101 acquires video data of the state in which the toothbrush (toothbrush part 11a) is being brushed on the teeth or gums. The determination unit 102 determines from the still images included in the video data whether the deformation state of the toothbrush bristles or bristle bundles exceeds a predetermined range. The notification unit 103 notifies the user based on the determination result that the deformation state exceeds a predetermined range. 【0080】 According to this, the user can be notified if the deformation of the toothbrush bristles or bristle bundles exceeds a predetermined range. For example, the user can be notified if they are applying excessive pressure to their teeth or gums with the toothbrush. Therefore, it is possible to prevent the user from applying excessive pressure to their teeth or gums with the toothbrush and to assist in brushing in a way that does not damage the teeth or gums. 【0081】For example, the determination unit 102 detects the intrusion of a portion of a toothbrush bristle or bristle bundle into a detection area in a still image, which is an area located at a predetermined distance from the position of the toothbrush bristles or bristle bundle when not brushing. Based on the detected intrusion, the determination unit 102 determines that the deformation state exceeds a predetermined range. 【0082】 Therefore, by performing image processing, it is possible to detect when a user is applying excessive pressure to their teeth or gums with a toothbrush. In other words, since it is possible to detect when a user is applying excessive pressure to their teeth or gums with a toothbrush without installing a pressure sensor in the toothbrush, a toothbrush with reduced costs can be used. 【0083】 For example, the determination unit 102 determines the deformation state based on each of the multiple still images included in the video data. The notification unit 103 makes a notification based on the fact that the frequency of detected intrusions exceeds a predetermined threshold. 【0084】 According to this, for example, it is possible to suppress notifications when it is determined that excessive pressure has been applied instantaneously. Therefore, notifications due to false detections can be suppressed. 【0085】 For example, the determination unit 102 sets the detection area 400 based on the type of toothbrush. 【0086】 Therefore, based on a detection area 400 appropriately set according to the type of toothbrush, it is possible to determine whether or not the deformation state exceeds a predetermined range. Thus, the deformation state can be appropriately determined according to the type of toothbrush. 【0087】 [4. Modified Examples] In the above embodiment, the determination unit 102 of the mobile terminal 70 sets a region a predetermined distance away from the position of the bristles or bristle bundles of the toothbrush part 11a as a detection region 400 based on a still image of the non-brushing state, and determines the deformation state by detecting the entry of bristles or bristle bundles into the set detection region 400. However, the determination unit 102 may also determine the deformation state of the bristles or bristle bundles using a learning model obtained by machine learning. 【0088】In this case, the memory unit 104 stores the learning model. The learning model is a learning model obtained by machine learning using training data that associates test images of teeth or gums being brushed with the pressure applied by the toothbrush to the teeth or gums in the above state in the test images. 【0089】 The determination unit 102 may apply the still image contained in the moving image data to a learning model stored in the storage unit 104 to identify the pressing pressure as the deformation state of the hair or hair bundle, and determine that the deformation state of the hair or hair bundle has exceeded a predetermined range based on whether the identified pressing pressure exceeds a predetermined threshold. 【0090】 Therefore, a determination process using a learning model can be used to determine whether or not the deformation state exceeds a predetermined range. 【0091】 Furthermore, the learning model may be trained using training data set for each type of toothbrush head. Therefore, based on the learning model trained using training data appropriately set according to the type of toothbrush, it is possible to determine whether the deformation state exceeds a predetermined range. Thus, the deformation state can be appropriately determined according to the type of toothbrush. 【0092】 The toothbrush system according to the embodiments of this disclosure has been described above, but this disclosure is not limited to these embodiments. 【0093】 For example, the above description mainly described the processing performed on the mobile terminal 70. At least some of these processes may be performed on the main unit 10 or the cloud server 80. 【0094】 Furthermore, each processing unit included in the toothbrush system according to the above embodiment is typically implemented as an LSI, which is an integrated circuit. These may be individually integrated into a single chip, or some or all of them may be integrated into a single chip. 【0095】Furthermore, integrated circuit implementation is not limited to LSIs; it may also be achieved using dedicated circuits or general-purpose processors. Alternatively, an FPGA (Field Programmable Gate Array), which can be programmed after LSI manufacturing, or a reconfigurable processor capable of reconfiguring the connections and settings of circuit cells within the LSI, may be used. 【0096】 Furthermore, in each of the above embodiments, each component may be implemented by being composed of dedicated hardware or by executing a software program suitable for each component. Each component may also be implemented by a program execution unit such as a CPU or processor reading and executing a software program recorded on a recording medium such as a hard disk or semiconductor memory. 【0097】 Furthermore, this disclosure may be implemented as a control method or the like performed by a toothbrush system. Alternatively, this disclosure may be implemented as a main unit, a portable terminal, or a cloud server included in a toothbrush system. 【0098】 Furthermore, the division of functional blocks in the block diagram is just one example; multiple functional blocks can be implemented as a single functional block, a single functional block can be divided into multiple parts, or some functions can be moved to other functional blocks. In addition, the functions of multiple functional blocks with similar functions can be processed in parallel or time-sharing by a single piece of hardware or software. 【0099】 Furthermore, the order in which each step in the flowchart is performed is illustrative for the purpose of specifically illustrating this disclosure, and may be in a different order. Also, some of the above steps may be performed simultaneously (in parallel) with other steps. 【0100】 Although a toothbrush system, etc., relating to one or more embodiments has been described above based on embodiments, this disclosure is not limited to these embodiments. Without departing from the spirit of this disclosure, various modifications that a person skilled in the art could conceive of may be applied to these embodiments, and forms constructed by combining components from different embodiments may also be included within the scope of one or more embodiments. 【0101】 This disclosure is applicable to toothbrush systems. 【0102】 10 Main body 10a Head 10b Handle 10c Neck 11a, 11b Toothbrush 12 Imaging optical system 14 Image sensor 26A, 26B, 26C, 26D LED 36, 40 Actuator 50 Central control unit 54 LED control unit 56 Lens driver 58 Wireless communication module 60 Power control unit 62 Controller 64 Memory 66 Battery 68 Coil 69 Charger 70 Mobile terminal 72 Touchscreen 80 Cloud server 90 Position sensor 101 Acquisition unit 102 Judgment unit 103 Notification unit 104 Storage unit 400, 401, 410 Detection area

Claims

1. A toothbrush system comprising: an acquisition unit that acquires video data of a toothbrush being used to brush teeth or gums; a determination unit that determines from still images included in the video data whether the deformation of the bristles or bristle bundles of the toothbrush exceeds a predetermined range; and a notification unit that notifies the user based on the determination result that the deformation exceeds the predetermined range.

2. The toothbrush system according to claim 1, wherein the determination unit detects the entry of a part of the bristles or bristle bundle into a detection area in the still image, which is a detection area located at a predetermined distance from the position of the bristles or bristle bundle of the toothbrush when it is not being brushed, and determines, based on the entry, that the deformation state exceeds the predetermined range.

3. The toothbrush system according to claim 2, wherein the determination unit determines the deformation state based on each of the plurality of still images included in the motion image data, and the notification unit makes the notification based on the fact that the frequency of entry has exceeded a predetermined threshold.

4. The toothbrush system according to claim 2 or 3, wherein the determination unit sets the detection area based on the type of toothbrush.

5. The toothbrush system according to claim 1, further comprising a storage unit that stores a learning model obtained by machine learning using training data that associates a test image of the state in which the tooth or gum is being brushed with the toothbrush with the pressure applied by the toothbrush to the tooth or gum in the state of the test image, wherein the determination unit applies the still image included in the moving image data to the learning model to identify the pressure as the deformation state, and determines that the deformation state has exceeded the predetermined range based on the identified pressure exceeding a predetermined threshold.

6. The toothbrush system according to claim 5, wherein the learning model is machine-trained using training data set for each type of toothbrush.

7. A notification method performed by a toothbrush system, comprising: acquiring video data of a state in which a tooth or gum is being brushed with a toothbrush; determining from still images included in the video data whether the deformation state of the bristles or bundle of bristles of the toothbrush exceeds a predetermined range; and notifying the user based on the determination that the deformation state exceeds the predetermined range.

8. A program for causing a computer to execute the notification method described in claim 7.

Images