Toothbrush state determination method, toothbrush state determination device, computer program product, toothbrush state determination system, and computer-readable recording medium

Abstract

The invention provides a toothbrush state determination method, a toothbrush state determination device, a computer program product, a toothbrush state determination system, and a computer-readable recording medium. A method executed by a computer is disclosed. The method comprises the following steps: acquiring an image generated by shooting a brush head part of a toothbrush by a camera device, wherein the brush head part comprises a bristle bundle and a bristle planting seat for supporting the bristle bundle; identifying, from the image, at least one of a bristle bundle region corresponding to the bristle bundle and a bristle base region corresponding to the bristle base; calculating an index value indicating the degree of inclination of the toothbrush with respect to a reference orientation at the time of imaging on the basis of the size of at least one of the bristle bundle region and the bristle holder region in the image; and changing the content of the notification to the user in accordance with whether or not the index value deviates from a predetermined reference range.

Description

Technical Field

[0001] The present disclosure relates to a method, device, system and program for determining a state of a toothbrush. Background Art

[0002] Patent document 1 discloses a toothbrush timely replacement system that can achieve timely replacement of toothbrushes. The system determines the degree of deformation of the toothbrush bristles based on an image taken from the direction of the bristle tips of the toothbrush, and issues an alarm when the degree of deformation of the bristle bundle exceeds a preset limit. This can urge the user to replace the toothbrush in a timely manner.

[0003] Prior art literature

[0004] Patent Literature

[0005] Patent Document 1: Japanese Patent Application Publication No. 2019-080752 Summary of the invention

[0006] Problem that the invention aims to solve

[0007] The present disclosure provides a determination method capable of more accurately determining a state such as the degree of deterioration of a toothbrush, and a method for assisting such determination.

[0008] Solutions for solving problems

[0009] A method in one embodiment of the present invention is a method executed by a computer, the method comprising the following steps: acquiring an image generated by photographing a brush head of a toothbrush with a camera device, the brush head comprising bristles and a bristle seat supporting the bristles; identifying at least one of a bristle bundle area corresponding to the bristle bundle and a bristle seat area corresponding to the bristle seat from the image; calculating an index value representing the degree of inclination of the toothbrush relative to a reference posture at the time of photographing based on the size of at least one of the bristle bundle area and the bristle seat area in the image; and changing the content of a notification to a user according to whether the index value deviates from a specified reference range.

[0010] A system in another aspect of the present disclosure comprises: a server computer; and a terminal device connected to the server computer via a network. The terminal device acquires an image generated by photographing a brush head of a toothbrush by a camera device, and sends the image to the server computer, wherein the brush head includes a bristle bundle and a bristle implantation seat supporting the bristle bundle. The server computer performs the following processing: identifying at least one of a bristle bundle area corresponding to the bristle bundle and a bristle implantation seat area corresponding to the bristle implantation seat from the image; calculating an index value indicating the degree of inclination of the toothbrush relative to a reference posture at the time of photographing based on the size of at least one of the bristle bundle area and the bristle implantation seat area in the image; and sending information related to the index value to the terminal device. The terminal device changes the content of the notification to the user according to whether the index value deviates from a specified reference range.

[0011] The general or specific manner of the present disclosure can be implemented by a device, a system, a method, an integrated circuit, a computer program or a computer-readable recording medium, or any combination thereof. The computer-readable recording medium can be either a volatile recording medium or a non-volatile recording medium. The recording medium can be any storage medium such as a semiconductor recording medium, a magnetic recording medium or an optical recording medium. The device is not limited to a single device, but can also be a system composed of more than two devices. In the case where the device is composed of more than two devices, the two or more devices can be configured in one device or separately configured in two or more separate devices.

[0012] Effects of the Invention

[0013] According to one aspect of the present disclosure, it is possible to more accurately determine a state such as the degree of deterioration of a toothbrush. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 This is a diagram illustrating a terminal device and a toothbrush.

[0015] Figure 2 This is a block diagram showing a configuration example of a terminal device.

[0016] Figure 3 This is a diagram schematically showing a structure in which a terminal device communicates with an external device via a network.

[0017] Figure 4 is a flowchart illustrating an example of actions performed by the processing circuit.

[0018] Figure 5 FIG. 1 is a diagram showing an example of a shooting instruction screen.

[0019] Figure 61 is a diagram showing an example of a reference posture of a toothbrush.

[0020] Figure 7 This is a diagram for explaining a method of estimating the inclination of a toothbrush with respect to a reference posture based on the sizes of the bristle base and bristle bundles of the toothbrush.

[0021] Figure 8 This is a diagram showing an example of an image of a brush head of a toothbrush.

[0022] Fig. 9 It is a diagram showing an example of a reference range of an index value.

[0023] Fig.10 : is a graph showing an example of the relationship between the index value β / y and the angle of rotation around the X-axis.

[0024] Fig.11A This is a diagram showing an example of a method of estimating the inclination of a toothbrush based on the contour of a bristle bundle region.

[0025] Fig. 11B This is a diagram showing an example of a method of estimating the inclination of a toothbrush based on the contour of a bristle bundle region.

[0026] Fig. 11C This is a diagram showing an example of a method of estimating the inclination of a toothbrush based on the contour of a bristle bundle region.

[0027] Fig.12 This is a diagram showing an example of display of the determination result displayed on the display of the terminal device.

[0028] Fig.13 It is shown Figure 4 Flowchart of a specific example of the process of step S180 shown.

[0029] Fig.14A 1 is a diagram showing an example of a hair bundle region recognized from an image.

[0030] Fig. 14B This is a diagram showing an example in which the hair bundle region is divided into a plurality of partial regions.

[0031] Fig. 14C This is a diagram showing another example in which the hair bundle region is divided into a plurality of partial regions.

[0032] Fig.14D This is a diagram showing another example in which the hair bundle region is divided into a plurality of partial regions.

[0033] Fig.15A 1 is a diagram showing an example of an image in which a hair bundle region having a deformed shape is extracted.

[0034] Fig. 15BFIG. 1 is a diagram showing an example of region segmentation in an image in which a hair bundle region with a deformed shape is extracted.

[0035] Fig.16A This is a diagram for explaining a method of correcting hair length based on inclination.

[0036] Fig. 16B This is a diagram for explaining a method of correcting hair length based on inclination.

[0037] Fig. 16C This is a diagram for explaining a method of correcting hair length based on inclination.

[0038] Fig.17 : is a graph showing examples of the hair length before correction and the hair length after correction.

[0039] Fig.18 It is shown Figure 4 A flowchart of another specific example of the process of step S180 is shown.

[0040] Fig.19 It is a diagram showing an example of a table indicating the correspondence between determination results and display contents.

[0041] Fig. 20 1 is a diagram showing an example of a photographing instruction screen for acquiring an image capable of determining the unraveling state of hair bundles.

[0042] Fig.21 This is a diagram for explaining a method of estimating the inclination of the toothbrush around the Z axis with respect to the reference posture.

[0043] Fig. 22 1 is a flowchart showing an example of a deterioration determination process performed based on the spreading state of the hair bundle in addition to the hair length.

[0044] Fig.23 It is shown Fig. 22 Flowchart of a specific example of the process of step S810 shown.

[0045] Fig.24A It is a diagram for explaining the side portion change amounts c1 and c2.

[0046] Fig. 24B It is a diagram for explaining the side portion change amounts c1 and c2.

[0047] Fig.25 1 is a diagram showing an example of a table storing the values ​​of threshold values ​​Th1 to Th4. DETAILED DESCRIPTION

[0048] The following is a detailed description of the embodiments with reference to the accompanying drawings. However, unnecessary detailed descriptions are sometimes omitted. For example, detailed descriptions of well-known matters or repeated descriptions of substantially the same structures are sometimes omitted. In addition, the attached drawings and the following descriptions are provided to enable those skilled in the art to fully understand the technology of the present disclosure, and are not intended to limit the subject matter described in the claims.

[0049] (structure)

[0050] Figure 1 It is a diagram illustrating the terminal device 10 and the toothbrush 20 in this embodiment. Figure 2 1 is a block diagram showing a configuration example of the terminal device 10. The terminal device 10 in this embodiment is a mobile terminal such as a smart phone or a tablet computer. Figure 2 As shown, the terminal device 10 includes a state determination system 100, a camera 210, and a display 220. The camera 210 is an example of an imaging device that acquires an image of an object. The display 220 includes a touch screen and has the functions of an input device and a display device. The state determination system 100 is a computer system that determines the state of the toothbrush 20 (e.g., the degree of deterioration of the bristle bundle 22) based on the image of the toothbrush 20 captured by the camera 210.

[0051] The toothbrush 20 may deteriorate due to wear of the bristle tips of the bristle bundle 22 or deformation of the side of the bristle bundle 22 during use. The state determination system 100 has, for example, a function of estimating the degree of degradation due to wear of the bristle tips or deformation of the bristle bundle 22 and determining whether the toothbrush 20 is in a state suitable for use. In addition, the state determination system 100 also has a function of determining whether the posture of the toothbrush 20 at the time of shooting is in a posture suitable for determination. The state determination system 100 may also have a function of determining the type of the toothbrush 20 based on the image.

[0052] like Figure 1 As shown, the toothbrush 20 includes a handle 25 extending in one direction and a brush head 26 located at the front end of the handle 25. The brush head 26 includes a bristle bundle 22 and a bristle implantation seat 21 for supporting the bristle bundle 22.

[0053] exist Figure 1 , a three-dimensional orthogonal coordinate system defined by mutually orthogonal X-axis, Y-axis, and Z-axis is illustrated. The X-axis is set along the direction in which the handle 25 extends, the Y-axis is set along the direction in which the hair bundle 22 extends, and the Z-axis is set to be a direction orthogonal to the X-axis and the Y-axis. In this specification, the positive direction or negative direction of the X-axis is sometimes referred to as the "first direction", the positive direction or negative direction of the Z-axis is sometimes referred to as the "second direction", and the positive direction or negative direction of the Y-axis is sometimes referred to as the "third direction".

[0054] The hair implantation seat 21 has a hair implantation surface 21a extending in a first direction (i.e., a direction parallel to the X-axis) and a second direction (i.e., a direction parallel to the Z-axis), the first direction being along the direction in which the handle portion 25 extends, and the second direction being perpendicular to the first direction. The hair bundles 22 extend from the hair implantation surface 21a along a third direction (i.e., a direction parallel to the Y-axis), and the third direction is perpendicular to both the first direction and the second direction.

[0055] A plurality of holes are provided on the hair-implanting surface 21a, and a plurality of hairs are implanted in each hole. The number of holes, the arrangement of the holes, and the number of hairs implanted in each hole vary depending on the type of toothbrush 20. The aggregate of hairs implanted in the hair-implanting surface 21a is a hair bundle 22. The cluster of hairs in the hair bundle 22 that is implanted in each hole is called a "partial hair bundle". That is, the hair bundle 22 is an aggregate of a plurality of partial hair bundles.

[0056] The toothbrush 20 may be a non-electric ordinary toothbrush or an electric toothbrush. When the toothbrush 20 is a non-electric toothbrush, the user holds the handle 25 and slides the brush head 26 in the X direction or the Z direction to brush the teeth. When the toothbrush 20 is an electric toothbrush, the brush head 26 vibrates by the operation of the built-in electric motor, and the teeth are brushed by the vibration.

[0057] like Figure 2 As shown, the state determination system 100 includes a processing circuit 110, a storage device 120, a communication interface (I / F) 130, and an input / output interface (I / F) 140. The processing circuit 110, the storage device 120, the communication I / F 130, and the input / output I / F 140 are connected to each other via a bus.

[0058] The processing circuit 110 can be an integrated circuit including one or more processors such as a central processing unit (CPU). The processing circuit 110 can also be implemented by an FPGA (Field Programmable Gate Array) equipped with a CPU, a GPU (Graphics Processing Unit), an ASIC (Application Specific Integrated Circuit), an ASSP (Application Specific Standard Product), or a combination of two or more circuits selected from these circuits.

[0059] The storage device 120 is, for example, a device including one or more recording media such as a semiconductor storage medium, a magnetic recording medium, or an optical recording medium. The storage device 120 can include one or more memories such as a ROM (Read Only Memory) or a RAM (Random Access Memory). The storage device 120 stores a computer program executed by a processor of the processing circuit 110, data referenced by the processing circuit 110 during processing, and various data generated by the processing circuit 110. The processing circuit 110 can perform various actions described below by executing the computer program stored in the storage device 120. The storage device 120 can be either a single storage device (storage) or a collection of multiple storage devices.

[0060] The communication I / F 130 is an interface including a circuit for wireless communication. The state determination system 100 can communicate with external devices via the communication I / F 130. For example, the state determination system 100 can communicate with external devices such as a cloud server via the communication I / F 130.

[0061] Figure 3 The structure in which the terminal device 10 and the external device 40 communicate via the network 30 such as the Internet is schematically shown. Figure 3 The external device 40 shown is a server computer on the cloud. The terminal device 10 can update the program executed by the processing circuit 110 or update information such as a threshold used in a determination process described later by communicating with the external device 40.

[0062] The input / output I / F 140 is an interface for sending and receiving signals between the state determination system 100 and devices such as the camera 210 and the display 220. The processing circuit 110 can obtain the image generated by the camera 210 via the input / output I / F 140. In addition, in this specification, "image" refers to data or signals of a digital image. The processing circuit 110 can also cause the display 220 to display information via the input / output I / F 140. The display 220 in this embodiment has a touch screen and functions as a display device and an input device. The processing circuit 110 receives a signal generated by a touch operation performed on the display 220 via the input / output I / F 140, and controls the display of the display 220 based on the signal.

[0063] (action)

[0064] Figure 4 is a flowchart showing an example of the operation performed by the processing circuit 110. The processing circuit 110 performs Figure 4The operation of steps S110 to S190 shown in the figure is used to determine the state of the toothbrush 20, such as the degree of deterioration. Next, the operation of each step will be described.

[0065] In step S110, the processing circuit 110 causes the display 220 to display a photographing instruction screen. The photographing instruction screen is a screen that displays information for urging the user to photograph the brush head 26 of the toothbrush 20 from the side.

[0066] Figure 5 FIG. 1 is a diagram showing an example of a shooting instruction screen. Figure 5 The screen shown is displayed after the application installed in the terminal device 10 is started. This screen is a guidance screen for urging the user to shoot the brush head 26 of the toothbrush 20 from the side in the negative direction of the Z axis (ie, the first direction). Figure 5 The shooting instruction screen shown includes a shooting button 224 and a window 222 displaying a predetermined preview image captured by the camera 210. A frame, i.e., a guide line 223, is displayed in the window 222 to assist the user in shooting. The user, in accordance with the displayed instructions, directs the camera 210 toward the brush head 26 so that the brush head 26 of the toothbrush 20 enters the guide line 223. In addition, the processing circuit 110 may also output information urging the user to shoot the brush head 26 of the toothbrush 20 from the side in a manner different from the display (e.g., sound). When the user presses the shooting button 224, the shooting process is executed, and the image data is recorded in the storage device 120. Before the shooting button 224 is pressed, the processing circuit 110 executes the processing of steps S120 to S150.

[0067] In step S120, the processing circuit 110 identifies the area corresponding to the brush head 26 of the toothbrush 20 from the image frame of the preview image (hereinafter referred to as the "preview image" or simply "image"). Specifically, the processing circuit 110 identifies at least one of the bristle bundle area corresponding to the bristle bundle 22 of the brush head 26 and the bristle implant seat area corresponding to the bristle implant seat 21 from the preview image. In the identification of at least one of the bristle bundle area and the bristle implant seat area, for example, an algorithm based on machine learning such as segmentation can be used. Through segmentation, each pixel in the image can be classified into categories such as "hair bundle", "hair implant seat", and "background". In this recognition process, the processing circuit 110 can distinguishably detect the bristle bundle area, the bristle implant seat area, and the background, etc. by inputting the preview image (or an image obtained by performing a predetermined preprocessing on the preview image) to a learning model that has been pre-trained and stored in the storage device 120.

[0068] The processing circuit 110 may also be configured to not only extract the hair bundle area and the hair transplantation seat area in the image, but also determine the type of the brush portion based on the image. If the types of the brush portion are different, the hair transplantation seat 21 and the hair bundle 22 may have different shapes, heights, or widths. The processing circuit 110 can determine the type of the brush portion by inputting the image into a learned model for determining the type of the brush portion based on the image of the brush head 26. In addition, the type of the brush portion may be pre-registered by the user instead of the processing circuit 110 determining the type of the brush portion based on the image.

[0069] In step S130, the processing circuit 110 determines whether the posture of the toothbrush 20 is appropriate based on the recognition result of at least one of the bristle bundle area and the bristle implantation seat area in the image. For example, the processing circuit 110 can estimate the inclination of the toothbrush 20 relative to the reference posture based on the contour or size of at least one of the bristle bundle area and the bristle implantation seat area in the image. It is possible to determine whether the posture of the toothbrush 20 is appropriate based on the estimated inclination.

[0070] Figure 6 An example of a reference posture of the toothbrush 20 is shown. Figure 6 As shown, the reference posture is the posture of the toothbrush 20 when the camera 210 is facing the negative direction of the Z axis to shoot the brush head 26. In this embodiment, the posture of the toothbrush 20 when the horizontal direction of the image (including the preview image) generated by the camera 210 is consistent with the X direction and the vertical direction of the image is consistent with the Y direction is set as the reference posture. In this embodiment, the inclination of the toothbrush 20 relative to the reference posture corresponding to the rotation angle around the X direction is estimated. In this embodiment, as Figure 6 As shown in FIG. 1 , the direction of rotation when the bristle tips of the toothbrush 20 are tilted toward the front around the X-axis is defined as positive.

[0071] In step S130, the processing circuit 110 calculates an index value representing the inclination of the toothbrush 20 relative to the reference posture based on the contour or size of at least one of the hair bundle area and the hair implant seat area in the image. When the index value is within the specified reference range, the processing circuit 110 determines that the posture of the toothbrush 20 is appropriate and proceeds to step S150. On the contrary, when the index value deviates from the reference range, the processing circuit 110 determines that the posture of the toothbrush 20 is inappropriate and proceeds to step S140.

[0072] The index value may be the rotation angle around the X-axis itself, or another value that changes according to the rotation angle. For example, the index value may also be calculated based on the size of the hair implant seat area in the image in the Y direction (third direction). In addition, in this specification, the size of the area or object in the image in the Y direction is referred to as the "height" of the area or object.

[0073] Figure 72 is a diagram for explaining a method of estimating the inclination of the toothbrush 20 relative to the reference posture based on the sizes of the bristle base 21 and the bristle bundle 22 of the toothbrush 20 in the image. Figure 7 As shown, the width of the hair transplantation seat area 323 corresponding to the hair transplantation seat 21 in the image (i.e., the dimension in the X direction) is set to x, and the height of the hair transplantation seat area 323 (i.e., the dimension in the Y direction) is set to y. In addition, the height of the hair bundle area 324 corresponding to the hair bundle 22 in the image (i.e., the dimension in the Y direction) is set to α, and the width of the hair bundle area 324 (i.e., the dimension in the X direction) is set to β. Here, the dimensions x, y, α, and β can be expressed, for example, in units of the number of pixels in the image.

[0074] The height y of the hair-implanting seat area 323 and the height α of the hair-bundle area 324 in the image change according to the size of the rotation angle of the toothbrush 20 around the X-axis. On the other hand, the width x of the hair-implanting seat area 323 and the width β of the hair-bundle area 324 hardly change due to the rotation of the toothbrush 20 around the X-axis. Therefore, the processing circuit 110 can evaluate the inclination of the toothbrush 20 by using y, α, the ratio of y to β, the ratio of y to x, the ratio of α to β, or the ratio of α to x as an index value.

[0075] The processing circuit 110 can obtain the dimensions x and y of the hair implant seat area 323 and the dimensions α and β of the hair bundle area 324 in the image based on the image captured by the camera 210 (a preview image in this embodiment). For example, the processing circuit 110 first rotates the acquired image so that the long side direction of the hair implant seat, i.e., the X direction, is consistent with the horizontal direction of the image. On this basis, the processing circuit 110 sets the height of the identified hair implant seat area 323 to y, the width of the hair implant seat area 323 to x, the height of the hair bundle area 324 to α, and the width of the hair bundle area 324 to β.

[0076] Figure 8 An example of an image of the brush head 26 of the toothbrush 20 is shown. Figure 8 The image shown shows an example of a case where the rotation angle of the toothbrush 20 about the X-axis is approximately -10°. In this example, the hair implant seat area 323 is thicker as it approaches the central portion and becomes thinner as it approaches the end. The processing circuit 110 can, for example, set the thickness of the thickest central portion of the hair implant seat area 323, or the average value of the thickness of each portion of the hair implant seat area 323 to y for processing. Similarly, the processing circuit 110 can set the height of the central portion of the hair bundle area 324, or the average value of the height of each portion of the hair bundle area 324 to α for processing. Regarding the width x of the hair implant seat area 323 and the width β of the hair bundle area 324, the dimensions of the identified hair implant seat area 323 and the hair bundle area 324 in the X direction can be directly used.

[0077] The processing circuit 110 may also calculate, for example, the ratio β / y of the width β of the hair bundle area 324 to the height y of the hair implant seat area 323, or the ratio x / y of the width x of the hair implant seat area 323 to the height y of the hair implant seat area 323, as an index value. In this case, the processing circuit 110 may be configured to output a warning indicating that the posture of the toothbrush 20 is inappropriate when the ratio β / y or x / y deviates from a predetermined reference range. In this way, the index value can be calculated based on the ratio of the size of the hair bundle area 324 or the hair implant seat area 323 in the x direction (i.e., the first direction) to the size of the hair implant seat area 323 in the y direction (i.e., the third direction) in the image.

[0078] Fig. 9 is a diagram showing an example of a reference range of an index value. Fig. 9 In the example shown, the reference range of the index value β / y is predetermined for each type of brush and stored in the storage device 120. For example, when the type of brush is "Type 1" (normal), a range exceeding 4.4 is set as the reference range. When the type of brush is "Type 2" (ultra-fine), a range exceeding 4.2 is set as the reference range. When the type of brush is "Type 3" (long hair), a range exceeding 4.6 is set. Fig. 9 In the example shown in FIG. 1 , only the lower limit of the reference range is set, but the upper limit can also be set. Fig. 9 When only the lower limit value of the reference range is defined as in the example of , the lower limit value is regarded as a threshold value, and when the index value β / y exceeds the threshold value, it is determined that the posture of the toothbrush 20 is appropriate.

[0079] Fig.10 : is a graph showing an example of the relationship between the index value β / y and the angle of rotation around the X-axis. Fig.10 The diagram shown corresponds to Fig. 9 The "type 3" brush portion shown in the figure. As shown in the graph, there is a correlation between the index value β / y and the angle of rotation around the X-axis. In this example, the processing circuit 110 reads the reference range (or threshold) corresponding to the type of brush portion photographed from the storage device 120, and compares the index value β / y calculated based on the image with the threshold value 4.6. When the index value β / y is greater than the threshold value 4.6, the processing circuit 110 determines that the posture of the toothbrush 20 is appropriate. On the contrary, when the index value β / y is less than the threshold value 4.6, the processing circuit 110 determines that the posture of the toothbrush 20 is inappropriate.

[0080] The index value is not limited to β / y, and may be, for example, x / y, β / α, x / α, or their reciprocals. Figure 5As shown, the brush head 26 is photographed by entering the guide line 223 according to the displayed instructions. Therefore, the change in the distance between the camera 210 and the brush head 26 is small, and the change in the value of y or α in the image caused by the change in distance is also small. Therefore, y or α itself can also be used as an index value. Alternatively, the value obtained by multiplying the value obtained by dividing the actual size of the hair implant seat 21 in the X direction by the size x of the hair implant seat area 323 in the X direction in the image by y or α can be used as an index value. Similarly, the value obtained by dividing the actual size of the hair bundle 22 in the X direction by the size β of the hair bundle area 324 in the X direction in the image by y or α can also be used as an index value. Through such processing, the index value corresponding to the rotation angle around the X axis can be appropriately determined regardless of the distance between the camera 210 and the brush head 26. Here, the actual size of the hair implant seat 21 or the hair bundle 22 in the X direction of each toothbrush 20 is known, and its value can be pre-stored in the storage device 120. Regardless of which index value is used, a reference range (or threshold) of the index value can be recorded in advance in the storage device 120, and the tilt can be evaluated based on the comparison between the reference range and the index value. The reference range of the index value can be set to a range narrower than ±10° when converted into, for example, the rotation angle of the toothbrush 20 around the X-axis.

[0081] In the above example, the inclination of the toothbrush 20 relative to the reference posture is estimated based on the size of at least one of the bristle region 324 and the bristle seat region 323 in the image, but other methods may be used. For example, the inclination of the toothbrush 20 relative to the reference posture may also be estimated based on the contour of at least one of the bristle region 324 and the bristle seat region 323 in the image.

[0082] FIG. 11A to FIG. 11C This is a diagram showing an example of a method of estimating the inclination of the toothbrush 20 based on the contour of the bristle bundle region 324 . Fig.11A An example of an image of the brush head 26 when the rotation angle about the X-axis is approximately -10° is shown. Fig. 11B An example of an image of the brush head 26 when the rotation angle about the X-axis is approximately 0° is shown. Fig. 11C An example of an image of the brush head 26 when the rotation angle around the X-axis is about 10° is shown. In these figures, the recognized hair bundle area 324 is shown by a dotted line. In addition, a line segment connecting two points P1 and P2 at both ends of the boundary 322 between the hair bundle area 324 and the hair implantation seat area 323 is shown by a short dashed line.

[0083] In this example, the processing circuit 110 identifies the boundary 322 between the bristle bundle area 324 and the bristle implant seat area 323 from the image. The processing circuit 110 can estimate the tilt of the toothbrush 20 around the X-axis direction based on the shape of the boundary 322 between the bristle bundle area 324 and the bristle implant seat area 323, or the positions of at least three points on the boundary 322. For example, the processing circuit 110 can determine the tilt based on an index value calculated based on the positions of at least three points on the boundary 322, and information on the correspondence between a specified index value and the tilt pre-stored in a storage device. The at least three points on the boundary 322 can, for example, include a first point P1 and a second point P2 at both ends of the boundary 322, and a third point P3 located in the center of the boundary 322. The index value can be a value based on the distance d between the third point P3 and the line segment connecting the first point P1 and the second point P2. The distance d is Fig.11A and Fig. 11C The distance between the tops of the two white arrows and the black arrows facing each other shown in FIG. The ratio of the distance d to the distance between the first point P1 and the second point P2 can be used as an index value. Fig.11A In the example of , the third point P3 is located at the top of the boundary 322 having an upwardly convex shape. Fig. 11C In the example of FIG. 3 , the third point P3 is located at the bottom of the boundary 322 having a downwardly convex shape. The third point P3 may be a point on the boundary 322 that is farthest from the line segment connecting the first point P1 and the second point P2.

[0084] The distance d between point P3 and the line segment connecting point P1 and point P2 increases as the rotation angle of the toothbrush 20 around the X-axis increases. Therefore, by using the index value corresponding to the distance d, the inclination of the toothbrush 20 relative to the reference posture can be evaluated in the same way as in the above example. For example, the processing circuit 110 can determine that the posture of the toothbrush 20 is inappropriate when the index value proportional to the distance d is greater than a predetermined threshold value, and determine that the posture of the toothbrush 20 is appropriate when the index value proportional to the distance d is less than the threshold value.

[0085] In addition, the processing circuit 110 may identify the boundary between the bristle bundle area 324 and the background, or the boundary between the bristle implantation seat area 323 and the background instead of identifying the boundary between the bristle bundle area 324 and the bristle implantation seat area 323, and estimate the inclination of the toothbrush 20 based on the shape of the boundary or the positions of at least three points on the boundary. As for the boundary between the bristle bundle area 324 and the background, and the boundary between the bristle implantation seat area 323 and the background, since the shapes also change according to the inclination of the toothbrush 20, the same processing as described above can be applied.

[0086] The process of determining whether the posture of the toothbrush 20 is appropriate is not limited to the above example, and the process may be performed based on a learned model pre-trained by machine learning such as a deep neural network (DNN). As described above, the size and shape of the bristle bundle area 324 and the bristle seat area 323 in the image change according to the inclination of the toothbrush 20 relative to the reference posture. Therefore, it is possible to estimate the inclination of the toothbrush 20 relative to the reference posture, or estimate whether the posture of the toothbrush 20 is appropriate, based on, for example, an image of the toothbrush 20 to estimate the inclination of the toothbrush 20 relative to the reference posture, or estimate whether the posture of the toothbrush 20 is appropriate, based on a learned model that estimates the inclination of the toothbrush 20 relative to the reference posture, or whether the posture of the toothbrush 20 is appropriate.

[0087] If in Figure 4 If it is determined in step S130 shown in FIG. 1 that the posture of the toothbrush 20 is inappropriate, the process proceeds to step S140. In step S140, the processing circuit 110 causes the display 220 to display a warning indicating that the posture of the toothbrush 20 is inappropriate. For example, the processing circuit 110 Figure 5 The color of the guide line 223 shown is changed to a color indicating a warning, such as red, and the shooting button 224 is turned into an inactive state. Thus, the user can be notified that the posture of the toothbrush 20 is inappropriate. At this time, the processing circuit 110 can also display information such as a message indicating that the posture of the toothbrush 20 is inappropriate as a warning.

[0088] The processing circuit 110 is not limited to display, and may also output warnings by other methods. The output of the warning may include the following situations: controlling at least one of a display device (such as the display 220), a light emitting device, a speaker, and a tactile device to apply at least one of visual stimulation, auditory stimulation, and tactile stimulation to the user to indicate a warning. At least one of such a display device, a light emitting device, a speaker, and a tactile device may be built into the terminal device 10 or may be a device external to the terminal device 10. By applying a warning based on at least one of visual stimulation, auditory stimulation, and tactile stimulation to the user, the user can be effectively notified that the posture of the toothbrush 20 during shooting is inappropriate.

[0089] When it is determined in step S130 that the posture of the toothbrush 20 is appropriate, the process proceeds to step S150. In step S150, the processing circuit 110 enables the shooting button 224 so that the user can perform the shooting operation. At this time, the processing circuit 110 changes the color of the guide line 223 to a normal color such as green. Thus, the user can recognize that the posture of the toothbrush 20 is appropriate and click the shooting button 224.

[0090] As described above, the processing circuit 110 in the present embodiment changes the content of the notification to the user according to whether the index value indicating the degree of inclination of the toothbrush 20 relative to the reference posture during shooting deviates from the prescribed reference range. The processing circuit 110 issues a notification indicating a warning when the index value deviates from the reference range, and does not issue a notification indicating a warning when the index value is within the reference range, or issues a notification indicating suitability. The notification indicating a warning can be provided by, for example, changing the color of the guide line 223 to a color indicating a warning, such as red, or displaying a warning message, or emitting a sound indicating a warning. The notification indicating suitability can be provided by, for example, changing the color of the guide line 223 to a color indicating a suitability, such as green or yellow, or displaying a message indicating that the posture is appropriate, or emitting a sound indicating that the posture is appropriate. Thus, the user can understand whether the posture of the toothbrush 20 during shooting is appropriate.

[0091] In step S160, the processing circuit 110 determines whether the user has performed a shooting operation. That is, the processing circuit 110 determines whether the user has clicked the shooting button 224. If the shooting operation has been performed, the process proceeds to step S170. If the shooting operation has not been performed, the process returns to step S120. The processes of steps S120 to S160 are repeated until the user has performed a shooting operation.

[0092] In step S170, the processing circuit 110 sends a command to generate an image (i.e., a still image) to the camera 210 in response to the user's shooting operation. The camera 210 drives the image sensor in response to the command, thereby generating an image of the brush head 26 of the toothbrush 20. The processing circuit 110 stores the image generated by the camera 210 in the storage device 120. Thus, an image of the toothbrush 20 in a posture that is not significantly tilted relative to the reference posture is acquired.

[0093] In addition, in this embodiment, the image is acquired in response to the user's shooting operation, but the shooting may also be performed automatically. For example, after the processing circuit 110 determines that the posture of the toothbrush is appropriate (yes) in step S130, it may send a command to generate an image to the camera 210 without going through the processing of steps S150 and S160.

[0094] After step S170, the process proceeds to step S180. In step S180, the processing circuit 110 determines the degradation state of the bristle bundle 22 of the toothbrush 20 based on the acquired image. For example, the processing circuit 110 estimates the bristle length of the toothbrush 20 based on the image, and determines the degree of degradation of the toothbrush 20 based on the estimated bristle length. In this embodiment, if the degradation state can be determined, the process proceeds to step S190, and if the degradation state cannot be determined, the process returns to step S120. The details of the process of step S180 will be described later.

[0095] In step S190, the processing circuit 110 outputs the determination result. The determination result includes information related to the degradation of the toothbrush 20. For example, the processing circuit 110 causes the display 220 to display information such as to what extent the degradation of the toothbrush 20 has progressed or whether the toothbrush 20 needs to be replaced. The output of the determination result is not limited to display, and the user may be notified by light, sound, vibration, or other methods.

[0096] Fig.12 2 is a diagram showing a display example of the determination result displayed on the display 220 of the terminal device 10. Fig.12 In the example shown, a message such as "The brush part has deteriorated. Replacement is recommended." is displayed as the determination result. In this way, the processing circuit 110 causes the display 220 to display information related to the deterioration of the toothbrush 20 (for example, information indicating the degree of deterioration). The user can recognize the state of the toothbrush 20 based on the displayed information. Fig.12 In the example of FIG. 1 , a sentence "ultra-fine bristle large size" indicating the type of the brush unit is also displayed. In addition to the name of the type, information such as the product number or a link to the website of the brush unit can also be displayed. By displaying information indicating the type of toothbrush 20, the user can clearly understand the type of toothbrush 20 he is using. As a result, the user can replace the toothbrush 20 with the same toothbrush 20 as the one used before without error.

[0097] Next, Figure 4 A specific example of the processing of step S180 shown in FIG.

[0098] Fig.13 is a flowchart showing a specific example of the processing of step S180. In this example, the processing circuit 110 executes Fig.13 The processing shown in the figure estimates the length of the bristles of the toothbrush 20 and determines the deterioration state of the toothbrush 20 based on the bristle length. In addition, the processing circuit 110 may determine the deterioration state of the toothbrush 20 by considering not only the bristle length of the toothbrush 20 but also the spreading state of the bristle bundles 22. Such an example will be described later.

[0099] In step S181, the processing circuit 110 identifies the hair bundle area from the image acquired in step S170 and divides the hair bundle area into a plurality of partial areas. For example, the identification can be performed using a segmentation algorithm such as semantic segmentation. The processing circuit 110 performs segmentation on the image, thereby dividing the hair bundle area into a plurality of partial areas. FIG. 11A to FIG. 11C. However, depending on the situation, the segmentation is sometimes not performed accurately, so that the hair bundle area is not correctly extracted. Therefore, in the present embodiment, the processing circuit 110 segments the identified hair bundle area into a plurality of partial areas arranged in a direction (e.g., X direction) intersecting the direction in which the hair bundle extends (i.e., Z direction), calculates the length of the hair bundle for each partial area, and estimates the hair length more accurately based on these lengths. Here, the "length of the hair bundle" refers to the size of the area occupied by the hair bundle in the direction in which the hair bundle extends in the image. Sometimes, the "length of the hair bundle" is also referred to as the "height of the hair bundle."

[0100] Fig.14A 3 is a diagram showing an example of the hair bundle region 324 recognized from the image. Fig. 14B 3 is a diagram showing an example in which the hair bundle region 324 is divided into a plurality of partial regions 326 . Fig. 14C This is a diagram showing another example in which the hair bundle region 324 is divided into a plurality of partial regions 326 .

[0101] exist Fig.14A In FIG. 3 , the outline of the hair bundle area 324 is represented by a white line. Fig. 14B and Fig. 14C In FIG. 3 , a plurality of partial regions 326 are represented by rectangular black dotted line frames. Fig. 14B In the example of , the plurality of partial regions 326 are arranged adjacent to each other (ie, without any gaps therebetween) in the X direction. Fig. 14C In the example of , the plurality of partial regions 326 are arranged discretely (i.e., separated by gaps) in the X direction. Fig. 14C As in the example of FIG. 1 , a case where a part of the hair bundle area 324 is associated with a plurality of discrete partial areas 326 is also interpreted as being equivalent to a case where “the hair bundle area is divided into a plurality of partial areas”.

[0102] As described above, the hair bundle 22 includes a plurality of partial hair bundles. Each partial hair bundle is a cluster of hairs implanted in a corresponding hole provided in the hair implantation seat 21. Each partial hair bundle has a specified width. The "width" of a partial hair bundle is equivalent to the width of the hole in which the partial hair bundle is implanted in the first direction (X direction) (for example, about 1 mm). The width of each of the plurality of partial regions 326 (i.e., the dimension in the X direction) can be set to a value substantially equal to the width of each of the plurality of partial hair bundles in the image. The processing circuit 110 can also set the plurality of partial regions 326 to surround at least the root of a partial hair bundle corresponding to each partial region. The height of each partial region 326 (i.e., the dimension in the Y direction) can be set, for example, to a value greater than the length in the Y direction of the longest hair included in the partial region 326.

[0103] exist Fig. 14B and Fig. 14CIn the example shown, each of the plurality of partial regions 326 is in the shape of a rectangle, but the shape of each partial region is not limited to a rectangle. Fig.14D As shown, the processing circuit 110 may divide the hair bundle area 324 into a plurality of partial areas 326 by a plurality of dividing lines 325 extending in the Y direction and arranged at predetermined intervals along the X direction. In this case, the shape of each partial area 326 is a shape different from a rectangle. The widths of the plurality of partial areas 326 do not need to be constant and may be different from each other.

[0104] As described above, the processing circuit 110 may also identify the type of the toothbrush 20 based on the acquired image. Alternatively, the processing circuit 110 may also acquire information indicating the type of the toothbrush 20 determined by other methods (e.g., input from a user). The processing circuit 110 may also set the number of divisions based on the plurality of partial regions 326 based on the type of the toothbrush 20. The number of partial bristle bundles (i.e., the number of holes) may differ depending on the type of the toothbrush 20. Therefore, in the case of division in which one partial region 326 corresponds to one partial bristle bundle, the number of divisions based on the plurality of partial regions 326 is set according to the type of the toothbrush 20.

[0105] The processing circuit 110 may also set the number of divisions based on the plurality of partial regions 326 based on the shape of the hair bundle region 324. For example, the number of divisions may be adjusted based on the ratio of the size of the hair bundle region 324 in the X direction to the size in the Y direction. Alternatively, when the hair bundle region 324 is extracted as a region of a deformed shape, the number of divisions may be adjusted so that the partial region 326 with an extremely small height is not included. By such an adjustment, it is easy to more accurately determine the height of the hair bundle region 324.

[0106] In step S182, the processing circuit 110 determines the height of the hair bundle region 324 based on the length (i.e., height) of the hair bundle in each of the plurality of partial regions 326. For example, the processing circuit 110 can be configured to determine the height of the hair bundle region 324 by the following processing. First, the processing circuit 110 obtains the maximum value of the length of the hair bundle included in each of the plurality of partial regions 326 for each of the plurality of partial regions 326. Specifically, the processing circuit 110 obtains the hair length (i.e., the size in the Y direction) according to the position of each of the partial regions 326 in the X direction, and sets the obtained maximum value of the hair length as the maximum value of the length of the hair bundle in the partial region 326. Next, the processing circuit 110 determines the minimum length among the maximum values ​​of the length of the hair bundle in the plurality of partial regions 326 as the length (or hair length) of the hair bundle region 324. By such processing, the length of the hair bundle in, for example, the most worn partial region 326 among the plurality of partial regions 326 can be determined as the length of the hair bundle region 324.

[0107] In step S183, the processing circuit 110 determines whether the height of the hair bundle area determined in step S182, that is, the minimum length among the maximum values ​​of the lengths of the hair bundles in the plurality of partial areas 326, is within a prescribed range. If the length is within the prescribed range, the process proceeds to step S185. If the length is outside the prescribed range, the process proceeds to step S184. The prescribed range can be defined by a lower limit value and an upper limit value of the length pre-set for each type of toothbrush 20. The lower limit value and the upper limit value can be recorded in advance in the storage device 120. If the length is smaller than the lower limit value of the prescribed range, or larger than the upper limit value, the process proceeds to step S184.

[0108] The lower limit and upper limit of the prescribed range are set based on the range of bristle lengths of the assumed toothbrush. The lower limit can be set, for example, to a value of about 70% to 80% of the bristle length of the brush portion of a new toothbrush. The upper limit can be set, for example, to a value of about 120% to 130% of the bristle length of the brush portion of a new toothbrush.

[0109] In addition, the prescribed range may be specified by only one of the lower limit value and the upper limit value. That is, the range above the lower limit value may be specified as the prescribed range, or the range below the upper limit value may be specified as the prescribed range. In the case where only the lower limit value is specified, if the height of the hair bundle area determined in step S182 is less than the lower limit value, the process proceeds to step S184. On the other hand, in the case where only the upper limit value is set, if the height of the hair bundle area determined in step S182 is greater than the upper limit value, the process proceeds to step S184.

[0110] In step S184, the processing circuit 110 causes the display 220 to display a warning including information for urging the user to take another photo. In addition, the processing circuit 110 is not limited to displaying, and can also output information for urging the user to take another photo by other means such as sound. After step S184, the process returns to step S120 to re-identify the brush head 26.

[0111] By the processing of steps S183 and S184 , when the height of the bristle bundle area determined in step S182 is out of the expected range, the user can be urged to re-photograph and re-acquire an image suitable for determining the state of the toothbrush 20 .

[0112] In addition, if it is determined in step S183 that the height of the hair bundle area is not within the prescribed range (No), the process of step S184 is not performed and the process returns to step S170. That is, the processing circuit 110 may also automatically perform the shooting and recognition process without depending on the user's operation. In other words, the processing circuit 110 may also perform image acquisition and hair bundle area recognition again when the minimum length among the maximum values ​​of the lengths of the hair bundles in the plurality of partial areas 326 deviates from the prescribed range. As a result, the user can automatically acquire an image suitable for determining the state of the toothbrush 20 again without performing the operation of shooting again.

[0113] In addition, when it is determined in step S183 that the height of the hair bundle area is not within the prescribed range (No), the processing of step S184 may not be performed, and the recognition processing may be performed again for the same image. That is, the processing circuit 110 may perform the recognition of the hair bundle area again when the minimum length among the maximum values ​​of the lengths of the hair bundles in the plurality of partial areas 326 deviates from the prescribed range. Even when the same image is used as the object, the hair bundle area may be correctly recognized by changing the image recognition algorithm, adjusting the parameters of the image recognition algorithm, or adjusting the brightness of the image. In this way, when it is determined in step S183 that it is No, the processing circuit 110 may perform the recognition of the hair bundle area again after performing the processing such as changing or adjusting the algorithm or adjusting the brightness of the image.

[0114] The processing circuit 110 may perform the following processing in step S182 instead of performing the processing of steps S183 and S184. First, similarly to the above-mentioned example, the processing circuit 110 obtains the maximum value of the length of the hair bundle included in each partial area 326 for each partial area 326 among the plurality of partial areas 326. Next, the processing circuit 110 excludes one or more partial areas among the plurality of partial areas 326 whose maximum value of the length of the hair bundle deviates from a prescribed range. As described above, the prescribed range can be specified by at least one of a lower limit value and an upper limit value. Thus, the partial area whose maximum value of the length of the hair bundle is an abnormal value is excluded. Next, the processing circuit 110 determines the minimum length among the maximum values ​​of the length of the hair bundle in the partial areas that are not excluded among the plurality of partial areas as the length of the hair bundle area (or hair length).

[0115] Alternatively, the processing circuit 110 may determine the height of the hair bundle area by the following processing. First, the processing circuit 110 excludes one or more partial areas 326 having a height that deviates from a predetermined range from among the plurality of partial areas 326. In this example, partial areas having abnormal height values ​​are excluded based on the heights of the partial areas 326 determined in the segmentation process. Next, the processing circuit 110 obtains the maximum value of the length of the hair bundle included in each of the partial areas that are not excluded from among the plurality of partial areas for each partial area. Furthermore, the processing circuit 110 determines the minimum length among the maximum values ​​of the length of the hair bundle in the partial areas that are not excluded as the height of the hair bundle area (or hair length).

[0116] When the height of the hair bundle area is determined by excluding the partial area having a height that is not usually considered, the processing of steps S183 and S184 is not performed, and the process proceeds to step S185 after step S182. By such processing, for example, when the hair bundle area is extracted in a deformed shape due to inaccurate segmentation and an extremely low partial area is set, it is possible to prevent the length of the hair bundle in the extremely low partial area from being processed as the hair length.

[0117] Fig.15A and Fig. 15B 3 is a diagram showing an example of an image in which the hair bundle region 324 having a deformed shape is extracted. Fig.15A The image shown in FIG. 1 is blurred, and as a result, the outline of the hair bundle area 324 is not accurately extracted. Fig. 15B If the hair is divided into a plurality of partial regions as shown, it is possible to set a segmented region with a very small height. Therefore, it is effective to set a threshold value Th of about 70% to 80% of the assumed hair length and exclude segmented regions with a height smaller than the threshold value Th. Fig. 15B In the example of FIG. 1 , two partial regions 326 a and 326 b located on the left are excluded, and the height of the hair bundle region 324 is determined based on the length of the hair bundle in the remaining partial region.

[0118] exist Fig.13 In step S185 shown in FIG. 1 , the processing circuit 110 estimates the inclination of the toothbrush 20 relative to the reference posture during the photographing. FIG. 11A to FIG. 11C That is, the processing circuit 110 estimates the inclination of the toothbrush 20 relative to the reference posture when photographing based on the contour of at least one of the bristle bundle area and the bristle implantation seat area identified from the image acquired in step S170. More specifically, the processing circuit 110 estimates the inclination based on the shape of the boundary between the bristle bundle area and the bristle implantation seat area, or the positions of at least three points on the boundary.

[0119] In the next step S186, the processing circuit 110 corrects the height of the bristle bundle region based on the estimated tilt to calculate the bristle length of the toothbrush 20. Thus, the bristle length can be calculated without the influence of the tilt of the toothbrush 20 at the time of shooting.

[0120] In this embodiment, by Figure 4 The processing of step S130 shown in the figure is to shoot the toothbrush 20 in a state where the inclination relative to the reference posture is small. However, even so, a deviation of an angle of, for example, 5 to 10 degrees may occur. In the case where the posture of the toothbrush 20 at the time of shooting is inclined relative to the reference posture, a difference occurs between the height of the hair bundle area determined in step S182 and the actual hair length. Therefore, in the present embodiment, the processing circuit 110 corrects the height of the hair bundle area determined in step S182 based on the inclination of the toothbrush 20 relative to the reference posture, thereby calculating the final hair length. That is, the processing circuit 110 estimates the hair length of the toothbrush 20 based on the inclination estimated in step S185 and the height of the hair bundle area determined in step S182. Through such processing, the influence of the inclination of the toothbrush 20 at the time of shooting can be removed to more accurately estimate the hair length. In addition, the processing of step S185 can also be performed before the processing of steps S181 and S182.

[0121] FIG. 16A to FIG. 16C 2 is a diagram for explaining a method of correcting hair length based on inclination. Here, the angle of rotation of the toothbrush 20 about the X-axis relative to the reference posture is θ, the actual hair length is a0, and the height of the hair bundle area in the image (i.e., the apparent hair length) is a. Figure 6 As shown, when rotating clockwise in the positive direction of the X-axis, a positive value is taken. Fig.16A It is a schematic diagram of an image of the toothbrush 20 in the case of θ<0 and the bristle bundle 22 and the bristle implantation base 21 viewed from the positive direction of the X-axis. Fig. 16B It is a schematic diagram of an image of the toothbrush 20 when θ=0 and the bristle bundle 22 and the bristle implantation base 21 viewed from the positive direction of the X-axis. Fig. 16C It is a schematic diagram of an image of the toothbrush 20 in the case of θ>0 and the bristle bundle 22 and the bristle implantation base 21 viewed from the positive direction of the X-axis.

[0122] like Fig.16A As shown in FIG. 1 , when θ<0, the hair length a0 can be obtained by calculating a0=(a+b) / cosθ. Here, b represents the height of the portion of the hair closest to the front of the hair bundle 22 that is hidden inside the hair implantation base 21 in the image. b can be, for example, Fig.11AAs shown in FIG. 1 , the distance d between the central point P3 on the boundary between the hair bundle area 324 and the hair implant seat area 323 and the line segment connecting the two points P1 and P2 at both ends of the boundary is determined. The apparent hair length a is the height of the hair bundle area determined in step S182. The angle θ is the angle of inclination estimated in step S185. As described above, the angle θ can also be determined based on the distance d. Information such as a table or function that specifies the correspondence between the distance d and the angle θ and the value b can be pre-stored in the storage device 120. The processing circuit 110 can refer to this information, calculate the angle θ and the value b based on the distance d, and calculate the hair length a0 by the operation of a0=(a+b) / cosθ.

[0123] like Fig. 16B As shown, when θ=0, the hair length a0 matches the apparent hair length a in the image. In this case, the processing circuit 110 can process the apparent hair length a as the actual hair length a0 without correcting it.

[0124] like Fig. 16C As shown in the figure, when θ>0, the hair length a0 can be obtained by calculating a0=(af) / cosθ. Here, f represents the height difference between the hair closest to the front and the hair closest to the back in the image. f increases as the angle θ increases. Information such as a table or function that specifies the relationship between the angle θ and the value f can be stored in the storage device 120 in advance. The processing circuit 110 can obtain the value f based on this information and calculate the hair length a0 by calculating a0=(af) / cosθ.

[0125] The processing circuit 110 may also process the value obtained by multiplying the above-mentioned a0 by a proportionality factor for converting the length in the image (e.g., pixel unit) to the actual length (e.g., millimeter unit) as the hair length a0. The proportionality factor may be obtained by, for example, dividing the actual size of the hair bundle 22 in the X direction, which is a known value, by the corresponding size β in the image (see Figure 8 Alternatively, the actual size of the hair implantation seat 21 in the X direction is divided by the corresponding size x in the image.

[0126] Fig.17 : is a graph showing examples of hair length before correction and hair length after correction. The solid line shows an example of the relationship between the height of the hair bundle area detected from the image, that is, the apparent hair length before correction and the angle of rotation about the X-axis. The dashed line shows an example of hair length after correction by the processing of this embodiment. As shown in the graph, the larger the absolute value of the rotation angle, the more the hair length is corrected, and a value close to the actual hair length can be obtained.

[0127] exist Fig.13In the example of step S183, the determination is performed between step S182 and step S185, but the same determination may be performed after step S186. Fig.18 is a flow chart showing such an implementation. Fig.18 In the example, after step S182, steps S185 and S186 are processed, and then step S183a is processed. In step S183a, the processing circuit 110 determines whether the hair length a0 calculated in step S186 is within the specified range. When the hair length a0 is within the specified range, the process proceeds to step S187. When the hair length a0 is outside the specified range, the process proceeds to step S184. The specified range can be specified using a lower limit value and an upper limit value of the height pre-set for each type of toothbrush 20. The lower limit value and the upper limit value can be pre-recorded in the storage device 120. When the hair length a0 is smaller than the lower limit value of the specified range, or larger than the upper limit value, the process proceeds to step S184. The process of step S184 is the same as that of step S184. Fig.13 The same process is used in the example. In addition, Fig.18 In the example of FIG. 1 , the process of step S184 may not be performed, and the hair bundle area may be recognized again based on the same image or an image captured again.

[0128] In step S187, the processing circuit 110 compares the hair length a0 with the first threshold value Th1 previously stored in the storage device 120. If a0<Th1, the process proceeds to step S189a. If a0≥Th1, the process proceeds to step S188.

[0129] In step S188, the processing circuit 110 compares the hair length a0 with the second threshold Th2 pre-stored in the storage device 120. The second threshold Th2 is greater than the first threshold Th1. If a0<Th2, the process proceeds to step S189b. If a0≥Th2, the process proceeds to step S189c.

[0130] In step S189a, the processing circuit 110 substitutes the value R1 into the determination result. In step S189b, the processing circuit 110 substitutes the value R2 into the determination result. In step S189c, the processing circuit 110 substitutes the value R3 into the determination result. The values ​​R1, R2, and R3 are associated with the display content.

[0131] Fig.19 An example of a table showing the correspondence between the judgment result and the display content is shown. Such a table is pre-stored in the storage device 120. Judgment result R1 corresponds to the display content "The brush part has deteriorated. Replacement is recommended." Judgment result R2 corresponds to the display content "It is almost time to replace." Judgment result R3 corresponds to the display content "It can still be used." The processing circuit 110 can refer to the table shown in FIG. Fig.19 The table shown in FIG. 2 makes the display 220 display a message corresponding to the determination result. For example, when the determination result is R1, the display 220 displays a message corresponding to the determination result. Fig.12 The screen shown.

[0132] As described above, according to the present embodiment, the inclination of the toothbrush 20 relative to the reference posture during shooting can be estimated based on the outline or size of at least one of the bristle bundle area and the bristle implantation seat area identified from the image of the brush head 26 of the toothbrush 20. Thus, it is possible to determine whether the posture of the toothbrush 20 during shooting is appropriate. In addition, the content of the notification to the user can be changed depending on whether the index value indicating the degree of inclination of the toothbrush 20 relative to the reference posture during shooting deviates from the reference range. For example, a notification indicating a warning can be made when the index value deviates from the reference range. In addition, a notification indicating suitability can be made when the index value is within the reference range. Thus, when the posture of the toothbrush 20 during shooting is inappropriate, the user can be notified of the situation, prompting the toothbrush 20 to be shot in the correct posture.

[0133] In addition, the bristle length of the toothbrush 20 can be estimated based on the estimated inclination of the toothbrush 20 and the height of the bristle bundle region determined by the image recognition process. Thus, the degree of deterioration of the toothbrush 20 can be determined based on the estimated bristle length.

[0134] Furthermore, the bristle region in the image of the toothbrush 20 can be divided into a plurality of partial regions, and the bristle length of the toothbrush 20 can be estimated based on the length of the bristle in each partial region. Thus, the bristle length of the toothbrush 20 can be estimated more accurately.

[0135] (Other Embodiments)

[0136] In the above-mentioned embodiment, Figure 4 In the operations of steps S110 to S170 shown in the figure, the user is informed whether the posture of the toothbrush 20 is appropriate based on the preview image before the user performs the shooting operation, but this function may be omitted. In this case, after the user performs the shooting operation, it may be determined whether the posture of the toothbrush 20 is appropriate based on the acquired image through the same processing as above, and if the posture is not appropriate, information for urging the user to shoot again is output.

[0137] In the above-mentioned embodiment, Fig.13 and Fig.18In steps S181 and S182 shown in the figure, the hair bundle area identified from the image is divided into a plurality of partial areas, the length of the hair bundle is calculated for each partial area, and the height of the hair bundle area in the image, i.e., the apparent hair length, is determined based on the length of the hair bundle. This process may be omitted, and the apparent hair length may be directly calculated based on the identified hair bundle area. In this case, for example, the hair length may be calculated according to the position in the X-axis direction in the hair bundle area through image recognition, and the minimum value, maximum value, average value, or median value of the length of the hair may be determined as the height of the hair bundle area.

[0138] Fig.13 and Fig.18 The processing of steps S185 and S186 shown in FIG. 1 is performed as required. For example, by strictly setting Figure 4 The reference range of the determination process in step S130 shown in FIG. 1 is set to the reference range of the determination process in step S130, and thus, when an image is captured in which the inclination of the toothbrush 20 relative to the reference posture is very small, the correction processes of steps S185 and S186 can be omitted. In this case, the height of the hair bundle area determined in step S182 can also be processed as the hair length a0.

[0139] exist Fig.13 and Fig.18 In the example of , a two-stage determination is performed by comparing the hair length a0 with two threshold values ​​Th1 and Th2, but this is only an example. For example, the hair length a0 may be compared with one threshold value to determine whether the toothbrush 20 needs to be replaced. Alternatively, the hair length a0 may be compared with three or more threshold values ​​to determine the degree of deterioration of the toothbrush 20 in a more detailed manner.

[0140] In the above-mentioned embodiment, Figure 6 As shown, the posture of the toothbrush 20 when the camera 210 is photographed in the negative direction of the Z axis (i.e., the second direction) is set as a reference posture, and the tilt caused by the rotation around the X axis is estimated. Instead of or in addition to this processing, the posture of the toothbrush 20 when the camera 210 is photographed in the negative direction of the X axis (i.e., the first direction) is set as a reference posture, and the tilt caused by the rotation around the Z axis is estimated. By photographing the toothbrush 20 in such a posture, an image that can determine the spread of the hair bundles can be obtained. In this case, the processing circuit 110 can determine the deterioration state of the toothbrush 20 by considering not only the hair length of the toothbrush 20 but also the spread of the hair bundles. Specifically, the processing circuit 110 obtains an image generated by photographing the brush head 26 by the camera 210 in the negative direction of the X axis (the first direction), and estimates the spread of the hair bundles based on the image. The degree of deterioration of the toothbrush 20 can be estimated based on the estimated hair length and the spread of the hair bundles.

[0141] Fig. 201 is a diagram showing an example of a photographing instruction screen for acquiring an image capable of determining the unraveling state of hair bundles. Fig. 20 The screen shown is a guide screen for urging the user to shoot the brush head 26 of the toothbrush 20 in the negative direction of the X-axis (i.e., the first direction). The user follows the displayed instructions and directs the camera 210 toward the front end of the brush head 26 so that the brush head 26 of the toothbrush 20 enters the guide line 223. When the user presses the shooting button 224, the shooting process is executed and the image data is recorded in the storage device 120.

[0142] exist Fig. 20 In the example, also with Figure 4 and Figure 5 The example shown in the figure similarly executes the processing of steps S110 to S170. Figure 5 The shooting and Fig. 20 In the case of photographing both sides as shown, the processing of steps S110 to S170 is repeated twice. Figure 5 The shooting and Fig. 20 Any of the shots shown can be performed first.

[0143] exist Fig. 20 In the example of , in step S130, the processing circuit 110 can be configured to calculate an index value indicating the degree of inclination relative to the reference posture caused by the rotation around the second direction (Z direction) based on the size of the hair implantation seat area in the third direction (Y direction) in the image. The processing circuit 110 can determine whether the posture of the toothbrush 20 is appropriate by comparing the index value with the reference range.

[0144] Fig.21 2 is a diagram for explaining a method for estimating the inclination of the toothbrush 20 about the Z axis relative to the reference posture. Fig.21 As shown, the width of the hair transplantation seat area 323 corresponding to the hair transplantation seat 21 in the image (i.e., the dimension in the Z direction) is set to z, and the height of the hair transplantation seat area 323 (i.e., the dimension in the Y direction) is set to y. In addition, the height of the hair bundle area 324 corresponding to the hair bundle 22 in the image (i.e., the dimension in the Y direction) is set to α, and the width of the hair bundle area 324 (i.e., the dimension in the Z direction) is set to γ.

[0145] The height y of the hair implant seat area 323 and the height α of the hair bundle area 324 in the image vary according to the size of the rotation angle of the toothbrush 20 around the Z axis. On the other hand, the width z of the hair implant seat area 323 hardly changes due to the rotation of the toothbrush 20 around the Z axis. Therefore, the processing circuit 110 can use values ​​such as y, α, the ratio of α to γ, the ratio of α to z, the ratio of y to γ, or the ratio of y to z as index values ​​to evaluate the inclination of the toothbrush 20 around the Z axis. For example, the processing circuit 110 can use γ / y or z / y as an index value, and determine that the posture of the toothbrush 20 is inappropriate when the index value deviates from the specified reference range. In this way, the index value can be calculated based on the ratio of the dimension z of the hair implant seat area 323 in the z direction (i.e., the second direction) or the dimension γ of the hair bundle area 324 in the z direction to the dimension y of the hair implant seat area 323 in the y direction (i.e., the third direction).

[0146] Fig. 22 1 is a flowchart showing an example of a deterioration determination process performed based on the spreading state of the hair bundle in addition to the hair length. Fig. 22 The processing shown can be Figure 4 The process is performed in step S180 shown in the figure.

[0147] exist Fig. 22 In the example, the processing circuit 110 determines in step S801 whether the category of the toothbrush 20 that is the object of state determination is the first type. The category of the toothbrush 20 is classified into multiple types such as "normal" (first type), "extra fine" (second type), and "long hair" (third type). The processing circuit 110 performs segmentation and other processing based on the acquired image, thereby determining the category of the toothbrush 20 and determining whether the category is the first type. If the category is the first type, the process proceeds to step S809. If the category is not the first type, the process proceeds to step S802. In addition, such category determination may be omitted.

[0148] In step S802, the processing circuit 110 determines the height of the hair bundle area based on the image obtained by the camera 210 shooting the toothbrush 20 in a second direction parallel to the Z axis. Fig.13 The processing of steps S181 and S182 shown in the figure is determined by the same method, or can be determined by other methods.

[0149] In step S803, the processing circuit 110 estimates the inclination of the toothbrush 20 relative to the reference posture during the photographing. Fig.13 The processing of step S185 shown is the same.

[0150] In step S804, the processing circuit 110 corrects the height of the hair bundle area based on the estimated inclination, thereby calculating the hair length a0. Fig.13 The processing of step S186 shown is the same.

[0151] In step S805, the processing circuit 110 determines whether the hair length a0 is less than the threshold value Th1. If a0<Th1, the process proceeds to step S807. If a0<Th1 is not true, the process proceeds to step S806.

[0152] In step S806, the processing circuit 110 determines whether the hair length a0 is less than a threshold value Th2. The threshold value Th2 is greater than the threshold value Th1. If a0<Th2, the process proceeds to step S808. If a0<Th2 is not true, the process proceeds to step S809.

[0153] In step S807, the processing circuit 110 sets the determination result to R1 and enters Figure 4 Step S190 is shown.

[0154] In step S808 , the processing circuit 110 sets the value “1” to the determination flag F. Thereafter, the process proceeds to step S810 .

[0155] In step S809 , the processing circuit 110 sets the value “2” to the determination flag F. Thereafter, the process proceeds to step S810 .

[0156] Next, refer to Fig.23 An example of the processing performed in step S810 will be described. Fig.23 This is a flowchart showing a specific example of the process of step S810.

[0157] In step S811 , the processing circuit 110 extracts the side portion changes c1 and c2 based on the image acquired by photographing in a first direction parallel to the X-axis.

[0158] Fig.24A and Fig. 24B It is a diagram for explaining the side portion change amounts c1 and c2. Fig.24A This is a diagram for explaining the degree of deformation of the bristle bundle 22 of the "ultra-fine" type toothbrush 20. Fig. 24B 2 is a diagram for explaining the degree of deformation of the bristle bundle 22 of the "normal" type toothbrush 20. The processing circuit 110 is based on Fig.24A and Fig. 24BThe deformation degree of the hair bundle 22 is determined by the left and right changes of the hair bundle 22 in the image shown, that is, the side changes c1 and c2. The side change c1 represents the difference in position between the leftmost part in the hair bundle area in the image and the leftmost part in the hair transplant seat area in the image in the horizontal direction of the image. The side change c2 represents the difference in position between the rightmost part in the hair bundle area in the image and the rightmost part in the hair transplant seat area in the image in the horizontal direction of the image. The processing circuit 110 can obtain the side changes c1 and c2 by image recognition.

[0159] In step S812, the processing circuit 110 determines whether both the side portion changes c1 and c2 are less than the threshold value Th3. If both c1 and c2 are less than the threshold value Th3, the process proceeds to step S813. If at least one of c1 and c2 is greater than or equal to Th3, the process proceeds to step S817.

[0160] In step S813, the processing circuit 110 determines whether both the side portion changes c1 and c2 are less than a threshold value Th4. Threshold value Th4 is less than threshold value Th3. When both c1 and c2 are less than threshold value Th4, the process proceeds to step S814. When at least one of c1 and c2 is greater than or equal to Th4, the process proceeds to step S816.

[0161] In step S814 , the processing circuit 110 determines whether the value of the determination flag F is 2. If F=2, the process proceeds to step S815 . If not, the process proceeds to step S816 .

[0162] In step S815, the processing circuit 110 sets the determination result to R3 and enters Figure 4 Step S190 is shown.

[0163] In step S816, the processing circuit 110 sets the determination result to R2 and enters Figure 4 Step S190 is shown.

[0164] In step S817, the processing circuit 110 sets the determination result to R1 and enters Figure 4 Step S190 is shown.

[0165] Fig.25 1 is a diagram showing an example of a table storing the values ​​of threshold values ​​Th1, Th2, Th3, and Th4 referenced by the processing circuit 110 in the above-mentioned processing. In this example, a table storing different values ​​of threshold values ​​Th1 to Th4 for each type of brush unit is stored in the storage device 120. By referring to such a table, the processing circuit 110 can make appropriate determinations for each type of brush unit. In addition, Fig.25The values ​​of the threshold values ​​Th1 to Th4 shown are examples, and values ​​different from the values ​​shown in the figure may be set.

[0166] Through the above processing, the degree of deterioration of the toothbrush 20 can be more appropriately determined based on the bristle length of the toothbrush 20 and the spreading state of the bristle bundles 22, and the determination result can be notified to the user.

[0167] Although the embodiments of the present disclosure have been described above, the embodiments are not limited thereto, and various modifications can be made within the scope of the gist of the embodiments. In addition, a part or all of the various embodiments can be combined to form a new embodiment.

[0168] In the above-mentioned embodiment, the state determination system 100 is provided in the terminal device 10. The present disclosure is not limited to this method, and the functions of the state determination system 100 may also be distributed in a plurality of devices. Figure 3 As shown, part of the functions of the state determination system 100 may also be possessed by an external device 40 such as a server computer on the cloud. For example, among the functions of the state determination system 100, the processing of acquiring an image and the processing of controlling the display may be performed by the terminal device 10, and the processing of determining the type of the toothbrush 20 in the image, the processing of determining whether the posture of the toothbrush 20 is appropriate, and the processing of determining the deterioration state of the toothbrush 20 may be performed by the external device 40. Alternatively, part of the processing of determining the type of the toothbrush 20 in the image, the processing of determining whether the posture of the toothbrush 20 is appropriate, and the processing of determining the deterioration state of the toothbrush 20 may be performed by the external device 40. In this way, part of the functions of the state determination system may also be performed by an external device 40 such as a cloud server that communicates with the terminal device 10 via a network. In this case, the terminal device 10 and the external device 40 cooperate to function as a state determination system. With such a structure, it is possible to achieve high-speed processing. In this way, the state determination system is not limited to one device, and can also be implemented by multiple devices.

[0169] As such, the state determination system can include a server computer and a terminal device connected to the server computer via a network. The terminal device can be configured to obtain an image generated by photographing a brush head of a toothbrush from the side by a camera device, and send the image to the server computer. The brush head includes a bristle bundle and a bristle seat supporting the bristle bundle. The server computer can also identify at least one of a bristle bundle area corresponding to the bristle bundle and a bristle seat area corresponding to the bristle seat from the image, estimate the inclination of the toothbrush relative to a reference posture at the time of photographing based on the contour of at least one of the bristle bundle area and the bristle seat area in the image, and send information based on the estimated result of the inclination to the terminal device. The information based on the estimated result of the inclination can be either information indicating the estimated inclination itself or information related to the bristle length of the toothbrush calculated based on the estimated result of the inclination.

[0170] The server computer may also calculate an index value indicating the degree of inclination of the toothbrush relative to the reference posture when photographed based on the contour or size of at least one of the hair bundle area and the hair implantation seat area in the image received from the terminal device, and send information related to the index value to the terminal device. The terminal device may also change the content of the notification to the user according to whether the index value deviates from a specified reference range.

[0171] In addition, the server computer may also identify a hair bundle area corresponding to a hair bundle from an image received from a terminal device, divide the hair bundle area in the image into a plurality of partial areas arranged in a direction intersecting the direction in which the hair bundle extends, estimate the hair length of the toothbrush based on the length of the hair bundle in each of the plurality of partial areas, and send information related to the hair length to the terminal device. The information related to the hair length may be information indicating the hair length itself or information related to the deterioration state of the toothbrush estimated based on the hair length.

[0172] (Note)

[0173] Based on the description of the above embodiments, the following technology is disclosed.

[0174] (Technology 1)

[0175] A method is a method executed by a computer, the method comprising the following steps:

[0176] Acquire an image generated by photographing a brush head of a toothbrush by a camera device, wherein the brush head includes a bristle bundle and a bristle implantation seat supporting the bristle bundle;

[0177] Identifying at least one of a hair bundle area corresponding to the hair bundle and a hair implantation seat area corresponding to the hair implantation seat from the image;

[0178] calculating an index value indicating a degree of inclination of the toothbrush relative to a reference posture at the time of photographing based on a size of at least one of the bristle bundle area and the bristle implantation seat area in the image; and

[0179] The content of the notification to the user is changed depending on whether the index value deviates from a predetermined reference range.

[0180] According to the above structure, an index value indicating the degree of inclination of the toothbrush relative to the reference posture during shooting can be calculated based on the size of at least one of the bristle bundle area and the bristle implantation seat area identified from the image of the brush head of the toothbrush, and the content of the notification to the user can be changed according to whether the index value deviates from the reference range. Thus, if the posture of the toothbrush during shooting is inappropriate, the user can be notified of the situation, so as to encourage the toothbrush to be shot in the correct posture.

[0181] (Technique 2)

[0182] The method according to technique 1, wherein:

[0183] The step of changing the content of the notification comprises the following steps:

[0184] When the index value deviates from the reference range, issuing a warning notification; and

[0185] When the index value is within the reference range, notification indicating compliance is performed.

[0186] According to the above configuration, it is possible to effectively inform the user whether the posture of the toothbrush during the image capture is appropriate.

[0187] (Technique 3)

[0188] The method according to technology 1 or 2, wherein,

[0189] The notification to the user includes: controlling at least one of a display device, a light emitting device, a speaker, and a tactile device to apply at least one of visual stimulation, auditory stimulation, and tactile stimulation to the user.

[0190] According to the above configuration, by applying at least one of visual stimulation, auditory stimulation and tactile stimulation to the user, the user can be informed whether the index value deviates from the reference range. Thus, the user can be effectively informed whether the posture of the toothbrush during shooting is appropriate.

[0191] (Technique 4)

[0192] The method according to any one of techniques 1 to 3, wherein:

[0193] The notification to the user includes causing a display device to display information indicating whether the posture of the toothbrush is appropriate.

[0194] According to the above configuration, the display device can display information indicating whether the posture of the toothbrush is appropriate. Thus, the user can be informed more effectively whether the posture of the toothbrush is appropriate.

[0195] (Technique 5)

[0196] The method according to any one of techniques 1 to 4, wherein:

[0197] The method further includes the steps of estimating the degree of deterioration of the toothbrush based on the size of the bristle bundle area and outputting information related to the deterioration of the toothbrush when the index value is within the reference range.

[0198] According to the above configuration, when the index value is within the reference range, the user can be notified of information related to the deterioration of the toothbrush, thereby enabling the user to determine whether the toothbrush needs to be replaced.

[0199] (Technique 6)

[0200] The method according to technique 5, wherein:

[0201] The step of outputting the information related to the degradation includes the step of causing a display device to display the information related to the degradation.

[0202] According to the above configuration, the display device can display information related to the deterioration of the toothbrush, so that the user can determine whether the toothbrush needs to be replaced based on the displayed information.

[0203] (Technique 7)

[0204] The method according to any one of techniques 1 to 6, wherein:

[0205] The toothbrush comprises a handle portion extending in one direction, the brush head portion is located at the front end of the handle portion,

[0206] The hair implantation seat has a hair implantation surface extending in a first direction and a second direction, wherein the first direction is along the direction in which the handle portion extends, and the second direction is perpendicular to the first direction.

[0207] The hair bundles extend from the hair implantation surface along a third direction, and the third direction is perpendicular to both the first direction and the second direction.

[0208] The reference posture is a posture of the toothbrush when the camera device photographs the brush head toward the second direction.

[0209] The index value is calculated based on the size (y) of the hair implant base area in the image in the third direction, and indicates a degree of inclination caused by rotation about the first direction with respect to the reference posture.

[0210] According to the above structure, the index value is calculated based on the size of the hair implantation seat area in the third direction in the image, and the index value represents the degree of inclination caused by rotating around the first direction relative to the reference posture. Thus, it can be determined that the toothbrush has rotated around the first direction relative to the reference posture.

[0211] (Technology 8)

[0212] The method according to technique 7, wherein:

[0213] The index value is calculated based on the ratio of the size (β) of the hair bundle area or the hair implantation base area in the image in the first direction to the size (y) of the hair implantation base area in the image in the third direction.

[0214] According to the above configuration, the index value is calculated based on the ratio of the size of the bristle bundle area in the first direction to the size of the bristle implantation base area in the third direction. This makes it possible to more accurately determine whether the toothbrush has rotated about the first direction relative to the reference posture.

[0215] (Technique 9)

[0216] The method according to any one of techniques 1 to 6, wherein:

[0217] The toothbrush comprises a handle portion extending in one direction, the brush head portion is located at the front end of the handle portion,

[0218] The hair implantation seat has a hair implantation surface extending in a first direction and a second direction, wherein the first direction is along the direction in which the handle portion extends, and the second direction is perpendicular to the first direction.

[0219] The hair bundles extend from the hair implantation surface along a third direction, and the third direction is perpendicular to both the first direction and the second direction.

[0220] The reference posture is a posture of the toothbrush when the camera captures the brush head from the first direction.

[0221] The index value is calculated based on the size (y) of the hair implant base area in the image in the third direction, and indicates a degree of inclination caused by rotation about the second direction with respect to the reference posture.

[0222] According to the above structure, the index value is calculated based on the size of the hair implantation seat area in the third direction in the image, and the index value represents the degree of inclination caused by the rotation around the second direction relative to the reference posture. Thus, it can be determined that the toothbrush has rotated around the second direction relative to the reference posture.

[0223] (Technology 10)

[0224] The method according to technique 9, wherein:

[0225] The index value is calculated based on the ratio of the size (γ) of the hair bundle area or the hair implantation base area in the second direction in the image to the size (y) of the hair implantation base area in the third direction in the image.

[0226] According to the above configuration, the index value is calculated based on the ratio of the size of the bristle bundle area in the second direction to the size of the bristle implantation base area in the third direction. This makes it possible to more accurately determine whether the toothbrush has rotated about the second direction relative to the reference posture.

[0227] (Technology 11)

[0228] A system includes a computer that executes the method according to any one of techniques 1 to 10.

[0229] According to the above structure, it is possible to provide a device that executes the method according to any one of techniques 1 to 10.

[0230] (Technology 12)

[0231] A program for causing a computer to execute the method according to any one of techniques 1 to 10.

[0232] According to the above structure, it is possible to provide a program for causing a computer to execute the method according to any one of techniques 1 to 10.

[0233] (Technology 13)

[0234] A system comprises: a server computer; and a terminal device connected to the server computer via a network, wherein:

[0235] The terminal device acquires an image generated by photographing a brush head of a toothbrush by a camera device, and sends the image to the server computer, wherein the brush head includes a bristle bundle and a bristle implantation seat supporting the bristle bundle.

[0236] The server computer performs the following processing:

[0237] Identifying at least one of a hair bundle area corresponding to the hair bundle and a hair implantation seat area corresponding to the hair implantation seat from the image;

[0238] calculating an index value indicating a degree of inclination of the toothbrush relative to a reference posture at the time of photographing based on a size of at least one of the bristle bundle area and the bristle implantation seat area in the image; and

[0239] sending information related to the indicator value to the terminal device,

[0240] The terminal device changes the content of notification to the user according to whether the indicator value deviates from a predetermined reference range.

[0241] According to the above configuration, the server computer and the terminal device cooperate to execute the above method, so that the processing can be accelerated.

[0242] (Technology 14)

[0243] A system according to technique 13, wherein:

[0244] The terminal device performs the following processing:

[0245] When the index value deviates from the reference range, issuing a warning notification; and

[0246] When the index value is within the reference range, notification indicating compliance is performed.

[0247] According to the above configuration, it is possible to more effectively inform the user whether the posture of the toothbrush is appropriate.

[0248] Industrial Applicability

[0249] The technology disclosed herein can be applied to a system capable of determining the deterioration state of a toothbrush, for example. Specifically, the technology disclosed herein can be applied to a device or application that determines the deterioration state of an electric toothbrush or a common manual toothbrush and prompts an appropriate replacement time.

[0250] Description of Reference Numerals

[0251] 10: terminal device; 20: toothbrush; 21: hair-planting seat; 21a: hair-planting surface; 22: hair bundle; 25: handle; 26: brush head; 100: state determination system; 110: processing circuit; 120: storage device; 130: communication I / F; 140: input and output I / F; 210: camera; 220: display.

Claims

1. A method for determining a toothbrush state is a method executed by a computer, the method comprising the following steps: Acquire an image generated by photographing a brush head of a toothbrush by a camera device, wherein the brush head includes a bristle bundle and a bristle implantation seat supporting the bristle bundle; Identifying at least one of a hair bundle area corresponding to the hair bundle and a hair implantation seat area corresponding to the hair implantation seat from the image; calculating an index value indicating a degree of inclination of the toothbrush relative to a reference posture at the time of photographing based on a size of at least one of the bristle bundle area and the bristle implantation seat area in the image; as well as The content of the notification to the user is changed depending on whether the index value deviates from a predetermined reference range.

2. The toothbrush status determination method according to claim 1, wherein: The step of changing the content of the notification comprises the following steps: When the index value deviates from the reference range, issuing a warning notification; and When the index value is within the reference range, notification indicating compliance is performed.

3. The toothbrush status determination method according to claim 1, wherein: The notification to the user includes: controlling at least one of a display device, a light emitting device, a speaker, and a tactile device to apply at least one of visual stimulation, auditory stimulation, and tactile stimulation to the user.

4. The toothbrush status determination method according to claim 1, wherein: The notification to the user includes causing a display device to display information indicating whether the posture of the toothbrush is appropriate.

5. The toothbrush state determination method according to any one of claims 1 to 4, wherein: The toothbrush state determination method further includes the steps of estimating the degree of deterioration of the toothbrush based on the size of the bristle bundle area and outputting information related to the deterioration of the toothbrush when the index value is within the reference range.

6. The toothbrush status determination method according to claim 5, wherein: The step of outputting the information related to the degradation includes the step of causing a display device to display the information related to the degradation.

7. The toothbrush state determination method according to any one of claims 1 to 4, wherein: The toothbrush comprises a handle portion extending in one direction, the brush head portion is located at the front end of the handle portion, The hair implantation seat has a hair implantation surface extending in a first direction and a second direction, wherein the first direction is along the direction in which the handle portion extends, and the second direction is perpendicular to the first direction. The hair bundles extend from the hair implantation surface along a third direction, and the third direction is perpendicular to both the first direction and the second direction. The reference posture is a posture of the toothbrush when the camera captures the brush head toward the second direction. The index value is calculated based on the size of the hair implant base area in the image in the third direction, and the index value indicates a degree of inclination caused by rotation about the first direction with respect to the reference posture.

8. The method for determining the state of a toothbrush according to claim 7, wherein: The index value is calculated based on a ratio of a size of the hair bundle region or the hair implantation seat region in the image in the first direction to a size of the hair implantation seat region in the image in the third direction.

9. The toothbrush state determination method according to any one of claims 1 to 4, wherein: The toothbrush comprises a handle portion extending in one direction, the brush head portion is located at the front end of the handle portion, The hair implantation seat has a hair implantation surface extending in a first direction and a second direction, wherein the first direction is along the direction in which the handle portion extends, and the second direction is perpendicular to the first direction. The hair bundles extend from the hair implantation surface along a third direction, and the third direction is perpendicular to both the first direction and the second direction. The reference posture is a posture of the toothbrush when the camera captures the brush head from the first direction. The index value is calculated based on the size of the hair implant base area in the image in the third direction, and the index value indicates a degree of inclination caused by rotation about the second direction with respect to the reference posture.

10. The method for determining the state of a toothbrush according to claim 9, wherein: The index value is calculated based on a ratio of a size of the hair bundle region or the hair implantation seat region in the image in the second direction to a size of the hair implantation seat region in the image in the third direction. 11 . A toothbrush state determination device comprising a computer for executing the toothbrush state determination method according to claim 1 . 12 . A computer program product comprising a computer program for causing a computer to execute the toothbrush state determination method according to claim 1 .

13. A toothbrush status determination system, comprising: a server computer; and a terminal device connected to the server computer via a network, wherein: The terminal device acquires an image generated by photographing a brush head of a toothbrush by a camera device, and sends the image to the server computer, wherein the brush head includes a bristle bundle and a bristle implantation seat supporting the bristle bundle. The server computer performs the following processing: Identifying at least one of a hair bundle area corresponding to the hair bundle and a hair implantation seat area corresponding to the hair implantation seat from the image; calculating an index value indicating a degree of inclination of the toothbrush relative to a reference posture at the time of photographing based on a size of at least one of the bristle bundle area and the bristle implantation seat area in the image; as well as sending information related to the indicator value to the terminal device, The terminal device changes the content of notification to the user according to whether the indicator value deviates from a predetermined reference range.

14. The toothbrush state determination system according to claim 13, wherein: The terminal device performs the following processing: When the index value deviates from the reference range, issuing a warning notification; as well as When the index value is within the reference range, notification indicating compliance is performed. 15 . A computer-readable recording medium having recorded thereon a computer program for causing a computer to execute the toothbrush state determination method according to claim 1 .

Images