Toothbrush condition determination method, system, and program

Abstract

To provide a method of estimating the inclination of a toothbrush in imaging.SOLUTION: A method executed by a computer is disclosed. The method comprises: acquiring an image generated by imaging a head part from a lateral side by an imaging device, the head part including bristle bundles and a block head supporting the bristle bundles in a toothbrush; recognizing from the image at least one of a bristle bundle area corresponding to the bristle bundles and a block head area corresponding to the block head; and estimating inclination of the toothbrush in imaging from a reference posture, on the basis of the contour of at least one of the bristle bundle area and the block head area in the image.SELECTED DRAWING: Figure 13

Description

[Technical field]

[0001] The present disclosure relates to a method, device, system, and program for determining the condition of a toothbrush. [Background technology]

[0002] Patent Document 1 discloses a timely toothbrush replacement system that enables timely replacement of toothbrushes. This system determines the degree of deformation of the toothbrush bristles based on an image taken from the direction of the toothbrush bristles, and issues an alarm if the degree of deformation of the bristles exceeds a preset limit. This makes it possible to prompt the user to timely replace the toothbrush. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2019-080752 A Summary of the Invention [Problem to be solved by the invention]

[0004] The present disclosure provides a method for more accurately determining the condition, such as the degree of deterioration, of a toothbrush, and a method for assisting in such determination. [Means for solving the problem]

[0005] A method according to one aspect of the present disclosure is a computer-executed method that includes obtaining an image generated by an imaging device photographing a head portion including a bristle bundle of a toothbrush and a bristle base supporting the bristle bundle from the side, recognizing from the image at least one of a bristle bundle region corresponding to the bristle bundle and a bristle base region corresponding to the bristle base, and estimating a tilt of the toothbrush from a reference position at the time of photographing based on a contour of at least one of the bristle bundle region and the bristle base region in the image.

[0006] A system according to another aspect of the present disclosure includes a server computer and a terminal device connected to the server computer via a network. The terminal device acquires an image generated by an imaging device photographing a head unit including a bristle bundle of a toothbrush and a bristle base supporting the bristle bundle from the side, and transmits the image to the server computer. The server computer recognizes at least one of a bristle bundle region corresponding to the bristle bundle and a bristle base region corresponding to the bristle base from the image, estimates an inclination of the toothbrush from a reference position at the time of photographing based on an outline of at least one of the bristle bundle region and the bristle base region in the image, and transmits information based on the estimated inclination to the terminal device.

[0007] A comprehensive or specific aspect of the present disclosure may be realized by an apparatus, a system, a method, an integrated circuit, a computer program, or a computer-readable recording medium, or any combination thereof. The computer-readable recording medium may be a volatile recording medium or a non-volatile recording medium. The recording medium may be any storage medium, such as a semiconductor recording medium, a magnetic recording medium, or an optical recording medium. The apparatus is not limited to a single apparatus, but may be a system composed of two or more apparatuses. When the apparatus is composed of two or more apparatuses, the two or more apparatuses may be arranged in one apparatus, or may be arranged separately in two or more separate apparatuses. Effect of the Invention

[0008] According to one aspect of the present disclosure, it becomes possible to more accurately determine the condition of a toothbrush, such as the degree of deterioration. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram illustrating a terminal device and a toothbrush. [Diagram 2] FIG. 2 is a block diagram showing an example of the configuration of a terminal device. [Diagram 3]FIG. 3 is a diagram illustrating a configuration in which a terminal device and an external device communicate with each other via a network. [Figure 4] FIG. 4 is a flowchart illustrating example operations performed by a processing circuit. [Diagram 5] FIG. 5 is a diagram showing an example of the shooting instruction screen. [Figure 6] FIG. 6 is a diagram showing an example of a reference position of the toothbrush. [Figure 7] FIG. 7 is a diagram for explaining a method for estimating the inclination of the toothbrush from the reference posture based on the dimensions of the bristle base and the bristle bundle of the toothbrush. [Figure 8] FIG. 8 is a diagram showing an example of an image of a toothbrush head. [Figure 9] FIG. 9 is a diagram showing an example of a reference range of the index value. [Figure 10] FIG. 10 is a graph showing an example of the relationship between the index value β / y and the angle of rotation around the X-axis. [Figure 11A] FIG. 11A is a diagram showing an example of a method for estimating the inclination of a toothbrush based on the contour of a bristle bundle region. [Figure 11B] FIG. 11B is a diagram showing an example of a method for estimating the inclination of a toothbrush based on the contour of a bristle bundle region. [Figure 11C] FIG. 11C is a diagram showing an example of a method for estimating the inclination of a toothbrush based on the contour of a bristle bundle region. [Figure 12] FIG. 12 is a diagram showing an example of the determination result displayed on the display of the terminal device. [Figure 13] FIG. 13 is a flowchart showing a specific example of the process of step S180 shown in FIG. [Figure 14A] FIG. 14A is a diagram showing an example of a hair bundle region recognized from an image. [Figure 14B] FIG. 14B is a diagram showing an example in which a hair bundle region is divided into a plurality of partial regions. [Figure 14C] FIG. 14C is a diagram showing another example in which the hair bundle region is divided into a plurality of partial regions. [Figure 14D]FIG. 14D is a diagram showing yet another example in which the hair bundle region is divided into a plurality of partial regions. [Figure 15A] FIG. 15A is a diagram showing an example of an image in which a hair bundle region with a distorted shape has been extracted. [Figure 15B] FIG. 15B is a diagram showing an example of region division in an image in which a hair bundle region with a distorted shape has been extracted. [Figure 16A] FIG. 16A is a diagram for explaining a method of correcting the hair length based on the inclination. [Figure 16B] FIG. 16B is a diagram for explaining a method of correcting the hair length based on the inclination. [Figure 16C] FIG. 16C is a diagram for explaining a method of correcting the hair length based on the inclination. [Figure 17] FIG. 17 is a graph showing an example of hair length before and after correction. [Figure 18] FIG. 18 is a flowchart showing another specific example of the process of step S180 shown in FIG. [Figure 19] FIG. 19 is a diagram showing an example of a table showing the correspondence between the determination results and the display contents. [Figure 20] FIG. 20 is a diagram showing an example of a shooting instruction screen for acquiring an image that enables determination of the degree of opening of the hair bundle. [Figure 21] FIG. 21 is a diagram for explaining a method for estimating the inclination of the toothbrush about the Z axis from the reference posture. [Figure 22] FIG. 22 is a flowchart showing an example of a deterioration determination process based on the degree of opening of the hair bundle in addition to the hair length. [Diagram 23] FIG. 23 is a flow chart showing a specific example of the process of step S810 shown in FIG. [Figure 24A] FIG. 24A is a diagram for explaining side change amounts c1 and c2. [Figure 24B] FIG. 24B is a diagram for explaining side change amounts c1 and c2. [Diagram 25] FIG. 25 is a diagram showing an example of a table storing the values ​​of the thresholds Th1 to Th4. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, the embodiments will be described in detail with reference to the drawings. However, more detailed explanation than necessary may be omitted. For example, detailed explanation of already well-known matters or duplicate explanation of substantially the same configuration may be omitted. Note that the attached drawings and the following description are provided to enable those skilled in the art to fully understand the technology of the present disclosure, and are not intended to limit the subject matter described in the claims.

[0011] (composition) FIG. 1 is a diagram illustrating a terminal device 10 and a toothbrush 20 in this embodiment. FIG. 2 is a block diagram showing an example of the configuration of the terminal device 10. The terminal device 10 in this embodiment is a mobile terminal such as a smartphone or a tablet computer. As shown in FIG. 2, the terminal device 10 includes a state determination system 100, a camera 210, and a display 220. The camera 210 is an example of an imaging device that captures an image of an object. The display 220 includes a touch screen and functions as an input device and a display device. The state determination system 100 is a computer system that determines the state of the toothbrush 20 (e.g., the state of the degree of deterioration of the bristle bundle 22, etc.) based on an image of the toothbrush 20 captured by the camera 210.

[0012] Use of the toothbrush 20 may cause deterioration, such as wear at the tips of the bristles of the bristle bundle 22 or deformation of the sides of the bristle bundle 22. The condition determination system 100 has a function of estimating the degree of deterioration, for example, wear at the tips of the bristles or deformation of the bristle bundle 22, and determining whether the toothbrush 20 is in a condition suitable for use. The condition determination system 100 also has a function of determining whether the posture of the toothbrush 20 at the time of photographing is suitable for determination. The condition determination system 100 may also have a function of identifying the type of toothbrush 20 based on the image.

[0013] 1, toothbrush 20 includes a handle portion 25 extending in one direction, and a head portion 26 located at the tip of handle portion 25. Head portion 26 includes a bristle bundle 22 and a bristle base 21 that supports bristle bundle 22.

[0014] 1 illustrates a three-dimensional Cartesian coordinate system defined by mutually orthogonal X-axis, Y-axis, and Z-axis. The X-axis is set along the direction in which the handle portion 25 extends, the Y-axis is set along the direction in which the bristle bundle 22 extends, and the Z-axis is set in a direction orthogonal to the X-axis and Y-axis. In this specification, the positive or negative direction of the X-axis may be referred to as the "first direction," the positive or negative direction of the Z-axis may be referred to as the "second direction," and the positive or negative direction of the Y-axis may be referred to as the "third direction."

[0015] The bristle base 21 has a bristle implantation surface 21a extending in a first direction (i.e., a direction parallel to the X-axis) along the extension direction of the handle portion 25, and in a second direction (i.e., a direction parallel to the Z-axis) perpendicular to the first direction. The bristle bundles 22 extend from the bristle implantation surface 21a in a third direction (i.e., a direction parallel to the Y-axis) perpendicular to both the first and second directions.

[0016] The bristle implantation surface 21a has a plurality of holes, and a plurality of bristles are implanted in each hole. The number of holes, the arrangement of the holes, and the number of bristles implanted in each hole vary depending on the type of toothbrush 20. A collection of bristles implanted in the bristle implantation surface 21a is a bristle bundle 22. Of the bristle bundle 22, a group of bristles implanted in each hole is called a "partial bristle bundle." In other words, the bristle bundle 22 is a collection of a plurality of partial bristle bundles.

[0017] Toothbrush 20 may be a general non-electric toothbrush or an electric toothbrush. When toothbrush 20 is a non-electric toothbrush, a user holds handle 25 and slides head 26 in a direction such as the X direction or Z direction to brush teeth. When toothbrush 20 is an electric toothbrush, head 26 vibrates due to the action of a built-in electric motor, and the teeth are brushed by the vibration.

[0018] 2, the state determination system 100 includes a processing circuit 110, a storage device 120, a communication interface (I / F) 130, and an input / output interface (I / F) 140. The processing circuit 110, the storage device 120, the communication I / F 130, and the input / output I / F 140 are connected to each other via a bus.

[0019] The processing circuit 110 may be an integrated circuit including one or more processors such as a central processing unit (CPU). The processing circuit 110 may be realized by a field programmable gate array (FPGA) equipped with a CPU, a graphics processing unit (GPU), an application specific integrated circuit (ASIC), an application specific standard product (ASSP), or a combination of two or more circuits selected from these circuits.

[0020] The storage device 120 is a device including one or more recording media, such as a semiconductor storage medium, a magnetic recording medium, or an optical recording medium. The storage device 120 may include one or more memories, such as a read only memory (ROM) or a random access memory (RAM). The storage device 120 stores computer programs executed by the processor of the processing circuit 110, data referenced by the processing circuit 110 in the course of processing, and various data generated by the processing circuit 110. The processing circuit 110 can execute various operations, which will be described later, by executing the computer programs stored in the storage device 120. The storage device 120 may be a single storage or a collection of multiple storages.

[0021] The communication I / F 130 is an interface including a circuit for performing wireless communication. The state determination system 100 can communicate with an external device via the communication I / F 130. For example, the state determination system 100 can communicate with an external device such as a cloud server via the communication I / F 130.

[0022] Fig. 3 is a schematic diagram showing a configuration in which the terminal device 10 and the external device 40 communicate with each other via a network 30 such as the Internet. The external device 40 shown in Fig. 3 is a server computer on the cloud. By communicating with the external device 40, the terminal device 10 can update a program executed by the processing circuit 110 and update information such as a threshold value used in the determination process described later.

[0023] The input / output I / F 140 is an interface for transmitting and receiving signals between the state determination system 100 and devices such as the camera 210 and the display 220. The processing circuit 110 can acquire an image generated by the camera 210 via the input / output I / F 140. In this specification, the term "image" refers to digital image data or signals. The processing circuit 110 can also display information on the display 220 via the input / output I / F 140. The display 220 in this embodiment is equipped with a touch screen and functions as a display device and an input device. The processing circuit 110 receives a signal generated by a touch operation performed on the display 220 via the input / output I / F 140, and controls the display of the display 220 based on the signal.

[0024] (operation) Fig. 4 is a flowchart showing an example of the operation executed by the processing circuit 110. The processing circuit 110 executes the operations of steps S110 to S190 shown in Fig. 4 to determine the state, such as the degree of deterioration, of the toothbrush 20. The operation of each step will be described below.

[0025] In step S110, the processing circuit 110 causes a photographing instruction screen to be displayed on the display 220. The photographing instruction screen is a screen that displays information that prompts the user to photograph the head portion 26 of the toothbrush 20 from the side.

[0026] FIG. 5 is a diagram showing an example of a shooting instruction screen. The screen shown in FIG. 5 is displayed after the application installed in the terminal device 10 is started. This screen is a guide screen that prompts the user to shoot the head part 26 of the toothbrush 20 from the side toward the negative direction of the Z axis (i.e., the first direction). The shooting instruction screen shown in FIG. 5 includes a window 222 that displays a preview image to be shot by the camera 210, and a shooting button 224. A guide line 223 that is a frame for assisting the user in shooting is displayed in the window 222. The user points the camera 210 toward the head part 26 of the toothbrush 20 so that the head part 26 fits within the guide line 223 according to the displayed instructions. The processing circuit 110 may output information prompting the user to shoot the head part 26 of the toothbrush 20 from the side in a form different from the display (for example, audio). When the user presses the shooting button 224, shooting processing is executed, and image data is recorded in the storage device 120. Before the photographing button 224 is pressed, the processing circuit 110 executes the processes of steps S120 to S150.

[0027] In step S120, the processing circuit 110 recognizes an area corresponding to the head part 26 of the toothbrush 20 from an image frame of the preview image (hereinafter referred to as a "preview image" or simply an "image"). Specifically, the processing circuit 110 recognizes at least one of a hair bundle area corresponding to the hair bundle 22 in the head part 26 and a hair base area corresponding to the hair base 21 from the preview image. For the recognition of at least one of the hair bundle area and the hair base area, an algorithm based on machine learning, such as segmentation, can be used. By segmentation, each pixel in the image can be classified into categories such as "hair bundle", "hair base", and "background". In this recognition process, the processing circuit 110 can distinguish and detect the hair bundle area, the hair base area, and the background by inputting the preview image (or an image obtained by performing a predetermined preprocessing on the preview image) to a learned model that has been trained in advance and stored in the storage device 120.

[0028] The processing circuit 110 may be configured to not only extract the bristle bundle region and the bristle base region in the image, but also to identify the type of brush based on the image. Different types of brushes may have different shapes, heights, or widths of the bristle base 21 and the bristle bundle 22. The processing circuit 110 can identify the type of brush by inputting the image into a trained model for identifying the type of brush from the image of the head portion 26. Note that instead of the processing circuit 110 identifying the type of brush based on the image, the type of brush may be registered in advance by the user.

[0029] In step S130, the processing circuit 110 determines whether the posture of the toothbrush 20 is correct based on the recognition result of at least one of the bristle bundle region and the bristle base region in the image. For example, the processing circuit 110 can estimate the inclination of the toothbrush 20 from the reference posture based on the outline or dimensions of at least one of the bristle bundle region and the bristle base region in the image. Based on the estimated inclination, it can determine whether the posture of the toothbrush 20 is correct.

[0030] FIG. 6 shows an example of the reference posture of the toothbrush 20. As shown in FIG. 6, the reference posture is the posture of the toothbrush 20 when the camera 210 faces in the negative direction of the Z axis and captures an image of the head unit 26. In this embodiment, the posture of the toothbrush 20 when the horizontal direction of the image (including the preview image) generated by the camera 210 coincides with the X direction and the vertical direction of the image coincides with the Y direction is set as the reference posture. In this embodiment, the inclination according to the rotation angle around the X direction with respect to the reference posture of the toothbrush 20 is estimated. In this embodiment, as shown in FIG. 6, the direction of rotation when the bristles of the toothbrush 20 are inclined toward the user around the X axis is set as positive.

[0031] In step S130, the processing circuit 110 calculates an index value indicating the inclination of the toothbrush 20 from the reference posture based on the outline or dimensions of at least one of the bristle bundle region and the bristle base region in the image. If the index value is within a predetermined reference range, the processing circuit 110 determines that the posture of the toothbrush 20 is correct, and proceeds to step S150. Conversely, if the index value is outside the reference range, the processing circuit 110 determines that the posture of the toothbrush 20 is not correct, and proceeds to step S140.

[0032] The index value may be the rotation angle around the X axis itself, or may be another value that changes depending on the rotation angle. For example, the index value may be calculated based on the dimension of the tufting base area in the Y direction (third direction) in the image. In this specification, the dimension of an area or object in the Y direction in the image is referred to as the "height" of the area or object.

[0033] Fig. 7 is a diagram for explaining a method for estimating the inclination of the toothbrush 20 from a reference posture based on the dimensions in an image of the bristle base 21 and bristle bundle 22 of the toothbrush 20. As shown in Fig. 7, the width (i.e., the dimension in the X direction) of the bristle base region 323 corresponding to the bristle base 21 in the image is defined as x, and the height (i.e., the dimension in the Y direction) of the bristle base region 323 is defined as y. Also, the height (i.e., the dimension in the Y direction) of the bristle bundle region 324 corresponding to the bristle bundle 22 in the image is defined as α, and the width (i.e., the dimension in the X direction) of the bristle bundle region 324 is defined as β. Here, each of the dimensions x, y, α, and β can be expressed in units of, for example, the number of pixels in the image.

[0034] The height y of the bristle base region 323 and the height α of the bristle bundle region 324 in the image vary according to the magnitude of the rotation angle about the X-axis of the toothbrush 20. On the other hand, the width x of the bristle base region 323 and the width β of the bristle bundle region 324 hardly change with the rotation about the X-axis of the toothbrush 20. Therefore, the processing circuit 110 can evaluate the inclination of the toothbrush 20 using y, α, the ratio of y to β, the ratio of y to x, the ratio of α to β, or the ratio of α to x as index values.

[0035] The processing circuit 110 can obtain the dimensions x, y of the hair implantation base area 323 and the dimensions α, β of the hair bundle area 324 in the image based on the image (preview image in this embodiment) captured by the camera 210. For example, the processing circuit 110 first rotates the acquired image so that the X direction, which is the longitudinal direction of the hair implantation base, coincides with the horizontal direction of the image. Then, the processing circuit 110 determines the height of the recognized hair implantation base area 323 as y, the width of the hair implantation base area 323 as x, the height of the hair bundle area 324 as α, and the width of the hair bundle area 324 as β.

[0036] FIG. 8 shows an example of an image of the head portion 26 of the toothbrush 20. The image shown in FIG. 8 shows an example in which the rotation angle of the toothbrush 20 around the X-axis is about -10°. In this example, the bristle base region 323 is thicker toward the center and thinner toward the end. The processing circuit 110 can process, for example, the thickness of the thickest central portion of the bristle base region 323 or the average value of the thickness of each portion of the bristle base region 323 as y. Similarly, the processing circuit 110 can process the height of the central portion of the bristle bundle region 324 or the average value of the height of each portion of the bristle bundle region 324 as α. For the width x of the bristle base region 323 and the width β of the bristle bundle region 324, the recognized dimensions in the X direction of the bristle base region 323 and the bristle bundle region 324 can be used as they are.

[0037] The processing circuit 110 may calculate, for example, a ratio β / y between the width β of the bristle bundle region 324 and the height y of the bristle base region 323, or a ratio x / y between the width x of the bristle base region 323 and the height y of the bristle base region 323, as an index value. In this case, the processing circuit 110 may be configured to output a warning indicating that the posture of the toothbrush 20 is not correct when the ratio β / y or x / y is outside a predetermined reference range. In this way, the index value may be calculated based on the ratio between the dimension of the bristle bundle region 324 or the bristle base region 323 in the x direction (i.e., the first direction) in the image and the dimension of the bristle base region 323 in the y direction (i.e., the third direction).

[0038] FIG. 9 is a diagram showing an example of a reference range of an index value. In the example shown in FIG. 9, a reference range of the index value β / y is determined in advance for each type of brush and stored in the storage device 120. For example, a range of more than 4.4 is set as the reference range when the type of brush is "Type 1" (normal). A range of more than 4.2 is set as the reference range when the type of brush is "Type 2" (ultra-fine). A range of more than 4.6 is set as the reference range when the type of brush is "Type 3" (long bristles). In the example of FIG. 9, only the lower limit of the reference range is set, but an upper limit may also be set. When only the lower limit of the reference range is set as in the example of FIG. 9, the lower limit is treated as a threshold value, and when the index value β / y exceeds the threshold value, the posture of the toothbrush 20 is determined to be appropriate.

[0039] FIG. 10 is a graph showing an example of the relationship between the index value β / y and the angle of rotation around the X-axis. The graph shown in FIG. 10 corresponds to the brush of "Type 3" shown in FIG. 9. As shown in this graph, there is a correlation between the index value β / y and the angle of rotation around the X-axis. In this example, the processing circuit 110 reads out a reference range (or threshold) corresponding to the type of brush photographed from the storage device 120, and compares the index value β / y calculated based on the image with the threshold value 4.6. If the index value β / y is greater than the threshold value 4.6, the processing circuit 110 determines that the posture of the toothbrush 20 is appropriate. Conversely, if the index value β / y is equal to or less than the threshold value 4.6, the processing circuit 110 determines that the posture of the toothbrush 20 is not appropriate.

[0040] The index value is not limited to β / y, and may be, for example, x / y, β / α, x / α, or the reciprocal thereof. In this embodiment, as shown in FIG. 5, the image of the head unit 26 is captured so as to fit within the guideline 223 according to the displayed instructions. For this reason, the variation in the distance between the camera 210 and the head unit 26 is small, and the variation in the value of y or α in the image caused by the variation in distance is also small. Therefore, y or α itself may be used as the index value. Alternatively, a value obtained by dividing the actual dimension of the hair implantation base 21 in the X direction by the dimension x in the X direction of the hair implantation base area 323 in the image and multiplying the value by y or α may be used as the index value. Similarly, a value obtained by dividing the actual dimension of the hair bundle 22 in the X direction by the dimension β in the X direction of the hair bundle area 324 in the image and multiplying the value by y or α may be used as the index value. By such processing, the index value according to the rotation angle around the X axis can be appropriately determined regardless of the distance between the camera 210 and the head unit 26. Here, the actual dimensions in the X direction of the bristle base 21 or bristle bundle 22 for each type of toothbrush 20 are known, and the values ​​can be stored in advance in the storage device 120. Whichever index value is used, a reference range (or threshold) of the index value is recorded in advance in the storage device 120, and the inclination can be evaluated based on a comparison between the reference range and the index value. The reference range of the index value can be set, for example, so that the range converted into the rotation angle around the X axis of the toothbrush 20 is narrower than ±10°.

[0041] In the above example, the inclination of toothbrush 20 from the reference posture is estimated based on the dimensions of at least one of bristle bundle region 324 and bristle base region 323 in the image, but other methods may be used. For example, the inclination of toothbrush 20 from the reference posture may be estimated based on the contours of at least one of bristle bundle region 324 and bristle base region 323 in the image.

[0042] 11A to 11C are diagrams showing an example of a method for estimating the inclination of the toothbrush 20 based on the contour of the bristle bundle region 324. FIG. 11A shows an example of an image of the head unit 26 when the rotation angle around the X axis is about -10°. FIG. 11B shows an example of an image of the head unit 26 when the rotation angle around the X axis is about 0°. FIG. 11C shows an example of an image of the head unit 26 when the rotation angle around the X axis is about 10°. In these figures, the recognized bristle bundle region 324 is shown by a dotted line. In addition, a line segment connecting two points P1 and P2 at both ends of the boundary 322 between the bristle bundle region 324 and the bristle base region 323 is shown by a dashed line.

[0043] In this example, the processing circuit 110 recognizes the boundary 322 between the bristle bundle region 324 and the bristle base region 323 from the image. The processing circuit 110 can estimate the inclination around the X-axis direction of the toothbrush 20 based on the shape of the boundary 322 between the bristle bundle region 324 and the bristle base region 323, or the positions of at least three points on the boundary 322. For example, the processing circuit 110 can determine the inclination based on an index value calculated based on the positions of at least three points on the boundary 322 and information prestored in a storage device that specifies the correspondence between the index value and the inclination. The at least three points on the boundary 322 may include, for example, a first point P1 and a second point P2 at both ends of the boundary 322, and a third point P3 located in the center of the boundary 322. The index value may be a value based on the distance d between the line segment connecting the first point P1 and the second point P2 and the third point P3. The distance d is the distance between the tips of the two opposing white and black arrows shown in each of Figs. 11A and 11C. The ratio of this distance d to the distance between the first point P1 and the second point P2 can be used as an index value. In the example of Fig. 11A, the third point P3 is located at the top of the boundary 322 having an upwardly convex shape. In the example of Fig. 11C, the third point P3 is located at the bottom of the boundary 322 having a downwardly convex shape. The third point P3 can be the point on the boundary 322 that is the furthest from the line segment connecting the first point P1 and the second point P2.

[0044] The distance d between point P3 and the line segment connecting points P1 and P2 increases as the rotation angle of toothbrush 20 around the X-axis increases. Therefore, by using an index value corresponding to distance d, it is possible to evaluate the inclination of toothbrush 20 from the reference posture, as in the above example. For example, when the index value proportional to distance d is greater than a predetermined threshold, processing circuit 110 can determine that the posture of toothbrush 20 is not proper, and when the index value proportional to distance d is equal to or less than the threshold, can determine that the posture of toothbrush 20 is proper.

[0045] The processing circuit 110 may recognize the boundary between the bristle bundle region 324 and the background, or the boundary between the bristle bundle region 323 and the background, instead of the boundary between the bristle bundle region 324 and the bristle bundle region 323, and estimate the inclination of the toothbrush 20 based on the shape of the boundary or the positions of at least three points on the boundary. The same processing as above can be applied to the boundary between the bristle bundle region 324 and the background, and the boundary between the bristle bundle region 323 and the background, since their shapes change depending on the inclination of the toothbrush 20.

[0046] The process of determining whether the posture of the toothbrush 20 is appropriate is not limited to the above example, and may be performed based on a learned model that has been trained in advance using machine learning such as a deep neural network (DNN). As described above, the dimensions and shapes of the bristle bundle region 324 and the bristle base region 323 in the image change depending on the inclination of the toothbrush 20 from the reference posture. Therefore, for example, based on an image of the toothbrush 20, based on a learned model that estimates the inclination of the toothbrush 20 from the reference posture or whether the posture is appropriate, it is possible to estimate the inclination of the toothbrush 20 from the reference posture or to estimate whether the posture of the toothbrush 20 is appropriate.

[0047] If it is determined in step S130 shown in Fig. 4 that the posture of the toothbrush 20 is not correct, the process proceeds to step S140. In step S140, the processing circuit 110 causes the display 220 to display a warning indicating that the posture of the toothbrush 20 is not correct. For example, the processing circuit 110 changes the color of the guideline 223 shown in Fig. 5 to a color indicating a warning, such as red, and sets the photographing button 224 to an inactive state. This makes it possible to inform the user that the posture of the toothbrush 20 is not correct. At this time, the processing circuit 110 may display information such as a message indicating that the posture of the toothbrush 20 is not correct as a warning.

[0048] The processing circuit 110 may output the warning by other methods, not limited to display. The output of the warning may include controlling at least one of a display device (e.g., the display 220), a light-emitting device, a speaker, and a haptic device to provide at least one of a visual stimulus, an auditory stimulus, and a tactile stimulus indicating the warning to the user. At least one of such a display device, a light-emitting device, a speaker, and a haptic device may be built into the terminal device 10, or may be an external device to the terminal device 10. By providing a warning to the user by at least one of a visual stimulus, an auditory stimulus, and a tactile stimulus, the user can be effectively informed that the posture of the toothbrush 20 at the time of photographing is not appropriate.

[0049] If it is determined in step S130 that the posture of the toothbrush 20 is correct, the process proceeds to step S150. In step S150, the processing circuit 110 activates the photographing button 224 to enable the user to perform a photographing operation. At this time, the processing circuit 110 changes the color of the guide line 223 to a color indicating normality, such as green. This allows the user to recognize that the posture of the toothbrush 20 is correct and tap the photographing button 224.

[0050] In this way, the processing circuit 110 in this embodiment changes the content of the notification to the user depending on whether or not the index value indicating the degree of inclination from the reference posture of the toothbrush 20 at the time of shooting is outside a predetermined reference range. The processing circuit 110 issues a notification indicating a warning when the index value is outside the reference range, and issues a notification indicating conformity or does not issue a notification indicating a warning when the index value is within the reference range. The notification indicating a warning can be issued, for example, by changing the color of the guideline 223 to a color indicating a warning such as red, displaying a warning message, or issuing a sound indicating a warning. The notification indicating conformity can be issued, for example, by changing the color of the guideline 223 to a color indicating conformity such as green or yellow, displaying a message indicating that the posture is proper, or issuing a sound indicating that the posture is proper. This allows the user to know whether the posture of the toothbrush 20 at the time of shooting is proper or not.

[0051] In step S160, the processing circuit 110 determines whether or not a user has performed a shooting operation. That is, the processing circuit 110 determines whether or not the user has tapped the shooting button 224. If a shooting operation has been performed, the process proceeds to step S170. If a shooting operation has not been performed, the process returns to step S120. The processes from step S120 to S160 are repeated until a shooting operation is performed by the user.

[0052] In step S170, in response to a photographing operation by the user, processing circuitry 110 sends a command to camera 210 to generate an image (i.e., a still image). In response to the command, camera 210 drives an image sensor to generate an image of head portion 26 of toothbrush 20. Processing circuitry 110 stores the image generated by camera 210 in storage device 120. This allows an image of toothbrush 20 in a position that is not significantly tilted from the reference position to be obtained.

[0053] In this embodiment, the image is acquired in response to the user's photographing operation, but the photographing may be performed automatically. For example, after it is determined in step S130 that the toothbrush posture is correct (Yes), the processing circuit 110 may send a command to the camera 210 to generate an image without going through the processes of steps S150 and S160.

[0054] After step S170, the process proceeds to step S180. In step S180, the processing circuit 110 determines the deterioration state of the bristle bundle 22 of the toothbrush 20 based on the acquired image. For example, the processing circuit 110 estimates the bristle length of the toothbrush 20 based on the image, and determines the degree of deterioration of the toothbrush 20 based on the estimated bristle length. In this embodiment, if the deterioration state can be determined, the process proceeds to step S190, and if the deterioration state cannot be determined, the process returns to step S120. Details of the process of step S180 will be described later.

[0055] In step S190, processing circuitry 110 outputs the determination result. The determination result includes information regarding deterioration of toothbrush 20. For example, processing circuitry 110 causes display 220 to display information such as the extent to which toothbrush 20 has deteriorated or whether or not toothbrush 20 needs to be replaced. The output of the determination result is not limited to display, and may be notified to the user by a method such as light, sound, or vibration.

[0056] FIG. 12 is a diagram showing an example of the display of the judgment result displayed on the display 220 of the terminal device 10. In the example shown in FIG. 12, a message saying "The brush is deteriorated. We recommend replacing it" is displayed as the judgment result. In this way, the processing circuit 110 causes the display 220 to display information on the deterioration of the toothbrush 20 (for example, information indicating the degree of deterioration). The user can recognize the state of the toothbrush 20 based on the displayed information. In the example of FIG. 12, the phrase "ultra-fine bristle large" indicating the type of brush is also displayed. In addition to the name of the type, information such as a product number or a link to the website of the brush may be displayed. By displaying the information indicating the type of toothbrush 20, the user can clearly understand the type of toothbrush 20 that he or she is using. This allows the user to replace the toothbrush 20 with the same toothbrush 20 that was used until then without fail.

[0057] Next, a specific example of the process of step S180 shown in FIG. 4 will be described.

[0058] Fig. 13 is a flow chart showing a specific example of the process of step S180. In this example, processing circuitry 110 estimates the bristle length of toothbrush 20 by executing the process shown in Fig. 13, and determines the deterioration state of toothbrush 20 based on the bristle length. Note that processing circuitry 110 may determine the deterioration state of toothbrush 20 by further taking into consideration not only the bristle length of toothbrush 20, but also the degree to which bristle bundles 22 are open. Such an example will be described later.

[0059] In step S181, the processing circuit 110 recognizes the hair bundle region from the image acquired in step S170 and divides the hair bundle region into a plurality of partial regions. The recognition may be performed according to a segmentation algorithm such as semantic segmentation. The processing circuit 110 performs segmentation on the image to extract the hair bundle region as a closed region in the image, for example, as shown by the dotted frame in Figs. 11A to 11C. However, in some cases, the segmentation may not be performed accurately, and the hair bundle region may not be extracted correctly. Therefore, in this embodiment, the processing circuit 110 divides the recognized hair bundle region into a plurality of partial regions arranged in a direction (for example, X direction) intersecting the direction in which the hair bundle extends (i.e., Z direction), obtains the length of the hair bundle for each partial region, and estimates the hair length more accurately based on these lengths. Here, the "length of the hair bundle" means the dimension of the region occupied by the hair bundle in the direction in which the hair bundle extends in the image. The "length of the hair bundle" may also be called the "height of the hair bundle."

[0060] Fig. 14A is a diagram showing an example of a hair bundle region 324 recognized from an image. Fig. 14B is a diagram showing an example in which the hair bundle region 324 is divided into a plurality of partial regions 326. Fig. 14C is a diagram showing another example in which the hair bundle region 324 is divided into a plurality of partial regions 326.

[0061] In Fig. 14A, the outline of the hair bundle region 324 is represented by a white line. In Fig. 14B and Fig. 14C, the multiple partial regions 326 are represented by rectangular black dotted frames. In the example of Fig. 14B, the multiple partial regions 326 are adjacent to each other in the X direction (i.e., without any gaps between them). In the example of Fig. 14C, the multiple partial regions 326 are discretely arranged in the X direction (i.e., with gaps between them). In the present disclosure, associating a part of the hair bundle region 324 with multiple discrete partial regions 326, as in the example of Fig. 14C, is also interpreted as "dividing the hair bundle region into multiple partial regions."

[0062] As described above, the hair bundle 22 includes a plurality of partial hair bundles. Each partial hair bundle is a group of hairs planted in a corresponding hole provided in the hair planting base 21. Each partial hair bundle has a predetermined width. The "width" of a partial hair bundle corresponds to the width (e.g., about 1 mm) in the first direction (X direction) of the hole in which the partial hair bundle is planted. The width (i.e., the dimension in the X direction) of each of the plurality of partial regions 326 may be set to a value approximately equal to the width of each of the plurality of partial hair bundles in the image. The processing circuit 110 may set the plurality of partial regions 326 so that each partial region surrounds at least the root portion of a corresponding partial hair bundle. The height (i.e., the dimension in the Y direction) of each partial region 326 may be set to a value equal to or greater than the length in the Y direction of the longest hair included in that partial region 326, for example.

[0063] In the example shown in Figures 14B and 14C, each of the partial regions 326 has a rectangular shape, but the shape of each partial region is not limited to a rectangle. For example, as shown in Figure 14D, the processing circuit 110 may divide the hair bundle region 324 into a plurality of partial regions 326 by a plurality of division lines 325 extending in the Y direction arranged at a predetermined interval along the X direction. In this case, the shape of each partial region 326 is different from a rectangle. The widths of the partial regions 326 do not need to be constant and may be different from each other.

[0064] As described above, the processing circuitry 110 may identify the type of toothbrush 20 based on the acquired image. Alternatively, the processing circuitry 110 may acquire information indicating the type of toothbrush 20 identified by other methods (e.g., input from a user). The processing circuitry 110 may set the number of divisions into the plurality of partial regions 326 based on the type of toothbrush 20. The number of partial bristle bundles (i.e., the number of holes) may differ depending on the type of toothbrush 20. Therefore, when dividing the toothbrush 20 so that one partial region 326 corresponds to one partial bristle bundle, the number of divisions into the plurality of partial regions 326 is set according to the type of toothbrush 20.

[0065] The processing circuitry 110 may set the number of divisions into multiple partial regions 326 based on the shape of the hair bundle region 324. For example, the number of divisions may be adjusted according to the ratio of the X-direction dimension to the Y-direction dimension of the hair bundle region 324. Alternatively, if the hair bundle region 324 is extracted as a region with a distorted shape, the number of divisions may be adjusted so that partial regions 326 with extremely small heights are not included. Such adjustments make it easier to determine the height of the hair bundle region 324 more accurately.

[0066] In step S182, the processing circuit 110 determines the height of the hair bundle region 324 based on the length (i.e., height) of the hair bundle in each of the multiple partial regions 326. For example, the processing circuit 110 may be configured to determine the height of the hair bundle region 324 by the following process. First, the processing circuit 110 obtains the maximum value of the hair bundle length included in each of the multiple partial regions 326 for each partial region 326. Specifically, the processing circuit 110 obtains the hair length (i.e., the dimension in the Y direction) for each position in the X direction in each partial region 326, and sets the obtained maximum value of the hair length as the maximum value of the hair bundle length in that partial region 326. Next, the processing circuit 110 determines the minimum length of the maximum values ​​of the hair bundle lengths in the multiple partial regions 326 as the length (or hair length) of the hair bundle region 324. By such a process, the length of the hair bundle in, for example, the partial region 326 where wear has progressed the most among the multiple partial regions 326 can be determined as the length of the hair bundle region 324.

[0067] In step S183, processing circuit 110 determines whether the height of the bristle bundle region determined in step S182, i.e., the minimum length among the maximum values ​​of the bristle bundle lengths in the multiple partial regions 326, is within a predetermined range. If the length is within the predetermined range, proceed to step S185. If the length is outside the predetermined range, proceed to step S184. The predetermined range may be defined by a lower limit value and an upper limit value of the length that are preset for each type of toothbrush 20. The lower limit value and the upper limit value may be recorded in memory device 120 in advance. If the length is smaller than the lower limit value or larger than the upper limit value of the predetermined range, proceed to step S184.

[0068] The lower and upper limits of the predetermined range are set based on the expected range of bristle lengths of toothbrushes. The lower limit may be set to a value of, for example, about 70% to 80% of the bristle length of a new brush. The upper limit may be set to a value of, for example, about 120% to 130% of the bristle length of a new brush.

[0069] The predetermined range may be determined by only one of the lower limit and the upper limit. That is, the range equal to or greater than the lower limit may be determined as the predetermined range, or the range equal to or less than the upper limit. When only the lower limit is determined, if the height of the hair bundle region determined in step S182 is smaller than the lower limit, the process proceeds to step S184. On the other hand, when only the upper limit is determined, if the height of the hair bundle region determined in step S182 is greater than the upper limit, the process proceeds to step S184.

[0070] In step S184, the processing circuit 110 displays a warning including information urging the user to retake the image on the display 220. The processing circuit 110 may output information urging the user to retake the image in another form such as sound, not limited to display. After step S184, the process returns to step S120, and the head unit 26 is recognized again.

[0071] By the processing of steps S183 and S184, if the height of the bristle bundle region determined in step S182 deviates from the expected range, the user can be prompted to re-take the photograph so as to re-obtain an image suitable for determining the state of toothbrush 20.

[0072] If it is determined in step S183 that the height of the bristle bundle region is not within the predetermined range (No), the process may return to step S170 without performing the process of step S184. That is, the processing circuit 110 may automatically perform the photographing and recognition processes without the user's operation. In other words, the processing circuit 110 may perform image acquisition and recognition of the bristle bundle region again when the minimum length among the maximum values ​​of the bristle bundle lengths in the multiple partial regions 326 is outside the predetermined range. This allows an image suitable for determining the state of the toothbrush 20 to be automatically reacquired without the user performing an operation to re-photograph the image.

[0073] Also, if it is determined in step S183 that the height of the hair bundle region is not within a predetermined range (No), the recognition process may be performed again on the same image without performing the process of step S184. That is, processing circuitry 110 may perform recognition of the hair bundle region again if the minimum length among the maximum values ​​of the hair bundle lengths in the multiple partial regions 326 is outside the predetermined range. Even when the same image is the target, the hair bundle region may be correctly recognized by changing the image recognition algorithm, adjusting the parameters of the image recognition algorithm, or adjusting the brightness of the image. In this way, if processing circuitry 110 determines No in step S183, it may perform processing such as changing or adjusting the algorithm, or adjusting the brightness of the image, and then perform recognition of the hair bundle region again.

[0074] Instead of performing the processes of steps S183 and S184, the processing circuit 110 may perform the following process in step S182. First, the processing circuit 110 determines the maximum value of the hair bundle length included in each of the multiple partial regions 326 for each partial region 326, as in the above example. Next, the processing circuit 110 excludes one or more partial regions from the multiple partial regions 326 whose maximum hair bundle length value is outside a predetermined range. The predetermined range may be specified by at least one of a lower limit value and an upper limit value as described above. This excludes partial regions whose maximum hair bundle length value is an abnormal value. Next, the processing circuit 110 determines the minimum length of the maximum hair bundle lengths in the partial regions that have not been excluded from among the multiple partial regions as the length (or hair length) of the hair bundle region.

[0075] Alternatively, the processing circuit 110 may determine the height of the hair bundle region by the following process. First, the processing circuit 110 excludes one or more partial regions 326 that have heights outside a predetermined range from among the multiple partial regions 326. In this example, partial regions with abnormal height values ​​are excluded based on the heights of each partial region 326 determined in the division process. Next, the processing circuit 110 determines, for each partial region, the maximum value of the hair bundle length contained in each partial region that has not been excluded from among the multiple partial regions. Furthermore, the processing circuit 110 determines the minimum length of the maximum hair bundle lengths in the partial regions that have not been excluded as the height (or hair length) of the hair bundle region.

[0076] When determining the height of a hair bundle region by excluding partial regions having heights that are not normally considered in this way, the processes of steps S183 and S184 are not performed, and the process proceeds to step S185 after step S182. This process makes it possible to prevent the length of the hair bundle in an extremely low partial region from being processed as the hair length, for example, when an extremely low partial region is set due to an inaccurate segmentation resulting in the extraction of a hair bundle region with a distorted shape.

[0077] 15A and 15B are diagrams showing examples of images in which a hair bundle region 324 with a distorted shape has been extracted. The image shown in FIG. 15A is blurred, and as a result, the contour of the hair bundle region 324 is not accurately extracted. If such a hair bundle region 324 is divided into a plurality of partial regions as shown in FIG. 15B, a divided region with an extremely small height may be set. Therefore, it is effective to set a threshold value Th of, for example, about 70% to 80% of the expected hair length, and exclude divided regions having a height smaller than the threshold value Th. In the example of FIG. 15B, the two partial regions 326a and 326b located on the left side are excluded, and the height of the hair bundle region 324 is determined based on the length of the hair bundle in the remaining partial regions.

[0078] In step S185 shown in Fig. 13, the processing circuit 110 estimates the inclination of the toothbrush 20 from the reference posture at the time of photographing. This process is performed in the same manner as that described with reference to Figs. 11A to 11C. That is, the processing circuit 110 estimates the inclination of the toothbrush 20 from the reference posture at the time of photographing based on the outline of at least one of the bristle bundle region and the bristle base region recognized from the image acquired in step S170. More specifically, the processing circuit 110 estimates the inclination based on the shape of the boundary between the bristle bundle region and the bristle base region, or the positions of at least three points on the boundary.

[0079] In the next step S186, the processing circuit 110 corrects the height of the bristle bundle region based on the estimated tilt and calculates the bristle length of the toothbrush 20. This makes it possible to calculate the bristle length from which the influence of the tilt of the toothbrush 20 at the time of photographing has been removed.

[0080] In this embodiment, the toothbrush 20 is photographed with a small inclination from the reference posture by the process of step S130 shown in FIG. 4. However, even so, an angle deviation of, for example, about 5 degrees to 10 degrees may occur. If the posture of the toothbrush 20 at the time of photographing is inclined from the reference posture, a difference occurs between the height of the bristle bundle region determined in step S182 and the actual bristle length. Therefore, in this embodiment, the processing circuit 110 calculates the final bristle length by correcting the height of the bristle bundle region determined in step S182 based on the inclination from the reference posture of the toothbrush 20. That is, the processing circuit 110 estimates the bristle length of the toothbrush 20 based on the inclination estimated in step S185 and the height of the bristle bundle region determined in step S182. By such a process, the influence of the inclination at the time of photographing the toothbrush 20 can be removed, and the bristle length can be estimated more accurately. The process of step S185 may be performed before the processes of steps S181 and S182.

[0081] 16A to 16C are diagrams for explaining a method of correcting bristle length based on inclination. Here, the angle of rotation around the X-axis with respect to the reference posture of the toothbrush 20 is θ, the actual bristle length is a0, and the height of the bristle bundle region in the image (i.e., the apparent bristle length) is a. The angle θ takes a positive value when rotating clockwise toward the positive direction of the X-axis as shown in FIG. 6. FIG. 16A is a schematic diagram of an image of the toothbrush 20 when θ<0 and the bristle bundle 22 and the bristle implantation base 21 viewed from the positive direction of the X-axis. FIG. 16B is a schematic diagram of an image of the toothbrush 20 when θ=0 and the bristle bundle 22 and the bristle implantation base 21 viewed from the positive direction of the X-axis. FIG. 16C is a schematic diagram of an image of the toothbrush 20 when θ>0 and the bristle bundle 22 and the bristle implantation base 21 viewed from the positive direction of the X-axis.

[0082] As shown in FIG. 16A, when θ<0, the hair length a0 can be calculated by a0=(a+b) / cosθ. Here, b represents the height of the part of the hair bundle 22 located at the forefront hidden behind the hair implantation base 21 in the image. For example, as shown in FIG. 11A, b can be determined based on the distance d between the line segment connecting two points P1 and P2 at both ends of the boundary between the hair bundle region 324 and the hair implantation base region 323 and the center point P3 on the boundary. The apparent hair length a is the height of the hair bundle region determined in step S182. The angle θ is the angle of inclination estimated in step S185. As described above, the angle θ can also be determined based on the distance d. Information such as a table or function that specifies the correspondence between the distance d, the angle θ, and the value b can be stored in advance in the storage device 120. The processing circuit 110 can refer to this information to determine the angle θ and the value b from the distance d, and calculate the hair length a0 by the calculation of a0=(a+b) / cos θ.

[0083] 16B, when θ=0, the hair length a0 coincides with the apparent hair length a in the image. In this case, the processing circuit 110 can process the apparent hair length a as the actual hair length a0 without correcting it.

[0084] As shown in FIG. 16C, when θ>0, the hair length a0 can be calculated by a0=(af) / cosθ. Here, f represents the difference in height between the foremost hair and the innermost hair in the image. f increases as the angle θ increases. Information such as a table or function that defines the relationship between the angle θ and the value f can be stored in advance in the storage device 120. The processing circuit 110 can calculate the value f based on the information and calculate the hair length a0 by a0=(af) / cosθ.

[0085] The processing circuit 110 may process the value obtained by multiplying the above a0 by a proportionality coefficient for converting the length in the image (e.g., in pixels) to an actual length (e.g., in millimeters) as the hair length a0. The proportionality coefficient can be obtained, for example, by dividing the actual dimension in the X direction of the hair bundle 22, which is a known value, by the corresponding dimension β in the image (see FIG. 8). Alternatively, the proportionality coefficient can be obtained by dividing the actual dimension in the X direction of the hair implantation base 21 by the corresponding dimension x in the image.

[0086] Fig. 17 is a graph showing an example of hair length before and after correction. The solid line shows an example of the relationship between the height of the hair bundle region detected from the image, i.e., the apparent hair length before correction, and the angle of rotation around the X axis. The dashed line shows an example of the hair length corrected by the processing of this embodiment. As shown in this graph, the larger the absolute value of the rotation angle, the more the hair length is corrected, and a value closer to the actual hair length is obtained.

[0087] In the example of FIG. 13, the determination of step S183 is performed between step S182 and step S185, but a similar determination may be performed after step S186. FIG. 18 is a flowchart showing such an embodiment. In the example of FIG. 18, after step S182, the processes of steps S185 and S186 are performed, and then the process of step S183a is performed. In step S183a, the processing circuit 110 determines whether the bristle length a0 calculated in step S186 is within a predetermined range. If the bristle length a0 is within the predetermined range, the process proceeds to step S187. If the bristle length a0 is outside the predetermined range, the process proceeds to step S184. The predetermined range may be defined by a lower limit value and an upper limit value of the height that are preset for each type of toothbrush 20. The lower limit value and the upper limit value may be recorded in the storage device 120 in advance. If the bristle length a0 is smaller than the lower limit value or larger than the upper limit value of the predetermined range, the process proceeds to step S184. The process of step S184 is the same as the process in the example of FIG. 13. In the example of FIG. 18, the process of step S184 may not be performed, and the hair bundle region may be recognized again based on the same image or a re-captured image.

[0088] In step S187, the processing circuit 110 compares the length a0 with a first threshold value Th1 pre-stored in the storage device 120. If a0 < Th1, the process proceeds to step S189a. If a0 ≥ Th1, the process proceeds to step S188.

[0089] In step S188, the processing circuit 110 compares the length a0 with a second threshold value Th2 pre-stored in the storage device 120. The second threshold value Th2 is greater than the first threshold value Th1. If a0 < Th2, the process proceeds to step S189b. If a0 ≥ Th2, the process proceeds to step S189c.

[0090] In step S189a, the processing circuit 110 assigns the value R1 to the determination result. In step S189b, the processing circuit 110 assigns the value R2 to the determination result. In step S189c, the processing circuit 110 assigns the value R3 to the determination result. The values R1, R2, and R3 are associated with the display contents.

[0091] FIG. 19 shows an example of a table indicating the correspondence between the determination result and the display contents. Such a table is pre-stored in the storage device 120. The determination result R1 corresponds to the display content "The brush is deteriorated. It is recommended to replace it." The determination result R2 corresponds to the display content "It is about time to replace it." The determination result R3 corresponds to the display content "It can still be used." The processing circuit 110 can display, on the display 220, a message corresponding to the determination result by referring to a table as shown in FIG. 19. For example, when the determination result is R1, a screen as shown in FIG. 12 is displayed.

[0092] As described above, according to this embodiment, the inclination of the toothbrush 20 from the reference posture at the time of photographing can be estimated based on the outline or dimensions of at least one of the bristle bundle region and the bristle base region recognized from the image of the head portion 26 of the toothbrush 20. This makes it possible to determine whether the posture of the toothbrush 20 at the time of photographing is appropriate or not. In addition, the content of the notification to the user can be changed depending on whether the index value indicating the degree of inclination from the reference posture of the toothbrush 20 at the time of photographing is outside the reference range. For example, a notification indicating a warning can be issued when the index value is outside the reference range. In addition, a notification indicating conformity can be issued when the index value is within the reference range. This makes it possible to inform the user that the posture of the toothbrush 20 at the time of photographing is not appropriate and to encourage the user to photograph the toothbrush 20 in the correct posture.

[0093] In addition, the bristle length of the toothbrush 20 can be estimated based on the estimated inclination of the toothbrush 20 and the height of the bristle bundle region determined by image recognition processing. This makes it possible to determine the degree of deterioration of the toothbrush 20 based on the estimated bristle length.

[0094] Furthermore, the bristle region in the image of the toothbrush 20 can be divided into a plurality of partial regions, and the bristle length of the toothbrush 20 can be estimated based on the length of the bristle in each partial region. This makes it possible to estimate the bristle length of the toothbrush 20 more accurately.

[0095] (Other embodiments) In the above embodiment, in the operations of steps S110 to S170 shown in Fig. 4, the user is notified of whether or not the posture of toothbrush 20 is correct based on the preview image before the user performs the photographing operation, but this function may be omitted. In that case, after the user performs the photographing operation, whether or not the posture of toothbrush 20 is correct may be determined based on the acquired image by the same process as described above, and if the posture is not correct, information may be output that prompts the user to photograph again.

[0096] In the above embodiment, in steps S181 and S182 shown in Figures 13 and 18, the hair bundle region recognized from the image is divided into a plurality of partial regions, the hair bundle length is obtained for each partial region, and the height of the hair bundle region in the image, i.e., the apparent hair length, is determined based on the hair bundle lengths. This process may be omitted and the apparent hair length may be determined directly from the recognized hair bundle region. In that case, for example, the hair length for each position in the X-axis direction in the hair bundle region may be obtained by image recognition, and the minimum, maximum, average, or median value of the hair lengths may be determined as the height of the hair bundle region.

[0097] The processes of steps S185 and S186 shown in Fig. 13 and Fig. 18 are executed as necessary. For example, by strictly setting the reference range of the determination process in step S130 shown in Fig. 4, if an image in which the inclination of the toothbrush 20 from the reference posture is sufficiently small is captured, the correction processes of steps S185 and S186 may be omitted. In that case, the height of the bristle bundle region determined in step S182 may be treated as the bristle length a0.

[0098] 13 and 18, a two-stage determination is performed in which the bristle length a0 is compared with two thresholds Th1 and Th2, but this is merely an example. For example, the bristle length a0 may be compared with one threshold to determine whether or not the toothbrush 20 needs to be replaced. Alternatively, the bristle length may be compared with three or more thresholds to determine the degree of deterioration of the toothbrush 20 in more detail.

[0099] In the above embodiment, as shown in FIG. 6, the posture of the toothbrush 20 when the camera 210 photographs in the negative direction of the Z axis (i.e., the second direction) is set as a reference posture, and the tilt associated with the rotation around the X axis is estimated. Instead of or in addition to this process, the posture of the toothbrush 20 when the camera 210 photographs in the negative direction of the X axis (i.e., the first direction) may be set as a reference posture, and the tilt associated with the rotation around the Z axis may be estimated. By photographing the toothbrush 20 in such a posture, an image that allows the degree of opening of the bristle bundle to be determined can be acquired. In this case, the processing circuit 110 can determine the deterioration state of the toothbrush 20 by taking into account not only the bristle length of the toothbrush 20 but also the degree of opening of the bristle bundle. Specifically, the processing circuit 110 acquires an image generated by photographing the head unit 26 with the camera 210 facing the negative direction of the X axis (the first direction), and estimates the degree of opening of the bristle bundle based on the image. The degree of deterioration of the toothbrush 20 can be estimated based on the estimated bristle length and the degree of opening of the bristle bundle.

[0100] Fig. 20 is a diagram showing an example of a shooting instruction screen for acquiring an image that enables determination of the degree of opening of the bristle bundle. The screen shown in Fig. 20 is a guide screen that prompts the user to shoot an image of head portion 26 of toothbrush 20 in the negative direction of the X-axis (i.e., the first direction). Following the displayed instructions, the user points camera 210 toward the tip of head portion 26 of toothbrush 20 so that head portion 26 fits within guideline 223. When the user presses shooting button 224, shooting processing is executed and image data is recorded in storage device 120.

[0101] In the example of Fig. 20, the processes of steps S110 to S170 are executed as in the examples shown in Fig. 4 and Fig. 5. When both the shooting shown in Fig. 5 and the shooting shown in Fig. 20 are performed, the processes of steps S110 to S170 are repeated twice. Either the shooting shown in Fig. 5 or the shooting shown in Fig. 20 may be performed first.

[0102] 20, in step S130, the processing circuit 110 may be configured to calculate an index value indicating the degree of tilt caused by rotation about the second direction (Z direction) with respect to the reference posture, based on the dimension of the bristle base region in the third direction (Y direction) in the image. The processing circuit 110 can determine whether the posture of the toothbrush 20 is appropriate by comparing the index value with a reference range.

[0103] Fig. 21 is a diagram for explaining a method for estimating the inclination about the Z axis from the reference posture of the toothbrush 20. As shown in Fig. 21, the width (i.e., the dimension in the Z direction) of the bristle base region 323 corresponding to the bristle base 21 in the image is defined as z, and the height (i.e., the dimension in the Y direction) of the bristle base region 323 is defined as y. In addition, the height (i.e., the dimension in the Y direction) of the bristle bundle region 324 corresponding to the bristle bundle 22 in the image is defined as α, and the width (i.e., the dimension in the Z direction) of the bristle bundle region 324 is defined as γ.

[0104] The height y of the bristle base region 323 and the height α of the bristle bundle region 324 in the image vary according to the magnitude of the rotation angle of the toothbrush 20 around the Z axis. On the other hand, the width z of the bristle base region 323 hardly changes with the rotation of the toothbrush 20 around the Z axis. Therefore, the processing circuit 110 can evaluate the inclination of the toothbrush 20 around the Z axis using values ​​such as y, α, the ratio of α to γ, the ratio of α to z, the ratio of y to γ, or the ratio of y to z as index values. For example, the processing circuit 110 can determine that the posture of the toothbrush 20 is not appropriate when the index value is outside a predetermined reference range using γ / y or z / y as an index value. In this way, the index value can be calculated based on the ratio of the dimension z of the bristle base region 323 in the z direction (i.e., the second direction) or the dimension γ of the bristle bundle region 324 in the z direction to the dimension y of the bristle base region 323 in the y direction (i.e., the third direction).

[0105] 22 is a flowchart showing an example of a deterioration determination process based on the degree of opening of the hair bundle in addition to the hair length. The process shown in FIG. 22 can be performed in step S180 shown in FIG.

[0106] In the example of FIG. 22, in step S801, the processing circuit 110 determines whether the type of the toothbrush 20 to be subjected to the state determination is the first type. The types of the toothbrush 20 are classified into a plurality of types such as, for example, "normal" (first type), "ultra-thin" (second type), and "long bristles" (third type). The processing circuit 110 identifies the type of the toothbrush 20 by performing processing such as segmentation based on the acquired image, and determines whether the type is the first type. If the type is the first type, the process proceeds to step S809. If the type is not the first type, the process proceeds to step S802. Note that such type determination may be omitted.

[0107] In step S802, the processing circuit 110 determines the height of the bristle region based on the image acquired by the camera 210 photographing the toothbrush 20 in the second direction parallel to the Z axis. The height of the bristle region may be determined in the same manner as the processing of steps S181 and S182 shown in FIG. 13, or may be determined by other methods.

[0108] In step S803, the processing circuit 110 estimates the inclination of the toothbrush 20 from the reference posture at the time of photographing. This processing is the same as the processing of step S185 shown in FIG. 13.

[0109] In step S804, the processing circuit 110 corrects the height of the bristle region based on the estimated inclination to calculate the bristle length a0. This processing is the same as the processing of step S186 shown in FIG. 13.

[0110] In step S805, the processing circuit 110 determines whether the bristle length a0 is smaller than the threshold Th1. If a0 < Th1, the process proceeds to step S807. If a0 < Th1 is not satisfied, the process proceeds to step S806.

[0111] In step S806, the processing circuit 110 determines whether the bristle length a0 is smaller than the threshold Th2. The threshold Th2 is larger than the threshold Th1. If a0 < Th2, the process proceeds to step S808. If a0 < Th2 is not satisfied, the process proceeds to step S809.

[0112] In step S807, the processing circuit 110 sets the determination result as R1, and proceeds to step S190 shown in FIG.

[0113] In step S808, the processing circuit 110 sets the value of the determination flag F to "1." Then, the process proceeds to step S810.

[0114] In step S809, the processing circuit 110 sets the value of the determination flag F to "2." Then, the process proceeds to step S810.

[0115] Next, an example of the process executed in step S810 will be described with reference to Fig. 23. Fig. 23 is a flow chart showing a specific example of the process in step S810.

[0116] In step S811, the processing circuit 110 extracts side changes c1 and c2 based on an image acquired by photographing in a first direction parallel to the X-axis.

[0117] FIG. 24A and FIG. 24B are diagrams for explaining the side change amounts c1 and c2. FIG. 24A is a diagram for explaining the degree of deformation of the bristle bundle 22 of the "ultra-fine" type toothbrush 20. FIG. 24B is a diagram for explaining the degree of deformation of the bristle bundle 22 of the "normal" type toothbrush 20. The processing circuit 110 determines the degree of deformation of the bristle bundle 22 based on the side change amounts c1 and c2, which are the left-right change amounts of the bristle bundle 22 in the images shown in FIG. 24A and FIG. 24B. The side change amount c1 indicates the difference in position in the horizontal direction of the image between the leftmost part of the bristle bundle region in the image and the leftmost part of the bristle base region in the image. The side change amount c2 indicates the difference in position in the horizontal direction of the image between the rightmost part of the bristle bundle region in the image and the rightmost part of the bristle base region in the image. The processing circuit 110 can obtain the side change amounts c1 and c2 by image recognition.

[0118] In step S812, the processing circuit 110 determines whether both of the side changes c1 and c2 are smaller than the threshold Th3. If both of c1 and c2 are smaller than the threshold Th3, the process proceeds to step S813. If at least one of c1 and c2 is equal to or greater than Th3, the process proceeds to step S817.

[0119] In step S813, the processing circuit 110 determines whether both of the side changes c1 and c2 are smaller than a threshold Th4. The threshold Th4 is smaller than the threshold Th3. If both of c1 and c2 are smaller than the threshold Th4, the process proceeds to step S814. If at least one of c1 and c2 is equal to or greater than Th4, the process proceeds to step S816.

[0120] In step S814, the processing circuit 110 determines whether or not the value of the determination flag F is 2. If F=2, the process proceeds to step S815. If F=2 is not true, the process proceeds to step S816.

[0121] In step S815, the processing circuit 110 sets the determination result as R3, and proceeds to step S190 shown in FIG.

[0122] In step S816, the processing circuit 110 sets the determination result as R2, and proceeds to step S190 shown in FIG.

[0123] In step S817, the processing circuit 110 sets the determination result as R1, and proceeds to step S190 shown in FIG.

[0124] Fig. 25 is a diagram showing an example of a table storing the values ​​of thresholds Th1, Th2, Th3, and Th4 referred to by the processing circuit 110 in the above processing. In this example, a table storing different values ​​of thresholds Th1 to Th4 for each type of brush is stored in the storage device 120. By referring to such a table, the processing circuit 110 can make an appropriate determination for each type of brush. Note that the values ​​of thresholds Th1 to Th4 shown in Fig. 25 are merely examples, and values ​​different from those shown in the figure may be set.

[0125] Through the above process, the degree of deterioration of toothbrush 20 can be more appropriately determined based on the bristle length of toothbrush 20 and the degree of opening of bristle bundles 22, and the user can be informed of the determination result.

[0126] Although the embodiments of the present disclosure have been described above, the embodiments are not limited to these, and various modifications are possible within the scope of the gist of the embodiments. In addition, it is also possible to combine some or all of the various embodiments to create a new embodiment.

[0127] In the above-described embodiment, the state determination system 100 is provided in the terminal device 10. The present disclosure is not limited to such a form, and the functions of the state determination system 100 may be distributed among a plurality of devices. As shown in FIG. 3, a part of the functions of the state determination system 100 may be held by an external device 40 such as a server computer on a cloud. For example, among the functions of the state determination system 100, the terminal device 10 may execute the process of acquiring an image and the process of controlling the display, and the external device 40 may execute the process of determining the type of toothbrush 20 in the image, the process of determining whether the posture of the toothbrush 20 is appropriate, and the process of determining the deterioration state of the toothbrush 20. Alternatively, the external device 40 may execute a part of the process of determining the type of toothbrush 20 in the image, the process of determining whether the posture of the toothbrush 20 is appropriate, and the process of determining the deterioration state of the toothbrush 20. In this way, a part of the functions of the state determination system may be executed by an external device 40 such as a cloud server that communicates with the terminal device 10 via a network. In that case, the terminal device 10 and the external device 40 cooperate to function as a state determination system. With such a configuration, it is possible to speed up the processing. In this way, the state determination system is not limited to being realized by one device, but may be realized by a plurality of devices.

[0128] In this way, the state determination system may include a server computer and a terminal device connected to the server computer via a network. The terminal device may be configured to acquire an image generated by an imaging device photographing a head unit including a bristle bundle and a bristle base supporting the bristle bundle from the side, and transmit the image to the server computer. The server computer may recognize at least one of a bristle bundle region corresponding to the bristle bundle and a bristle base region corresponding to the bristle base from the image, estimate an inclination of the toothbrush from a reference posture at the time of photographing based on an outline of at least one of the bristle bundle region and the bristle base region in the image, and transmit information based on the estimated inclination to the terminal device. The information based on the estimated inclination may be information indicating the estimated inclination itself, or may be information regarding the bristle length of the toothbrush calculated based on the estimated inclination.

[0129] The server computer may calculate an index value indicating the degree of inclination of the toothbrush from a reference position at the time of photographing based on the outline or dimensions of at least one of the bristle bundle region and the bristle base region in the image received from the terminal device, and transmit information regarding the index value to the terminal device. The terminal device may change the content of the notification to the user depending on whether the index value is outside a predetermined reference range.

[0130] The server computer may also recognize a bristle bundle region corresponding to the bristle bundle from an image received from a terminal device, divide the bristle bundle region in the image into a plurality of partial regions aligned in a direction intersecting the direction in which the bristle bundle extends, estimate the bristle length of the toothbrush based on the length of the bristle bundle in each of the plurality of partial regions, and transmit information regarding the bristle length to the terminal device. The information regarding the bristle length may be information indicating the bristle length itself, or may be information regarding the deterioration state of the toothbrush estimated based on the bristle length.

[0131] (Additional Note) The above description of the embodiments discloses the following techniques.

[0132] (Technology 1) 1. A computer-implemented method comprising: Obtaining an image generated by an imaging device photographing a head portion including a tuft of bristles of a toothbrush and a tufting base supporting the tuft of bristles from the side; Recognizing at least one of a hair bundle region corresponding to the hair bundle and a hair implantation base region corresponding to the hair implantation base from the image; estimating an inclination of the toothbrush from a reference position at the time of photographing based on a contour of at least one of the bristle bundle region and the bristle base region in the image; The method includes:

[0133] According to the above configuration, the inclination of the toothbrush from the reference posture at the time of photographing can be estimated based on the outline of at least one of the bristle bundle region and the bristle base region recognized from the image of the toothbrush head. This makes it possible to determine whether the posture of the toothbrush at the time of photographing is appropriate.

[0134] (Technology 2) The recognizing step includes: recognizing the hair bundle region from the image; determining a dimension of the tuft region in a direction along which the tuft extends; Including, The method further comprises: estimating a bristle length of the toothbrush based on the inclination and a dimension of the bristle tuft area in a direction along which the bristle tuft extends. The method described in technique 1.

[0135] According to the above configuration, the bristle length of the toothbrush can be estimated based on the estimated inclination of the toothbrush and the height of the bristle bundle region, and the degree of deterioration of the toothbrush can be determined based on the estimated bristle length.

[0136] (Technology 3) The toothbrush includes a handle portion extending in one direction, the head portion being located at a tip of the handle portion, The bristle base has a bristle implantation surface extending in a first direction along a direction in which the handle portion extends and in a second direction perpendicular to the first direction, the tufts extend from the implantation surface in a third direction perpendicular to both the first direction and the second direction, the reference posture is a posture of the toothbrush when the imaging device faces the second direction and captures an image of the head portion, Estimating the inclination includes estimating a rotation angle of the toothbrush about the first direction with respect to the reference attitude as the inclination. The method according to technique 1 or 2.

[0137] According to the above configuration, the angle of rotation about the first direction along which the handle of the toothbrush extends can be estimated as the inclination of the toothbrush. This makes it possible to determine whether the posture of the toothbrush is correct or not based on the angle of rotation about the first direction of the toothbrush at the time of photographing.

[0138] (Technology 4) The recognizing step includes recognizing a boundary between the hair bundle region and the hair implantation base region from the image, estimating the inclination includes estimating the inclination based on a shape of a boundary between the hair bundle region and the hair implantation table region in the image, or based on positions of at least three points on the boundary. The method according to any one of techniques 1 to 3.

[0139] According to the above configuration, the inclination can be estimated based on the shape of the boundary between the bristle bundle region and the bristle base region in the image of the toothbrush, or the positions of at least three points on the boundary, thereby making it possible to more accurately estimate the inclination of the toothbrush.

[0140] (Technology 5) The method according to technique 4, wherein estimating the slope includes determining the slope based on an index value calculated based on the positions of the at least three points and information pre-stored in a storage device that defines a correspondence between the index value and the slope.

[0141] According to the above configuration, the inclination can be determined based on an index value calculated based on the positions of at least three points on the boundary between the bristle bundle region and the bristle base region in the image of the toothbrush, and information that defines the correspondence between the index value and the inclination of the toothbrush, thereby making it possible to more accurately estimate the inclination of the toothbrush.

[0142] (Technology 6) the at least three points include a first point and a second point at opposite ends of the boundary, and a third point located in the center of the boundary; The method described in technique 5, wherein the index value is a value based on a distance between the third point and a line segment connecting the first point and the second point.

[0143] According to the above configuration, an index value can be calculated based on the distance between a line segment connecting a first point and a second point on both ends of the boundary between the bristle bundle region and the bristle base region in the image of the toothbrush, and a third point located in the center of the boundary. By using such an index value, the inclination of the toothbrush can be estimated more accurately.

[0144] (Technology 7) The recognizing step includes recognizing a boundary between the hair bundle region and a background from the image; estimating the inclination includes estimating the inclination based on a shape of a boundary between the hair bundle region and the background in the image, or based on positions of at least three points on the boundary. The method according to any one of techniques 1 to 3.

[0145] According to the above configuration, the inclination can be estimated based on the shape of the boundary between the bristle bundle region and the background in the image of the toothbrush, or the positions of at least three points on the boundary, thereby making it possible to more accurately estimate the inclination of the toothbrush.

[0146] (Technology 8) The recognition step includes recognizing a boundary between the hair implantation base region and a background from the image; estimating the inclination includes estimating the inclination based on a shape of a boundary between the hair implantation base region and the background in the image, or based on positions of at least three points on the boundary. The method according to any one of techniques 1 to 3.

[0147] According to the above configuration, the inclination can be estimated based on the shape of the boundary between the bristle base region and the background in the image of the toothbrush, or the positions of at least three points on the boundary, thereby making it possible to more accurately estimate the inclination of the toothbrush.

[0148] (Technology 9) The estimating of the hair length comprises: Calculating the hair length by correcting a dimension of the hair bundle region in the direction in which the hair bundle extends based on the inclination; The method according to claim 2,

[0149] According to the above configuration, the bristle length of the toothbrush can be calculated by correcting the height of the bristle bundle region in the image based on the estimated tilt, thereby making it possible to calculate the bristle length more accurately.

[0150] (Technology 10) estimating a degree of deterioration of the toothbrush based on the estimated bristle length; outputting information regarding deterioration of the toothbrush; The method according to any one of claims 2 to 9, further comprising:

[0151] According to the above configuration, the degree of deterioration of the toothbrush can be estimated based on the estimated bristle length, and information indicating the degree of deterioration of the toothbrush can be output, thereby allowing the user to know the degree of deterioration of the toothbrush.

[0152] (Technology 11) A method according to any one of techniques 1 to 10, further comprising outputting information prompting a user to take a photograph of the head portion of the toothbrush from the side.

[0153] According to the above configuration, it is possible to output information that prompts the user to take an image of the toothbrush head from the side, whereby an image of the toothbrush can be generated by the user taking an image of the toothbrush head from the side using the imaging device.

[0154] (Technology 12) An apparatus comprising a computer for executing the method according to any one of techniques 1 to 11.

[0155] According to the above configuration, it is possible to provide an apparatus for executing the method according to any one of Techniques 1 to 11.

[0156] (Technology 13) A program for causing a computer to execute the method according to any one of techniques 1 to 11.

[0157] According to the above configuration, it is possible to provide a program for causing a computer to execute the method according to any one of Techniques 1 to 11.

[0158] (Technology 14) A server computer; a terminal device connected to the server computer via a network; A system comprising: The terminal device acquires an image generated by an imaging device photographing a head unit including a tuft of bristles and a bristle base supporting the tuft of bristles from the side, and transmits the image to the server computer; The server computer, Recognizing at least one of a hair bundle region corresponding to the hair bundle and a hair implantation base region corresponding to the hair implantation base from the image; estimating an inclination of the toothbrush from a reference position at the time of photographing based on a contour of at least one of the bristle bundle region and the bristle base region in the image; transmitting information based on the tilt estimation result to the terminal device; system.

[0159] According to the above configuration, the server computer and the terminal device cooperate to execute the above-mentioned method, so that the processing speed can be increased. [Industrial Applicability]

[0160] The technology disclosed herein can be applied to a system capable of determining the deterioration state of a toothbrush, for example. Specifically, the technology disclosed herein can be applied to a device or application that determines the deterioration state of an electric toothbrush or a general manual toothbrush and indicates the appropriate time to replace the toothbrush. [Explanation of symbols]

[0161] 10 Terminal Equipment 20 Toothbrush 21 Hair transplantation table 21a Flocked surface 22 Hair bundle 25 Handle 26 Head section 100 Status Judgment System 110 Processing circuit 120 Storage device 130 Communication I / F 140 Input / Output Interface 210 Camera 220 Display

Claims

1. 1. A computer-implemented method comprising: Obtaining an image generated by an imaging device photographing a head portion including a tuft of bristles of a toothbrush and a tufting base supporting the tuft of bristles from the side; Recognizing at least one of a hair bundle region corresponding to the hair bundle and a hair implantation base region corresponding to the hair implantation base from the image; estimating an inclination of the toothbrush from a reference position at the time of photographing based on a contour of at least one of the bristle bundle region and the bristle base region in the image; The method includes:

2. The recognizing Recognizing the hair bundle region from the image; determining a dimension of the tuft region in a direction along which the tuft extends; Including, The method further comprises: estimating a bristle length of the toothbrush based on the inclination and a dimension of the bristle tuft area in a direction along which the bristle tuft extends. The method of claim 1.

3. The toothbrush includes a handle portion extending in one direction, the head portion being located at a tip of the handle portion, The bristle base has a bristle implantation surface extending in a first direction along a direction in which the handle portion extends and in a second direction perpendicular to the first direction, the tufts extend from the implantation surface in a third direction perpendicular to both the first direction and the second direction, the reference posture is a posture of the toothbrush when the imaging device faces the second direction and captures an image of the head portion, Estimating the inclination includes estimating a rotation angle of the toothbrush about the first direction with respect to the reference attitude as the inclination. The method according to claim 1 or 2.

4. The recognizing step includes recognizing a boundary between the hair bundle region and the hair implantation base region from the image, estimating the inclination includes estimating the inclination based on a shape of a boundary between the hair bundle region and the hair implantation table region in the image, or on positions of at least three points on the boundary. The method according to claim 1 or 2.

5. 5. The method according to claim 4, wherein estimating the slope includes determining the slope based on an index value calculated based on the positions of the at least three points and information pre-stored in a storage device that defines a correspondence between the index value and the slope.

6. the at least three points include a first point and a second point at opposite ends of the boundary, and a third point located in the center of the boundary; The method according to claim 5 , wherein the index value is a value based on a distance between the third point and a line segment connecting the first point and the second point.

7. The recognizing step includes recognizing a boundary between the hair bundle region and a background from the image; estimating the inclination includes estimating the inclination based on a shape of a boundary between the hair bundle region and the background in the image, or based on positions of at least three points on the boundary. The method according to claim 1 or 2.

8. The recognition step includes recognizing a boundary between the hair implantation base region and a background from the image; The step of estimating the inclination includes estimating the inclination based on a shape of a boundary between the hair implantation base region and the background in the image, or on positions of at least three points on the boundary. The method according to claim 1 or 2.

9. The estimating of the hair length comprises: Calculating the hair length by correcting a dimension of the hair bundle region in the direction in which the hair bundle extends based on the inclination; The method of claim 2 , comprising:

10. estimating a degree of deterioration of the toothbrush based on the estimated bristle length; outputting information regarding deterioration of the toothbrush; 10. The method of claim 2 or 9, further comprising:

11. The method of claim 1 or 2, further comprising outputting information prompting a user to take a side photograph of the head of the toothbrush.

12. An apparatus comprising a computer for carrying out the method according to claim 1 or 2.

13. A program for causing a computer to execute the method according to claim 1 or 2.

14. A server computer; a terminal device connected to the server computer via a network; A system comprising: The terminal device acquires an image generated by an imaging device photographing a head unit including a tuft of bristles and a bristle base supporting the tuft of bristles from the side, and transmits the image to the server computer; The server computer, Recognizing at least one of a hair bundle region corresponding to the hair bundle and a hair implantation base region corresponding to the hair implantation base from the image; estimating an inclination of the toothbrush from a reference position at the time of photographing based on a contour of at least one of the bristle bundle region and the bristle base region in the image; transmitting information based on the tilt estimation result to the terminal device; system.

Images