Judgment system and program

Abstract

To provide a determination system and a program which can accurately determine the replacement timing of a toothbrush.SOLUTION: A determination system according to the present disclosure comprises: an image acquisition unit which acquires an image of a toothbrush; a feature amount information acquisition unit which acquires feature amount information of the toothbrush; a reference information acquisition unit which acquires reference information being information corresponding to the toothbrush and about a determination criterion on whether or not to replace the toothbrush on the basis of the feature amount information; a determination unit which determines whether or not to replace the toothbrush on the basis of the image and the reference information; and an output unit which outputs information including the determination result by the determination unit.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] FIELD An embodiment of the present invention relates to a determination system and a program. [Background technology]

[0002] As part of oral care support for consumers, dental hygienists visually inspect a consumer's toothbrush, recommend replacement times, and provide guidance on brushing habits. However, this method is unlikely to lead to behavioral change unless the consumer visits a dental clinic. Furthermore, with the development of toothbrushes with diverse profiles in recent years, dental hygienists' judgments can easily vary. In particular, when it comes to replacement times, the appearance of bristles opening can vary depending on factors such as the type of bristles used and the size of the head, so an objective judgment of bristles opening tailored to the characteristics of each toothbrush is required.

[0003] Meanwhile, in recent years, as described in Patent Documents 1 to 3, techniques have been developed that determine when to replace a toothbrush and present the determination result to the user. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Special Publication No. 2003-532487 [Patent Document 2] Japanese Patent Application Laid-Open No. 2017-060661 [Patent Document 3] Japanese Patent Application Publication No. 2019-080752 Summary of the Invention [Problem to be solved by the invention]

[0005] However, it is not easy to determine when to replace a toothbrush based on the characteristics of each toothbrush, and sufficient accuracy in the determination has not been achieved.

[0006] The problem to be solved by the present invention is to provide a determination system and program that can accurately determine when to replace a toothbrush. [Means for solving the problem]

[0007] The present invention may include the following aspects. [1] A judgment system comprising: an image acquisition unit that acquires an image of a toothbrush; a feature information acquisition unit that acquires feature information of the toothbrush; a reference information acquisition unit that acquires reference information corresponding to the toothbrush based on the feature information, the reference information being information related to the criteria for determining whether or not the toothbrush should be replaced; a judgment unit that judges whether or not the toothbrush should be replaced based on the image and the reference information; and an output unit that outputs information including the judgment result by the judgment unit. [2] The determination system described in [1], wherein the feature information includes information indicating attributes of the toothbrush head. [3] The determination system described in [2], wherein the information indicating the attributes of the toothbrush head includes information regarding one or more of the attributes of the base of the head and the attributes of the bristles of the head. [4] A judgment system described in any of [1] to [3], further comprising a bristle opening degree estimation unit that estimates a bristle opening degree indicating the degree to which the bristles of the toothbrush are opened based on the image, and the judgment unit determines whether the toothbrush should be replaced by comparing the bristle opening degree with the reference information. [5] The determination system described in [4], wherein the degree of bristle opening is a value calculated based on geometric information of the toothbrush bristles in the image. [6] The determination system described in [5], wherein the hair opening degree estimation unit selects the geometric information used to calculate the hair opening degree according to the feature information. [7] A determination system described in any of [4] to [6], wherein the image acquisition unit acquires an image of the toothbrush before use as a first image and an image of the toothbrush after use as a second image, and the bristle opening degree estimation unit estimates the bristle opening degree by comparing the first image with the second image. [8] The determination system according to any one of [4] to [7], wherein the bristle opening degree estimation unit estimates the bristle opening degree for each different part of the toothbrush. [9] A judgment system described in any of [4] to [8], wherein the reference information acquisition unit acquires a bristle opening threshold value according to the feature information as the reference information, and the judgment unit judges whether the toothbrush should be replaced by comparing the bristle opening degree with the bristle opening threshold value.

[10] The determination system according to any one of [4] to [9], further comprising an instruction information generation unit that generates instruction information including a recommended tooth brushing method based on the degree of bristle opening.

[11] The determination system according to any one of [1] to

[10] , wherein the image acquisition unit acquires an image of the toothbrush as seen from at least the tip side of the toothbrush.

[12] A guide frame determination unit that determines the type of guide frame to assist a user terminal having an imaging unit that photographs the toothbrush in photographing the toothbrush according to the feature amount information; and a display control unit that displays the guide frame on a display unit provided in the user terminal. The determination system according to any one of [1] to

[11] , further comprising:

[13] The determination system according to any one of [1] to

[12] , wherein the determination unit determines whether the orientation of the toothbrush in the image of the toothbrush is appropriate.

[14] A program that causes a processor of an information processing device to execute an image acquisition step of acquiring an image of a toothbrush, a feature information acquisition step of acquiring feature information of the toothbrush, a reference information acquisition step of acquiring reference information corresponding to the toothbrush based on the feature information as information regarding the criteria for determining whether or not the toothbrush should be replaced, a judgment step of judging whether or not the toothbrush should be replaced based on the image and the reference information, and an output step of outputting information including the judgment result of the judgment step. [Effects of the Invention]

[0008] According to the present invention, it is possible to provide a determination system and program that can accurately determine when to replace a toothbrush. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a schematic diagram illustrating a determination system according to a first embodiment. [Figure 2] 1A and 1B are schematic diagrams showing the toothbrush before and after use. [Figure 3] FIG. 2 is a block diagram showing the functional configuration of the determination system according to the first embodiment. [Figure 4] FIG. 4 is a diagram showing an example of a screen displayed by a display unit. [Figure 5] FIG. 4 is a diagram showing an example of a screen displayed by a display unit. [Figure 6] FIG. 4 is a diagram showing an example of a screen displayed by a display unit. [Figure 7] FIG. 10 is a diagram illustrating an example of a determination result database. [Figure 8] 5A to 5C are diagrams illustrating an example of a method for estimating a hair opening degree by a hair opening degree estimating unit according to the first embodiment. [Figure 9] FIG. 10 is a diagram illustrating an example of a hair opening threshold table. [Figure 10] FIG. 3 is a flow diagram showing the flow of data in the determination system of the first embodiment. [Figure 11] FIG. 3 is a flowchart showing a determination method performed by the determination system according to the first embodiment. [Figure 12] FIG. 3 is a flowchart showing the flow of processing by the determination system according to the first embodiment. [Figure 13] 10A and 10B are diagrams illustrating an example of a method for estimating a hair opening degree by a hair opening degree estimating unit according to the second embodiment. [Figure 14] 10A and 10B are diagrams illustrating an example of a method for estimating a hair opening degree by a hair opening degree estimating unit according to the second embodiment. [Figure 15] FIG. 10 is a flowchart showing the flow of processing by a determination system according to a second embodiment. [Figure 16] 13A and 13B are diagrams illustrating an example of a method for estimating a hair opening degree by a hair opening degree estimating unit according to the third embodiment. [Figure 17] 13A and 13B are diagrams illustrating an example of a method for estimating a hair opening degree by a hair opening degree estimating unit according to the third embodiment. [Figure 18] 13A and 13B are diagrams illustrating an example of a method for estimating a hair opening degree by a hair opening degree estimating unit according to the third embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, a determination system and a program according to an embodiment of the present invention will be described with reference to the drawings. Note that the drawings are schematic or conceptual, and the relationship between the thickness and width of each part, the size ratio between parts, etc., are not necessarily the same as those in reality. Furthermore, even when the same part is shown, the dimensions and ratios may be expressed differently depending on the drawing. Furthermore, in this specification, "based on XX" means "based on at least XX" and includes cases where it is based on other elements in addition to XX. Furthermore, "based on XX" is not limited to cases where XX is used directly, but also includes cases where it is based on XX that has been calculated or processed. "XX" is any element (for example, information).

[0011] <Summary> The determination system according to the embodiment uses criteria based on the feature information of the toothbrush to determine when to use the toothbrush. This allows the determination system to change the criteria used depending on the characteristics of each toothbrush (e.g., the width of the toothbrush head, the shape of the bristles, the material of the bristles, etc.). This makes it possible to objectively determine whether a toothbrush needs replacing based on its characteristics.

[0012] First Embodiment FIG. 1 is a schematic diagram showing a determination system 1 of a first embodiment. The determination system 1 includes one or more user terminals 100 and a server 200 (an example of an "information processing device") connected to each other via a network N. A user takes an image of a toothbrush 10 using the user terminal 100. The user terminal 100 transmits the acquired image of the toothbrush 10 to the server 200. The server 200 determines whether the toothbrush 10 should be replaced. The server 200 transmits the determination result to the user terminal 100. The user terminal 100 presents the determination result to the user. Note that the determination system 1 may be a single determination device (an example of an "information processing device").

[0013] [Toothbrush 10 components] The toothbrush 10 to be determined by the determination system 1 is not particularly limited, and may be a typical manual toothbrush or an electric toothbrush. FIG. 1 shows a typical toothbrush 10. As shown in FIG. 1, the toothbrush 10 includes a head 12, a handle 14, and a neck 16. The head 12 includes a head substrate 20 and a brush section 22 with bristles planted on the head substrate 20. The head substrate 20 includes a bristle planting surface 24 on which the brush section 22 is planted. The brush section 22 includes one or more bristle bundles B. Each bristle bundle B is configured to include a bundle of multiple bristles.

[0014] FIG. 2 is a schematic diagram showing the state of a toothbrush before and after use. In the example of FIG. 2, each bristle bundle B of the toothbrush 10 (left diagram) before use (i.e., at the time of manufacture or immediately after purchase) extends approximately perpendicular to the bristle implantation surface 24. In contrast, each bristle bundle B of the toothbrush 10 after use (right diagram) is spread outward more than before use due to the pressure applied when brushing teeth. The determination system 1 can determine whether or not the toothbrush should be replaced based on the degree of bristle spreading. Note that the configuration of the toothbrush 10 is not limited to the above example. For example, the bristle bundles B may be spread out to some extent even before use. Furthermore, the bristles of the brush head 22 do not have to be bundled.

[0015] [Configuration of Judgment System 1] 3 is a block diagram showing the functional configuration of the determination system 1. The determination system 1 includes a user terminal 100 and a server 200 connected to each other via a network N. The user terminal 100 and the server 200 are preferably connected wirelessly, but may also be connected by wire. The network N includes one or more communication networks such as the Internet, an intranet, a wired LAN (Local Area Network), a wireless LAN, etc.

[0016] [Configuration of user terminal 100] The user terminal 100 may be any information processing device, such as a smartphone, a tablet terminal device, or a personal computer. For example, a user can install an application program for the determination system 1 according to this embodiment in his / her terminal device, and use the terminal device as the user terminal 100 according to this embodiment. Alternatively, a user may use a web browser installed in his / her terminal device to browse a web page for the determination system 1, and use the terminal device as the user terminal 100 according to this embodiment. The determination system 1 may be realized as a so-called web application.

[0017] A specific configuration of the user terminal 100 will be described with reference to Fig. 3. The user terminal 100 includes a processing unit 110, a storage unit 112, an operation unit 114, an imaging unit 116, a display unit 118, and a communication unit 120 as its functional units.

[0018] The processing unit 110 performs predetermined arithmetic processing on the information acquired by the user terminal 100. Specifically, the processing unit 110 reads and executes a program 150 stored in the storage unit 112, thereby performing various processes such as image recognition and display control.

[0019] In terms of hardware configuration, the processing unit 110 may be composed of, for example, an arithmetic processing device such as a CPU (Central Processing Unit), an MPU (Micro-Processing Unit) or a GPU (Graphics Processing Unit), a ROM (Read Only Memory), and a RAM (Random Access Memory).

[0020] The storage unit 112 stores various programs 150 executed by the processing unit 110, various data used for processing in the processing unit 110, etc. Specifically, the storage unit 112 can be configured as a large-capacity storage device such as a hard disk.

[0021] The program 150 may be provided in a state recorded on a computer-readable recording medium. The user terminal 100 reads the program 150 from this recording medium and stores it in the storage unit 112. However, the method for storing the program 150 in the storage unit 112 is not limited to this, and the program 150 may be pre-installed when the user terminal 100 is manufactured, provided via a network, or any other method may be used.

[0022] The operation unit 114 accepts user operations and transmits the accepted operations to the processing unit 110. The hardware configuration of the operation unit 114 is not particularly limited, and any configuration can be used, such as buttons provided on the user terminal 100, a touch panel provided on a display, or an external input device such as a mouse or keyboard.

[0023] The imaging unit 116 captures an image of an object. Specifically, as shown in FIG. 1, the imaging unit 116 captures an image of the toothbrush 10 and converts it into image data. As shown in FIG. 3, the imaging unit 116 transmits the image data to the processing unit 110. For example, the imaging unit 116 is a camera provided in the user terminal 100, such as a built-in camera or an external camera of a smartphone. Preferably, an external camera with high resolution can be used preferentially as the imaging unit 116. In addition, the user terminal 100 may be provided with any attachment (for example, a close-up lens, a light, etc.) to assist the imaging unit 116 in capturing images.

[0024] The display unit 118 displays text, images, and the like to the user based on the output of the processing unit 110. FIGS. 4 to 6 are diagrams showing example screens displayed by the display unit 118. As shown in FIGS. 1 and 4 to 6, the display unit 118 displays instructions and notifications to the user, and displays images captured by the imaging unit 116. For example, the display unit 118 can be configured with any display device such as a liquid crystal display or an organic EL display.

[0025] 3, the communication unit 120 communicates with other devices via the network N. Specifically, the communication unit 120 transmits data output from the processing unit 110 to other devices (e.g., the server 200). The communication unit 120 also receives data transmitted from other devices (e.g., the server 200) and transmits the data to the processing unit 110.

[0026] (Functional configuration of the processing unit 110) Next, the functional configuration of the processing unit 110 will be described with reference to Fig. 3. The processing unit 110 includes, as its functional units, an image acquisition unit 130, a feature amount information acquisition unit 132, a guide frame determination unit 134, a display control unit 136, an image recognition unit 138, a direction estimation unit 140, a determination unit 142, a photography guide unit 144, and an analysis region designation unit 146. These functional units are realized by the processing unit 110 reading out a program 150 stored in the storage unit 112 and executing the program.

[0027] (Image acquisition unit 130) The image acquisition unit 130 acquires the image captured by the imaging unit 116 as image data 152. The image acquisition unit 130 associates the image data 152 with information about the date of capture and stores the image data 152 in the storage unit 112. The image acquisition unit 130 also transmits the image data 152 and the information about the date of capture to the server 200 via the communication unit 120.

[0028] Preferably, when imaging unit 116 photographs toothbrush 10, imaging unit 116 photographs toothbrush 10 so that the tip of brush head 22 of toothbrush 10 (i.e., the opening of the bristles of brush head 22) is in focus. This is effective for accurately estimating the replacement time based on the opening of the bristles of brush head 22. There are no particular limitations on the method of focusing, and a user may manually focus on the bristles of brush head 22 and lock the focus. Alternatively, a series of photographs focused at various distances from imaging unit 116 may be taken, and an image in which the bristles of brush head 22 are in focus may be manually or automatically selected.

[0029] (Feature information acquisition unit 132) The feature amount information acquisition unit 132 acquires feature amount information 154 indicating the feature amount of the toothbrush 10. The feature amount of the toothbrush 10 is an amount that represents the characteristics or properties of the toothbrush 10. For example, the feature amount of the toothbrush 10 may be the following: (a) Model number, serial number, product name, product type, size, shape, and material of toothbrush 10 (b) Size (e.g., width, length, thickness), shape, and material of the head base material 20 (c) The size (for example, the width and length of the entire brush part 22), shape, and arrangement of the brush part 22 (d) The size (e.g., thickness and length of each bristle bundle B) of each bristle bundle B constituting the brush part 22, the shape, the number of bristle bundles B, and the arrangement (e.g., the position of each bristle bundle B on the bristle implantation surface 24, the positional relationship between the bristle bundles B, and the density of the bristle bundles B on the bristle implantation surface 24) (e) The size (e.g., thickness and length of hair), shape (e.g., straight hair, tapered hair, super tapered hair, etc.), and material (e.g., type, hardness, and elasticity of constituent material) of the hairs constituting hair bundle B (f) Codes such as barcodes (e.g., JAN (Japan Article Number) codes) and two-dimensional codes that encode at least part of the information described above.

[0030] The feature information acquisition unit 132 can acquire feature information 154 based on user input. For example, the feature information acquisition unit 132 can acquire information entered by the user as feature information 154 by displaying on the display unit 118 an instruction requesting the user to enter the product name of the toothbrush 10, the width of the head base material 20, and the like. The feature information acquisition unit 132 may also acquire feature information 154 entered by the user in advance. For example, the feature information acquisition unit 132 can display on the display unit 118 an instruction requesting the user to enter the product name of the toothbrush 10, the width of the head base material 20, and the like before starting to use the toothbrush 10. The feature information acquisition unit 132 can acquire information entered in accordance with the instruction as feature information 154. The feature information acquisition unit 132 also acquires information regarding the start date of use of the toothbrush 10, which is entered by the user.

[0031] The feature information acquisition unit 132 may acquire the feature information 154 based on the image data 152 acquired by the image acquisition unit 130. For example, the feature information acquisition unit 132 may identify the position of the head substrate 20 in the captured image by image recognition and automatically acquire the width of the head substrate 20. The feature information acquisition unit 132 may identify a code such as a two-dimensional code included in the image data 152 and acquire the code as the feature information 154. Alternatively, the feature information acquisition unit 132 may acquire the feature information 154 by accessing a web page on which the feature information 154 is recorded based on information (e.g., a URL) indicated by the code such as the two-dimensional code. When referring to the code, the feature information acquisition unit 132 can display an instruction on the display unit 118 requesting the user to capture an image of the code attached to the product or its package.

[0032] The feature information acquisition unit 132 may directly estimate the type of toothbrush 10 from an image of the toothbrush 10. The method for estimating the type of toothbrush 10 is not particularly limited. For example, the feature information acquisition unit 132 may search for an image that is most similar to the captured image by referring to a correspondence table between toothbrush 10 images and types. Alternatively, the feature information acquisition unit 132 may use machine learning to learn the relationship between toothbrush 10 images and types in advance. The images of toothbrush 10 used for machine learning may include images of toothbrush 10 before use, toothbrush 10 in various states that are still usable after first use, and toothbrush 10 due for replacement. The images used for learning may be images of toothbrush 10 that have actually been used, or may be images of toothbrush 10 used as a model with various loads, stroke counts, angles, etc. using a toothbrush tester or the like. It is also preferable to perform learning to identify objects other than toothbrushes (e.g., brushes, scrubbing pads, etc.).

[0033] The feature amount information acquisition unit 132 stores the acquired feature amount information 154 and information on the start date of use in the storage unit 112. In addition, the feature amount information acquisition unit 132 transmits the acquired feature amount information 154 and information on the start date of use to the server 200 via the communication unit 120.

[0034] (Guide frame determination unit 134) The guide frame determination unit 134 determines the type of guide frame G to assist in photographing the toothbrush 10. Here, the guide frame G is, for example, a frame line displayed on the display unit 118 when photographing is performed by the imaging unit 116, as shown in FIGS.

[0035] Specifically, the guide frame determination unit 134 can determine the type of guide frame G to use according to the feature amount information 154 of the toothbrush 10. This allows the guide frame determination unit 134 to select the type of guide frame G that is appropriate for each individual toothbrush 10 according to the features of the toothbrush 10. For example, the guide frame determination unit 134 can determine the optimal size and shape of the guide frame G for each toothbrush 10.

[0036] The guide frame determination unit 134 displays a guide frame G on the display unit 118 when capturing an image using the imaging unit 116. In addition, the guide frame determination unit 134 can display an instruction I1 on the display unit 118 requesting that the toothbrush 10 to be captured be captured so that the head 12 of the toothbrush 10 fits within the guide frame G, as shown in FIG. 4(a).

[0037] The guide frame determination unit 134 can select guide frames G corresponding to the handle 14 or the neck 16 in addition to the guide frame G corresponding to the head 12. In this case, the guide frame determination unit 134 can cause the display unit 118 to display an instruction requesting that the toothbrush 10 be photographed so that the handle 14 or the neck 16 is aligned with the guide frame G.

[0038] (Display control unit 136) The display control unit 136 controls the operation of the display unit 118 and causes desired information to be displayed on the display unit 118. For example, the display control unit 136 causes the display unit 118 to display the determination result of whether the toothbrush 10 should be used, received from the server 200 (described later). Specifically, when the communication unit 120 receives from the server 200 a determination result that "it is not yet time to replace the toothbrush 10," the display control unit 136 causes the display unit 118 to display a message and image M1 indicating that the photographed toothbrush 10 is still usable, as shown in FIG. 4(b). Preferably, the display control unit 136 can also display an estimated usable period and the like in the message M1 on the display unit 118. On the other hand, when the communication unit 120 receives from the server 200 a determination result that "it is time to replace the toothbrush 10," the display control unit 136 causes the display unit 118 to display a message and image M2 indicating that the photographed toothbrush 10 should be replaced, as shown in FIG. 4(c).

[0039] (Image recognition unit 138) The image recognition unit 138 performs image recognition on the image data 152. For example, the image recognition unit 138 can recognize the position, orientation, size, shape, etc. of the toothbrush 10 in the image. The image recognition unit 138 can also recognize the position, orientation, size, shape, etc. of each element constituting the toothbrush 10 (e.g., the head 12, the handle 14, the neck 16, the head base material 20, and the brush head 22). The method of image recognition is not particularly limited. For example, the image recognition unit 138 can perform image recognition by detecting the shape and color of the edges of objects in the image using a differential filter. Alternatively, the image recognition unit 138 may use a trained model that has learned the relationship between the image and the position of the toothbrush 10 in the image. The trained model may be a machine learning model using a neural network such as a convolutional neural network, or may be another algorithm.

[0040] (Orientation estimation unit 140) The orientation estimation unit 140 estimates the orientation of the toothbrush 10 in the image based on the recognition result of the image recognition unit 138. For example, the orientation estimation unit 140 can estimate whether the brush head 22 of the toothbrush 10 is facing toward the image capture unit 116, facing away from the image capture unit 116, or facing horizontally or diagonally. Specifically, the orientation estimation unit 140 can estimate the relationship between the orientation of the toothbrush 10 (e.g., the orientation of the brush head 22) and the orientation of the image capture unit 116 (e.g., the relative angle between them).

[0041] For example, the orientation estimation unit 140 can estimate whether the brush unit 22 faces the front side or the opposite side as viewed from the imaging unit 116, based on the result of recognition of the brush unit 22 by the image recognition unit 138. For example, if the image recognition unit 138 recognizes that most of the brush unit 22 is included in the image, the orientation estimation unit 140 estimates that the brush unit 22 faces the front side as viewed from the imaging unit 116. On the other hand, if the image recognition unit 138 recognizes that most of the brush unit 22 is not included in the image, the orientation estimation unit 140 estimates that the brush unit 22 faces the opposite side as viewed from the imaging unit 116.

[0042] The orientation estimation unit 140 can estimate the left-right orientation of the brush unit 22 based on the result of edge detection of the brush unit 22. For example, when the orientation estimation unit 140 estimates that the brush unit 22 faces the front side as seen from the imaging unit 116, if the edges of the brush unit 22 are symmetrical, the orientation estimation unit 140 estimates that the brush unit 22 faces the imaging unit 116. On the other hand, when the orientation estimation unit 140 estimates that the brush unit 22 faces the front side as seen from the imaging unit 116, if the edges of the brush unit 22 are asymmetrical, the orientation estimation unit 140 estimates that the brush unit 22 does not face the front side as seen from the imaging unit 116, but is tilted to the left or right.

[0043] The orientation estimation unit 140 may estimate the orientation of the toothbrush 10 based on the contours and positional relationship of the head 12, handle 14, and / or neck 16 recognized by the image recognition unit 138. For example, if the contours of the head 12, neck 16, etc. are symmetrical, the orientation estimation unit 140 can estimate that the toothbrush 10 is not tilted left or right. On the other hand, if the contours of the head 12, neck 16, etc. are asymmetrical, the orientation estimation unit 140 can estimate that the toothbrush 10 is tilted left or right. Alternatively, the orientation estimation unit 140 can acquire, based on the feature information 154, the structures of the head 12, handle 14, neck 16, etc. of the toothbrush 10 being photographed, or an image of the toothbrush 10 in an optimal orientation. The orientation estimation unit 140 can estimate the orientation of the toothbrush 10 by comparing this information with an actually photographed image of the toothbrush 10.

[0044] The orientation estimation unit 140 may estimate the orientation of the toothbrush 10 using a trained model that has learned the relationship between the image of the toothbrush 10 and information about the orientation of the toothbrush 10 .

[0045] (Judgment unit 142) The determination unit 142 determines whether the orientation of the toothbrush 10 is within a predetermined range. Specifically, the determination unit 142 determines whether the orientation of the toothbrush 10 estimated by the orientation estimation unit 140 is within a predetermined acceptable range. The determination unit 142 may use a constant numerical range as this acceptable range regardless of the type of toothbrush 10, or may use a different numerical range depending on the feature amount information 154.

[0046] (Photography Guide Section 144) The photographing guide unit 144 guides the user in photographing the toothbrush 10. Specifically, the photographing guide unit 144 notifies the user whether the orientation of the toothbrush 10 to be photographed is appropriate or not, based on the determination result of the determination unit 142. For example, if the determination unit 142 determines that the orientation of the toothbrush 10 is within the allowable range, the photographing guide unit 144 can notify the user that the orientation of the toothbrush 10 is appropriate. Alternatively, the photographing guide unit 144 does not need to provide a special notification in such a case. On the other hand, if the determination unit 142 determines that the orientation of the toothbrush 10 is outside the allowable range, the photographing guide unit 144 can display, on the display unit 118, a message indicating that the orientation of the toothbrush 10 is not appropriate and an instruction I2 requesting the user to photograph again, as shown in FIG. 5 . In this case, the photographing guide unit 144 may notify the user (e.g., in real time) in which direction the toothbrush 10 should be oriented. For example, if an image of the toothbrush 10 taken from above the brush portion 22 is required, the photographing guide unit 144 can guide the user in photographing so that the lens of the imaging unit 116 is positioned vertically above the center of the brush portion 22 and horizontally to the bristle surface 24.

[0047] When photographing the toothbrush 10, the photographing guide unit 144 may display on the display unit 118 an instruction requesting that the photograph be taken against a plain background. Image data 152 with a plain background is advantageous in that it makes it easier for the image recognition unit 138 to detect the brush unit 22. When the image recognition unit 138 recognizes that the background of the image data 152 is not plain, the photographing guide unit 144 may display on the display unit 118 an instruction requesting that the photograph be taken again so that the background is plain. Note that the photographing guide unit 144 may adjust the brightness and contrast of the image data 152 by known image processing. Furthermore, the photographing guide unit 144 may perform image processing so that the difference between the part recognized as the background by the image recognition unit 138 and the other parts becomes clearer.

[0048] The photographing guide unit 144 can display instructions on the display unit 118 to guide the user to ensure that the image is appropriately lit when photographing the toothbrush 10. For example, the photographing guide unit 144 can display instructions on the display unit 118 requesting that the toothbrush 10 be photographed in a well-lit location when photographing the toothbrush 10. The photographing guide unit 144 can also display instructions on the display unit 118 requesting that the toothbrush 10 be photographed in a well-lit location when the brightness detected by the imaging unit 116 during photographing or the brightness of the actually photographed image data 152 is below a predetermined value. Alternatively, the photographing guide unit 144 can automatically activate an auto-flash mode that automatically performs flash photography. Conversely, the photographing guide unit 144 can display instructions on the display unit 118 requesting that the toothbrush 10 be photographed in a slightly darker location or that the flash not be used when the brightness detected by the imaging unit 116 during photographing or the brightness of the actually photographed image data 152 is above a predetermined value.

[0049] The notification means by the photographing guide section 144 is not particularly limited, and may display a message or an image on the display section 118, or may use sound, vibration, or the like.

[0050] (Analysis area specification section 146) The analysis area designation unit 146 designates an analysis area AR to be analyzed from the image data 152 captured by the imaging unit 116. For example, after the imaging unit 116 captures an image, the analysis area designation unit 146 causes the display unit 118 to display an instruction I3 requesting the user to designate the area of ​​the brush unit 22 in the captured image using a frame F that allows the user to select an area, as shown in FIG. 6(a). After receiving a user's designation operation for the area, the analysis area designation unit 146 causes the display unit 118 to display an instruction I4 requesting the user to confirm that the designated area properly surrounds the brush unit 22, as shown in FIG. 6(b). The analysis area designation unit 146 designates the area designated by the user (i.e., the area surrounded by the frame F) as the analysis area AR. The analysis area designation unit 146 can extract only the analysis area AR from the original image to set it as an image to be analyzed, and transmit it to the server 200 via the communication unit 120. The analysis area designation unit 146 may transmit both the original image and information on the analysis area AR to the server 200.

[0051] 6(a), when displaying an instruction I3 for requesting designation of an analysis region AR on the display unit 118, the analysis region designation unit 146 may display examples of inappropriate region designations on the display unit 118. For example, the analysis region designation unit 146 may display, as an inappropriate example, a diagram in which part or all of the brush unit 22 is not surrounded by the frame F on the display unit 118. Furthermore, the analysis region designation unit 146 can determine whether the region designation by the user is appropriate based on the recognition result of the image recognition unit 138, etc. If the analysis region designation unit 146 determines that the region designation by the user is inappropriate, it may display, on the display unit 118, a message indicating that the brush unit 22 may not have been properly surrounded.

[0052] The analysis area designation unit 146 may automatically designate an analysis area AR in the captured image based on the recognition result of the image recognition unit 138. In this case, the analysis area designation unit 146 may cause the display unit 118 to display an instruction requesting confirmation of whether the automatically designated analysis area AR is appropriate.

[0053] When the display unit 118 displays the captured image, the processing unit 110 can appropriately adjust the brightness, contrast, color tone, etc. of the image as needed. For example, the processing unit 110 can adjust the tone of the image so that the brush part 22 in the image is easily recognizable.

[0054] [Server 200 configuration] The server 200 is a server device operated by an organization that provides the determination system 1, or the like.

[0055] A specific configuration of the server 200 will be described with reference to Fig. 3. The server 200 includes a processing unit 210, a storage unit 212, and a communication unit 214 as its functional units.

[0056] The processing unit 210 performs predetermined arithmetic processing on the information acquired by the server 200. Specifically, the processing unit 210 reads and executes a program 240 stored in the storage unit 212, thereby performing various processes such as image recognition and estimation.

[0057] In terms of hardware configuration, the processing unit 210, like the processing unit 110, can be configured with, for example, an arithmetic processing unit such as a CPU, an MPU, or a GPU, a ROM, a RAM, and the like.

[0058] Similar to the storage unit 112, the storage unit 212 stores various programs 240 executed by the processing unit 210, various data used for processing in the processing unit 210, etc. Specifically, the storage unit 212 can be configured as a large-capacity storage device such as a hard disk.

[0059] Like program 150, program 240 may be provided in a state recorded on a computer-readable recording medium, may be pre-installed when server 200 is manufactured, may be provided via a network, or may be provided in any other manner.

[0060] The storage unit 212 stores a determination result database 242 that accumulates various data related to each determination. FIG. 7 is a diagram showing an example of the determination result database 242. The determination result database 242 includes columns for "Determination Result ID," "Photographed Image," "Photographing Device Information," "Photographing Date," "Start Date of Use," "Feature Information," "Bristle Opening Degree," "Bristle Opening Threshold," and "Determination Result." The "Determination Result ID" column lists an identifier (ID) for identifying each determination process. The "Photographed Image" column lists an image of the toothbrush 10 and access information for accessing the image. The "Photographing Device Information" column lists information such as the model name and serial number of the user terminal 100 used to photograph the toothbrush 10. The "Photographing Date" column lists the date the image of the toothbrush 10 was photographed. The "Start Date of Use" column lists the date the user started using the toothbrush 10. The "Feature Information" column lists various feature information about the toothbrush 10. The "Bristle Opening Degree" column lists the value of the bristle opening degree estimated from the image of the toothbrush 10. The "bristle opening threshold" field lists the bristle opening threshold of the toothbrush 10 determined from the feature amount information. The "determination result" field lists the determination result based on the bristle opening degree and the bristle opening threshold.

[0061] The "Feature Information" column includes sub-items such as "Product ID," "Head Width," "Bristle Type," and "Bristle Hardness." The "Product ID" column lists the product name and model number that identify the type of toothbrush 10, as well as a code that encodes these. The "Head Width" column lists the width of the head substrate 20. The "Bristle Type" column lists the type of bristles that make up the brush portion 22 (e.g., "Straight" for straight bristles, "Tapered" for tapered bristles, and "Super Tapered" for super tapered bristles). The "Bristle Hardness" column lists the hardness of the bristles that make up the brush portion 22. In FIG. 7, "Bristle Hardness" is listed as a qualitative two-level rating of "Firm" and "Soft," but it may also be listed as a quantitative value or a display based on the "Bristle Hardness" standard of JIS S3016 (Toothbrushes).

[0062] The "Determination Result" column includes the sub-items "Replacement Necessary," "Usable Period," and "Instruction Information." The "Replacement Necessary" column lists the determination result of whether or not toothbrush 10 needs to be replaced ("Needed" or "Not Required"). The "Usable Period" column lists an estimated value for the period during which toothbrush 10 can be continuously used if toothbrush 10 does not need to be replaced. The "Instruction Information" column lists instruction information, such as advice to the user on how to brush their teeth, generated based on the image recognition results, the degree of bristles opening, feature information, etc.

[0063] 3, the communication unit 214 communicates with other devices via the network N. Specifically, the communication unit 214 transmits and receives data to and from other devices (for example, the user terminal 100).

[0064] (Functional configuration of processing unit 210) Next, the functional configuration of the processing unit 210 will be described with reference to Fig. 3. The processing unit 210 includes, as its functional units, an image acquisition unit 220, a feature amount information acquisition unit 222, an image recognition unit 224, an image correction unit 226, a hair opening degree estimation unit 228, a hair opening threshold acquisition unit 230, a determination unit 232, a usable period estimation unit 234, and a guidance information generation unit 236. These functional units are realized by the processing unit 210 reading out a program 240 stored in the storage unit 212 and executing the program.

[0065] (Image acquisition unit 220) The image acquisition unit 220 acquires the image data 152 (image data 244) received by the communication unit 214 from the user terminal 100. The image acquisition unit 220 stores the image data 244 in the storage unit 212. The image acquisition unit 220 writes information about the storage destination of the image data 244 and information about the shooting date to the determination result database 242 stored in the storage unit 212, as shown in FIG.

[0066] (Feature information acquisition unit 222) The feature amount information acquisition unit 222 acquires the feature amount information 154 received by the communication unit 214 from the user terminal 100, together with information on the start date of use of the toothbrush 10. The feature amount information acquisition unit 222 writes the acquired feature amount information 154 and information on the start date of use into the determination result database 242 stored in the storage unit 212, as shown in Fig. 7. Note that the feature amount information acquisition unit 222 writes only the acquired feature amount information 154 into the determination result database 242.

[0067] (Image recognition unit 224) Image recognition unit 224 performs image recognition of image data 244. Like image recognition unit 138, image recognition unit 224 can use any image recognition method to recognize the position, orientation, size, shape, etc. of toothbrush 10 and each of its components in the image.

[0068] (Image correction unit 226) The image correction unit 226 corrects the image data 244 to assist in image recognition and analysis. Specifically, the image correction unit 226 can correct distortions and tilts of the toothbrush 10 in the image. For example, the image correction unit 226 can correct the image data 244 based on distortions in the contours of the head 12 and neck 16 in the image so that the brush head 22 faces forward when viewed from the imaging unit 116. For example, the image correction unit 226 can correct the image data 244 based on correction data such as 3D model data of the toothbrush 10 stored in the storage unit 212. The image correction unit 226 can select the correction data to use based on feature information of the toothbrush 10. Note that the image correction method is not limited to the above example and can be performed by any means, such as machine learning.

[0069] The image correction unit 226 can also adjust the brightness, contrast, and tone of the image. For example, the image correction unit 226 adjusts the brightness, contrast, and tone of the image so that the bristles of the brush unit 22 are emphasized in the image. Specifically, the image correction unit 226 corrects the image data 244 so that the vicinity of the bristles of the brush unit 22 is darkened and the background is brightened, thereby increasing the difference in brightness and contrast between the bristles of the brush unit 22 and other areas. Conversely, the image correction unit 226 may adjust the vicinity of the bristles of the brush unit 22 to be brighter and the other areas to be darker. Furthermore, the image correction unit 226 may emphasize the bristles of the brush unit 22 in the image by changing the color tone of the image data 152 (e.g., to monotone) or by changing the tone to watercolor, sketch, illustration, or the like. Such image correction can improve the accuracy of analyzing the bristles opening state of the brush unit 22.

[0070] (Hair opening degree estimation unit 228) Bristle opening degree estimation unit 228 estimates the degree to which the bristles of toothbrush 10 are open by analyzing image data 244. In this specification, an index representing the degree to which the bristles of toothbrush 10 are open is referred to as the "bristle opening degree."

[0071] FIG. 8 is a diagram showing an example of a method for estimating the bristle opening degree by the bristle opening degree estimation unit 228 of the first embodiment. FIG. 8 shows image data 244 of the toothbrush 10. As described above, the image recognition unit 224 recognizes the toothbrush 10 and the position and shape of each component of the toothbrush 10 in the image. Based on the recognition result of the image recognition unit 224, the bristle opening degree estimation unit 228 obtains the maximum width d1 of the brush head 22 in the width direction (y direction) of the brush head 22 of the toothbrush 10 when viewed from above, and the maximum width d2 of the head base material 20 in the width direction (y direction). Note that the bristle opening degree estimation unit 228 may obtain the maximum width d2 of the head base material 20 from feature amount information of the toothbrush 10. An example of the "bristle opening degree" is the value d1 - d2. This value corresponds to the width of the brush head 22 that extends beyond the head base material 20 in the width direction.

[0072] The bristle opening degree estimation unit 228 may use, instead of d1-d2, the distance in the y direction from the end of the head base material 20 on either the left or right side to the end of the brush part 22 as the "bristle opening degree." The bristle opening degree estimation unit 228 may also obtain the minimum width d3 of the neck 16 in the width direction (y direction) and use the value of d1-d3 as the "bristle opening degree" instead of d1-d2. A similar calculation may also be performed in the length direction (x direction) instead of the width direction (y direction).

[0073] Alternatively, if initial values ​​for the dimensions of the brush head 22 and the like are given, the bristle opening degree estimation unit 228 can compare current values, such as the dimensions of the current toothbrush 10, acquired from the image data 244 with the initial values. The bristle opening degree estimation unit 228 can use the difference between the initial values ​​and the current values ​​as the "bristle opening degree." For example, if the feature information acquisition unit 222 acquires the maximum width d0 in the y direction of the brush head 22 of the toothbrush 10 before use as the feature information of the toothbrush 10, the bristle opening degree estimation unit 228 can use d1-d0 as the "bristle opening degree." This value corresponds to the amount of change in the width of the brush head 22 before and after use of the toothbrush 10.

[0074] The hair opening degree estimation unit 228 writes the value of the hair opening degree estimated from the image data 244 into the determination result database 242 stored in the storage unit 212.

[0075] The bristle opening degree estimation unit 228 may estimate the bristle opening degree for a portion of the brush part 22 in the image. For example, the bristle opening degree estimation unit 228 can divide the brush part 22 in the image into a plurality of sections, such as top, bottom, left, and right, based on the recognition result of the image recognition unit 224, and estimate the bristle opening degree for each section. If the brush part 22 is made up of two or more types of bristle bundles B, the bristle opening degree estimation unit 228 can divide the brush part 22 in the image into a plurality of sections according to the type of bristle bundle B, and estimate the bristle opening degree for each section.

[0076] (Hair opening threshold acquisition unit 230) The bristle opening threshold acquisition unit 230 (an example of a "reference information acquisition unit") acquires a bristle opening threshold (an example of "reference information") based on feature amount information of the photographed toothbrush 10. Here, the "bristle opening threshold" is a threshold for the degree of bristle opening used as a criterion for determining whether the toothbrush 10 should be replaced. Preferably, the bristle opening threshold is a quantitative value comparable to the degree of bristle opening estimated by the bristle opening degree estimation unit 228. For example, when the bristle opening degree estimation unit 228 estimates d1-d0 as the degree of bristle opening as described above, the bristle opening threshold acquisition unit 230 can acquire a value corresponding to d1-d0 as the bristle opening threshold. For example, the bristle opening threshold may be a value indicating that the toothbrush 10 should be replaced when the degree of bristle opening exceeds the bristle opening threshold, and indicating that the toothbrush 10 does not yet need to be replaced when the degree of bristle opening is equal to or less than the bristle opening threshold.

[0077] The bristle opening threshold corresponds to the feature information of the toothbrush 10 and can vary depending on attributes such as the dimensions, shape, and material of the toothbrush 10. Therefore, by using the optimal bristle opening threshold for each toothbrush 10 as a judgment criterion, it becomes possible to objectively determine when to replace the toothbrush 10 according to its attributes.

[0078] For example, the bristle spread threshold acquisition unit 230 can acquire the bristle spread threshold by referring to a bristle spread threshold table 246 stored in the storage unit 212. FIG. 9 is a diagram showing an example of the bristle spread threshold table 246. As shown in FIG. 9, the bristle spread threshold table 246 is a table that associates various feature information with bristle spread thresholds. As an example, the bristle spread threshold table 246 in FIG. 9 associates a combination of three feature items, "head base material width," "bristle type," and "bristle hardness" (each of which is an example of "information indicating the attributes of the toothbrush head") with a bristle spread threshold. For example, if the "head base material width" is 7 mm, the "bristle type" is "super tapered," and the "bristle hardness" is "firm," the bristle spread threshold is 2 mm.

[0079] The correspondence in the bristle opening threshold table 246 is based on the following criteria, for example. (a) If the width of the head base material is 7 mm, it is determined that it is time to replace the brush head when the bristles of the brush part 22 are spread apart by a total of 4 mm on both sides (that is, the bristles spread threshold is set to 4 mm). (b) If the width of the head substrate is 13 mm, the bristle opening threshold is set to 2 mm. (c) If the type of hair is super tapered hair, the hair opening threshold is set to 2 mm. (d) If the hair type is straight, the hair opening threshold is set to 5 mm. (e) If the bristle hardness is "hard," the bristle opening threshold is set to 2 mm. (f) If the bristle hardness is "soft," the bristle opening threshold is set to 5 mm.

[0080] Furthermore, if the acquired feature information is a product ID that identifies the type of toothbrush 10, the bristle opening threshold acquisition unit 230 can acquire the bristle opening threshold corresponding to the product ID, for example, based on the bristle opening threshold table 246, which contains information on the correspondence between product IDs and bristle opening thresholds. In this case, the above-mentioned "information indicating the attributes of the toothbrush head" is not necessarily required, and the bristle opening threshold acquisition unit 230 can acquire the bristle opening threshold directly based on the product ID. Note that if the product ID is unknown, the bristle opening threshold acquisition unit 230 may first estimate product information such as the product ID of the toothbrush 10 from feature information such as the "information indicating the attributes of the toothbrush head." In this case, the bristle opening threshold acquisition unit 230 can acquire the bristle opening threshold corresponding to the estimated product ID by referring to the bristle opening threshold table 246, which contains information on the correspondence between product IDs and bristle opening thresholds.

[0081] The method for acquiring the hair opening threshold value based on the feature amount information is not limited to the above example. For example, the hair opening threshold value acquisition unit 230 may use any algorithm such as a decision tree or a support vector machine. The hair opening threshold value acquisition unit 230 may also use a trained model that has trained the relationship between the feature amount and the hair opening threshold value using a neural network.

[0082] The bristle opening threshold acquisition unit 230 writes the acquired bristle opening threshold into the determination result database 242.

[0083] When the bristle opening degree estimation unit 228 estimates the bristle opening degree for each section of the brush unit 22, the bristle opening threshold acquisition unit 230 can also acquire the bristle opening threshold corresponding to each section.

[0084] (Judgment unit 232) The determination unit 232 determines whether the toothbrush 10 should be replaced based on the bristle opening degree and the bristle opening threshold. Specifically, the determination unit 232 compares the bristle opening degree estimated by the bristle opening degree estimation unit 228 with the bristle opening threshold acquired by the bristle opening threshold acquisition unit 230. If the estimated bristle opening degree exceeds the bristle opening threshold, the determination unit 232 determines that the toothbrush 10 should be replaced. On the other hand, if the estimated bristle opening degree is equal to or less than the bristle opening threshold, the determination unit 232 determines that the toothbrush 10 does not yet need to be replaced. The determination unit 232 writes the determination result to the determination result database 242.

[0085] If the estimation of the bristle opening degree and the acquisition of the bristle opening threshold are performed for each section of the brush unit 22, the determination unit 232 can also make the above determination for each section. Even if the brush unit 22 is composed of two or more types of bristle bundles B, the determination unit 232 can make the above determination for each section divided by the type of bristle bundle B.

[0086] (Usable period estimation unit 234) The usable period estimation unit 234 estimates the remaining usable period of the toothbrush 10 when the determination unit 232 determines that replacement is not necessary. For example, the usable period estimation unit 234 can estimate the usable period by subtracting the period of use from the usable period of a typical toothbrush 10 from the date of start of use to the date of photographing. Instead of the typical usable period, the usable period estimation unit 234 may use the usable period corresponding to the feature amount information of the target toothbrush 10. Furthermore, the usable period estimation unit 234 may estimate the usable period by extrapolating the date on which the bristle opening degree will reach the bristle opening threshold based on the date of start of use, the date of photographing, and the estimated bristle opening degree. Any other estimation method may be used.

[0087] The available period estimation unit 234 writes the estimated available period into the determination result database 242 .

[0088] (Guidance information generation unit 236) The instruction information generator 236 generates instruction information for teaching the user how to brush their teeth based on the image data 244 of the toothbrush 10, the estimated degree of bristle opening, the period of use, the feature amount information, etc. Specifically, the instruction information generator 236 estimates problems and areas for improvement in the user's tooth brushing, and outputs instruction information including these.

[0089] The instruction information generation unit 236 can obtain a reference value for the bristles' opening degree predicted from the trial period based on the usage period from the start date of use to the shooting date, and compare this with the bristles' opening degree estimated by the bristles' opening degree estimation unit 228. The reference value for the bristles' opening degree can be associated with the usage period, for example, using a predetermined mathematical formula or a correspondence table. The reference value for the bristles' opening degree can vary depending on the feature information. For example, if the estimated bristles' opening degree is greater than the reference value for the bristles' opening degree predicted from the usage period, the instruction information generation unit 236 generates instruction information indicating the possibility of overbrushing, which may be due to excessive toothbrushing pressure. In such a case, the instruction information generation unit 236 can include messages such as "Your brushing pressure may be too strong" or "The frequency and duration of toothbrushing may also lead to bristles becoming more open" in the instruction information. In addition, the instruction information generation unit 236 may include messages such as "Brushing too hard can damage the gums and mucous membranes of the mouth and cause tooth sensitivity," "Stop brushing while doing other things," "If you have prostheses (such as dentures or implants) or orthodontic appliances, your bristles are more likely to separate. Get instructions on brushing your teeth at a dentist's office," "Have regular dental checkups," and "How to choose a toothbrush and other tools" in the instruction information.

[0090] Conversely, if the estimated degree of bristle opening is smaller than the reference value of the degree of bristle opening expected from the period of use, the instruction information generation unit 236 generates instruction information pointing out that the time or frequency of tooth brushing may be insufficient. For example, the instruction information generation unit 236 may include messages such as "The results may vary depending on the number of times and time you brush your teeth" or "Make it a habit to brush your teeth three times a day (in Japan)" in the instruction information.

[0091] If the image recognition unit 224 recognizes that the bristles are only spread locally or that there is a bias in the spread of the bristles on the left and right sides of the brush head 22, the instruction information generation unit 236 can generate instruction information pointing out that the brushing is not being done evenly. For example, the instruction information generation unit 236 can include messages such as "Are you brushing the entire toothbrush evenly?", "Brush the entire toothbrush thoroughly while looking in the mirror," or "Stop brushing while doing other things" in the instruction information. The instruction information generation unit 236 can also generate more detailed instruction information depending on the spread of the bristles. For example, if the spread of the bristles is particularly large on the tip or handle side of the brush head 22, the instruction information generation unit 236 can include a message such as "Make sure you are brushing the occlusal surface (bite) thoroughly as well" in the instruction information.

[0092] When the hair opening degree estimation unit 228 estimates the hair opening degree for each section, even if the hair opening degree differs greatly from section to section, the instruction information generation unit 236 can generate instruction information that points out that the hair is not brushed evenly, regardless of the judgment result of the judgment unit 232.

[0093] When the image recognition unit 224 recognizes that the root of the bristle bundle B is colored in comparison with the image of a new toothbrush, the instruction information generation unit 236 can include in the instruction information a message such as "Your toothbrush may be dirty. Clean it or replace it with a new one." The instruction information generation unit 236 may also include in the instruction information the risks of continuing to use a dirty toothbrush (for example, the risk of bacteria breeding or infection or pneumonia).

[0094] The instruction information generating unit 236 may use the results of a questionnaire previously entered by the user. For example, the user terminal 100 can display on the display unit 118 a questionnaire containing information such as the number of times and duration of daily tooth brushing, whether or not the user wears prostheses or orthodontics, age, gender, oral symptoms (e.g., toothache, tooth sensitivity, gum pain, loose feeling, swelling, overgrown teeth, dry mouth, bad breath), number of teeth, oral frailty (users 40 years of age or older may be particularly requested to enter information regarding oral frailty), and toothbrushing method preferences (e.g., preferred toothbrush 10 hardness, preferred head 12 size, and preference for scrubbing or fine brushing), and accept user input. The user terminal 100 transmits the results of the questionnaire entered by the user to the server 200. The instruction information generating unit 236 of the server 200 can include a message based on the results of the questionnaire in the instruction information. The instruction information generating unit 236 may include other oral check results (saliva test, bite force, dry mouth, tongue and lip motor function test, tongue coating amount, tongue pressure, masticatory efficiency, swallowing function, etc.) in the instruction information.

[0095] [Data flow] Next, the flow of data in the determination system 1 will be described with reference to Fig. 10. Fig. 10 is a flow diagram showing the flow of data in the determination system 1.

[0096] 10, first, the user terminal 100 acquires image data 152 of the toothbrush 10. The server 200 estimates the degree of bristles opening based on the image data 152 (244).

[0097] Meanwhile, the user terminal 100 acquires feature amount information 154 of the toothbrush 10 based on user input or image recognition of the image data 152. The server 200 determines the bristle opening threshold based on the acquired feature amount information 154 and association information such as the bristle opening threshold table 246.

[0098] The user terminal 100 acquires the photographing date from the date and time information when the image data 152 was acquired. The user terminal 100 also acquires the date on which the toothbrush 10 was first used, based on an input by the user or the like.

[0099] Server 200 generates a determination result as to whether toothbrush 10 should be replaced, a usable period of toothbrush 10, and guidance information for the user based on the estimated bristle opening degree, the determined bristle opening threshold, and information on the photographing date and the start date of use. The guidance information may be based not only on the bristle opening degree and the bristle opening threshold, but also on the image recognition results and feature amount information of image data 152.

[0100] [Determination method flow] Next, the processing flow of the determination system 1 will be described with reference to Fig. 11 and Fig. 12. Fig. 11 is a flow diagram showing the determination method by the determination system 1 according to the first embodiment. Fig. 12 is a flow diagram showing the processing flow by the determination system 1 according to the first embodiment.

[0101] First, with reference to Figure 11, the overall flow of the determination method by the determination system 1 will be outlined. As shown in Figure 11, in the determination method of the determination system 1, one or more processors of the determination system 1 execute the processes of steps S1101 to S1105. The one or more processors include the processors of the user terminal 100 and / or the server 200. A single processor may execute all the processes, or multiple processors may execute the processes in a distributed manner. Hereinafter, when it is said that "a processor executes...", the entity executing the processes may be a single processor or multiple processors.

[0102] In step S1101, the processor acquires an image of the toothbrush 10. In step S1102, the processor acquires feature information 154 of the toothbrush 10. In step S1103, the processor acquires reference information, which is information corresponding to the toothbrush 10 and is information related to the criteria for determining whether or not the toothbrush 10 should be replaced, based on the feature information 154. For example, the reference information is a bristle opening threshold. In step S1104, the processor determines whether or not the toothbrush 10 should be replaced based on the image and the reference information. For example, the processor determines whether or not the toothbrush 10 should be replaced by comparing the bristle opening degree estimated from the image with the bristle opening threshold, which is the reference information. In step S1105, the processor outputs information including the determination result.

[0103] Next, a specific processing flow will be described with reference to FIG. 12. As shown in FIG. 12, first, in step S1201, the user terminal 100 accepts input of feature information 154 from the user. As a result, the feature information acquisition unit 132 acquires the feature information 154 of the toothbrush 10 to be evaluated. For example, the feature information acquisition unit 132 can display a product list (including brand, model number, head size, bristle type, bristle hardness, etc.) on the display unit 118 and request the user to select an item or enter an input. Alternatively, the feature information acquisition unit 132 can display an instruction on the display unit 118 requesting the user to read a barcode or two-dimensional code printed on the product or package. In this case, the feature information acquisition unit 132 can acquire the feature information 154 based on the code read by the user terminal 100 and association information including a correspondence between the code and the feature information. Alternatively, the feature information acquisition unit 132 can acquire the barcode or two-dimensional code itself as the feature information 154 of the toothbrush 10.

[0104] In step S1202, the user terminal 100 receives input of the start date of use of the toothbrush 10 from the user.

[0105] In step S1203, an image capturing application of the user terminal 100 is started, enabling image capturing by the imaging unit 116. The guide frame determination unit 134 determines the guide frame G to be used based on the feature amount information 154. The display control unit 136 displays the guide frame G on the image capturing screen as shown in FIG. 4(a), thereby assisting the user in capturing an image of the toothbrush 10.

[0106] In step S1204, image recognition unit 138 performs image recognition on the captured image of toothbrush 10, and orientation estimation unit 140 estimates the orientation of toothbrush 10. Such image recognition and orientation estimation may be performed on a still image captured by imaging unit 116, but are preferably performed in real time while the user is being photographed. In step S1205, determination unit 142 determines whether the estimated orientation of toothbrush 10 is within an acceptable range.

[0107] In step S1206, based on the determination result by the determination unit 142, the photographing guide unit 144 guides the photographing of the toothbrush 10. For example, if the determination unit 142 determines that the orientation of the toothbrush 10 is inappropriate, the photographing guide unit 144 notifies the user of this and instructs the user to correct the orientation, as shown in FIG.

[0108] In step S1207, the imaging unit 116 captures an image of the toothbrush 10. The image acquisition unit 130 acquires image data 152 of the toothbrush 10. The analysis area designation unit 146 may designate an analysis area AR in the image. In step S1208, the image acquisition unit 130 acquires the image capture date in association with the image data 152.

[0109] In step S1209, the communication unit 120 transmits the feature amount information 154, the information on the start date of use, the image data 152, and the information on the shooting date to the server 200 via the network N. The communication unit 214 of the server 200 receives this information from the communication unit 120. In step S1210, the image acquisition unit 220 acquires the image data 152 (244) and the information on the shooting date received from the user terminal 100, and writes these to the determination result database 242. In step S1211, the feature amount information acquisition unit 222 acquires the feature amount information 154 and the information on the start date of use received from the user terminal 100, and writes these to the determination result database 242.

[0110] In step S1212, the image correcting unit 226 corrects the image data 244 as necessary. Specifically, the image correcting unit 226 can correct the image data 244 so that the brush unit 22 faces forward when viewed from the imaging unit 116. The image correcting unit 226 can also adjust the brightness, contrast, tone, etc. of the image data 244 so that the tip of the brush unit 22 can be clearly distinguished from other parts of the image.

[0111] In step S1213, the image recognition unit 224 performs image recognition on the image data 244. For example, the image recognition unit 224 can recognize the position, size, contour shape, etc. of the head base material 20 and the brush part 22 in the image.

[0112] In step S1214, the bristle opening degree estimation unit 228 estimates the bristle opening degree based on the recognized positions, sizes, contour shapes, etc. of the head base material 20 and the brush part 22. For example, the bristle opening degree estimation unit 228 measures dimensions d1, d2, d3, etc. shown in Fig. 8 and writes d1-d2 and d1-d3 into the determination result database 242 as the bristle opening degree.

[0113] In step S1215, the hair spread threshold acquisition unit 230 acquires a hair spread threshold based on the feature amount information. For example, the hair spread threshold acquisition unit 230 refers to the hair spread threshold table 246 to acquire a hair spread threshold corresponding to the feature amount information, and writes the acquired hair spread threshold to the determination result database 242.

[0114] In step S1216, determination unit 232 compares the degree of bristle opening with a bristle opening threshold to determine whether or not to replace the toothbrush, and writes the determination result in determination result database 242. In step S1217, usable period estimation unit 234 estimates the remaining period for which toothbrush 10 can be used based on information on the start date of use and the date of photography, the degree of bristle opening, feature amount information, etc., and writes the estimated usable period in determination result database 242. In step S1218, instruction information generation unit 236 generates toothbrushing instruction information for the user based on information on the start date of use and the date of photography, the degree of bristle opening, feature amount information, etc., and writes the generated instruction information in determination result database 242.

[0115] In step S1219, the communication unit 214 transmits the determination result of the determination unit 232, the estimated usable period, and the generated guidance information to the user terminal 100 via the network N. In step S1220, the display control unit 136 displays the obtained determination result, usable period, and guidance information on the display unit 118 to notify the user. The user can check this information using application software or a web browser on the user terminal 100.

[0116] [Effects of Judgment System 1] The assessment system 1 configured as described above allows for the use of variable reference information (e.g., bristle opening threshold) based on feature information of the toothbrush 10, enabling objective assessment of bristle opening tailored to the characteristics of each toothbrush 10. In contrast, with conventional assessment techniques, for example, when the head substrate 20 is relatively wide, it is not immediately apparent that the bristles are open unless the bristles are significantly open. Therefore, visual assessment may delay replacement of the toothbrush 10. On the other hand, when the head substrate 20 is relatively narrow, even a slight bristle opening can appear to be wide, so visual assessment may lead to earlier replacement. Furthermore, when the brush head 22 is composed of tapered bristles, the tips of tapered bristles tend to bend more easily than the roots, so visual assessment tends to underestimate the bristles' opening. Therefore, visual assessment may delay replacement of the toothbrush 10.

[0117] According to this embodiment, the feature information may include information indicating the attributes of the head 12 of the toothbrush 10. Specifically, the information indicating the attributes of the toothbrush head may include information regarding one or more of the attributes of the base of the head and the attributes of the bristles of the head. By using information indicating the attributes of the head 12 (e.g., the width of the head base material 20, the type of bristles, etc.) that particularly have a significant impact on bristle opening as feature information of the toothbrush 10, the determination system 1 can accurately determine when to replace the toothbrush 10 in accordance with the characteristics of the individual toothbrush 10.

[0118] The determination system 1 according to this embodiment may further include a bristle opening degree estimation unit 228 that estimates the bristle opening degree, which indicates the degree to which the toothbrush bristles are open, based on an image. The determination unit 232 may determine whether the toothbrush 10 should be replaced by comparing the bristle opening degree with reference information. Specifically, the bristle opening degree may be a value calculated based on geometric information of the bristles of the toothbrush 10 in the image. This allows the determination system 1 to estimate the bristle opening degree from the image (e.g., from geometric information of the bristles, such as the width of the brush head 22 and the angle of the bristles in the image), enabling a more objective determination. The reason for the determination is also clear to the user. On the other hand, there are methods for determining when a photographed toothbrush 10 needs to be replaced using a trained model trained on a large number of images of various types of toothbrushes that need replacing. However, the accuracy of the determination significantly decreases when a product that has not been trained beforehand is used. The accuracy of the determination also decreases when the photographing conditions of the toothbrush 10 differ from those of the training images.

[0119] According to this embodiment, the bristles opening degree estimation unit 228 can select the geometric information used to calculate the bristles opening degree in accordance with the feature amount information. This allows the bristles opening degree estimation unit 228 to select the geometric information of the bristles that serves as the basis for calculating the bristles opening degree in accordance with the type of bristles, etc. For example, if the brush unit 22 is configured with super tapered bristles as described above, the bristles opening degree estimation unit 228 can estimate the bristles opening degree for the upper one-third of the bristles. In this way, the bristles opening degree estimation unit 228 can adopt the optimal estimation method depending on the type of bristles.

[0120] According to this embodiment, the bristle opening degree estimation unit 228 can estimate the bristle opening degree for each different part of the toothbrush 10. This allows the determination system 1 to detect unevenness in the user's tooth brushing habits. Furthermore, even when the brush head 22 is composed of two or more types of bristle bundles B, the determination system 1 can perform determination processing for each part of the toothbrush 10 according to the type of bristle bundle B.

[0121] According to this embodiment, the bristle opening threshold acquisition unit 230 (an example of a "reference information acquisition unit") can acquire a bristle opening threshold corresponding to the feature amount information as reference information. The determination unit 232 can determine whether the toothbrush 10 should be replaced by comparing the bristle opening degree with the bristle opening threshold. In this way, by using the bristle opening threshold, which can be numerically compared with the bristle opening degree, as reference information, a more objective determination becomes possible. Furthermore, if the reference information is simply numerical data, the calculation load on the determination system 1 can be significantly reduced compared to when the reference information is image data or the like.

[0122] The determination system 1 according to this embodiment may further include an instruction information generation unit that generates instruction information, including recommended toothbrushing methods, based on the degree of bristle opening. This allows not only determining when to replace the toothbrush from the degree of bristle opening, but also providing advice and instruction on toothbrushing based on the asymmetry of the degree of bristle opening. This allows for efficient use of acquired data, improving the usability of the determination system 1. In contrast, when only determining when to replace the toothbrush, even if the bristles are open due to poor brushing habits, the user may not recognize the underlying cause. Even if a dental clinic provides instructions, infrequent visits often result in poor brushing habits.

[0123] The determination system 1 according to this embodiment may include a guide frame determination unit 134 that determines, based on feature information, the type of guide frame G to assist the user terminal 100, which includes an imaging unit 116 that captures the toothbrush 10, in capturing an image of the toothbrush 10. The determination system 1 may also include a display control unit 136 that displays the guide frame G on the display unit 118 of the user terminal 100. The determination unit 232 may also determine whether the orientation of the toothbrush 10 in the image of the toothbrush 10 is appropriate. This provides appropriate guidance for the user's capturing operation, improving user convenience. In contrast, previous toothbrush determination technologies often lacked sufficient capture guidance, preventing users from understanding the detailed capture method. For example, they did not clearly explain how to capture an image suitable for analysis or how to specify an analysis area, leaving the user to make their own judgment. Furthermore, determination accuracy was low when the brightness, focus conditions, background, etc., during capture were inappropriate. The determination system 1 can improve determination accuracy by appropriately guiding the user's capture regarding such capture conditions.

[0124] Second Embodiment Next, a determination system 1 according to a second embodiment will be described with reference to Figures 13 to 15. The second embodiment differs from the first embodiment in that an image of the toothbrush 10 before use is used to estimate the degree of bristle opening. Below, differences from the above embodiment will be mainly described, and explanations of points in common with the above embodiment will not be repeated.

[0125] 13 and 14 are diagrams showing an example of a method for estimating the hair opening degree by the hair opening degree estimation unit 228 of the second embodiment. Fig. 15 is a flowchart showing the flow of processing by the determination system 1 according to the second embodiment.

[0126] In the second embodiment, as shown in FIG. 15 , the user terminal 100 requests the user to photograph the toothbrush 10 before using the toothbrush 10. When the user photographs an image of the toothbrush 10 before use as a first image, the image acquisition unit 130 acquires image data 152 of the toothbrush 10 before use as first image data 152a (step S1501). The image acquisition unit 130 transmits the first image data 152a to the server 200 via the communication unit 120. In addition, in step S1508, the image acquisition unit 130 acquires image data of the toothbrush 10 currently being used as second image data 152b and transmits it to the server 200 via the communication unit 120. As a result, the image acquisition unit 220 of the server 200 acquires the first image data 152a (244a) and the second image data 152b (244b) (step S1511).

[0127] Other processes in the user terminal 100 may be similar to those in the first embodiment, but may also use the first image data 152a of the toothbrush 10 before use acquired in step S1501 as appropriate. For example, when guiding the photographing of the toothbrush 10 in step S1507, the photographing guide unit 144 may execute photographing guidance so that the imaging unit 116 photographs the toothbrush 10 in the same orientation, size, etc. as the image of the toothbrush 10 before use.

[0128] In the server 200, the image recognition unit 224 may perform image recognition and the image correction unit 226 may perform image correction on both the first image data 244a and the second image data 244b (steps S1513 and S1514). In step S1515, the hair opening degree estimation unit 228 may estimate the hair opening degree by using both the two image data 244a and 244b.

[0129] Estimation of the bristle opening degree using the first image data 152a of the toothbrush 10 before use will be described in detail with reference to FIGS. 13 and 14. The bristle opening degree estimation unit 228 compares the first image data 244a and the second image data 244b to estimate the degree to which the bristles of the brush head 22 are opened before and after use as the "bristle opening degree." Specifically, the image correction unit 226 can correct the first image data 244a and the second image data 244b so that the position, orientation, size, etc. of the toothbrush 10 in the first image data 244a and the second image data 244b are aligned. This allows the bristle opening degree estimation unit 228 to compare the first image data 244a and the second image data 244b.

[0130] 13 and 14, the position of the bristle bundle B1 in the first image data 244a is indicated by a dashed line, and the position of the bristle bundle B2 in the second image data 244b is indicated by a solid line. For example, as shown in FIG. 13, when the position, orientation, and size of the toothbrush 10 in the first image data 244a and the second image data 244b are aligned, the bristle opening degree estimation unit 228 can estimate the "bristle opening degree" as the widthwise distance d4 between the rightmost end of the bristle bundle B1 in the first image data 244a and the rightmost end of the bristle bundle B2 in the widthwise direction (y direction) in the second image data 244b. The bristle opening threshold table 246 can include a value corresponding to d4 as the "bristle opening threshold." In this case, the "bristle opening threshold" is a value indicating the extent to which the rightmost position of the brush head 22 is allowed to open to the right from its pre-use position.

[0131] Alternatively, as shown in Fig. 14, the bristle opening degree estimation unit 228 can estimate the "bristle opening degree" as the area S1 of the region connecting the right end of each bristle bundle B1 forming the right edge of the brush head 22 in the first image data 244a with the right end of the corresponding bristle bundle B2 in the second image data 244b, with the position, orientation, and size of the toothbrush 10 aligned between the first image data 244a and the second image data 244b. The bristle opening threshold table 246 can include a value corresponding to S1 as the "bristle opening threshold." In this case, the "bristle opening threshold" is a value indicating the extent to which the right edge of the brush head 22 is allowed to open to the right from its pre-use position.

[0132] As described above, in the second embodiment, the image acquisition units 130, 220 acquire an image of the toothbrush 10 before use as a first image and an image of the toothbrush 10 after use as a second image, and the bristle opening degree estimation unit 228 estimates the bristle opening degree by comparing the first image with the second image. This allows the bristle opening degree estimation unit 228 to acquire the bristle opening degree to which the bristle bundles B are actually opened before and after use. This allows the bristle opening degree to be estimated with higher accuracy than when the width of the head base material 20 or the like is used as a reference. This can improve the accuracy of the determination by the determination unit 232.

[0133] <Third embodiment> Next, a determination system 1 according to a third embodiment will be described with reference to Figs. 16 to 18. The third embodiment differs from the first embodiment in that an image captured from the tip side of the toothbrush 10 is used to estimate the bristle opening degree. Below, differences from the above embodiments will be mainly described, and explanations of points in common with the above embodiments will not be repeated.

[0134] 16 to 18 are diagrams showing an example of a method for estimating the hair opening degree by the hair opening degree estimation unit 228 of the third embodiment. The flow of operation of the determination system 1 is basically the same as that of the first embodiment (FIG. 12) and the second embodiment (FIG. 15).

[0135] In the third embodiment, instead of or in addition to photographing the brush head 22 from above, the user terminal 100 requests the user to photograph the toothbrush 10 from the tip side of the head 12 as shown in Fig. 16. When the user photographs the toothbrush 10 from the tip side of the head 12, the image acquisition unit 130 acquires an image of the toothbrush 10 as seen from at least the tip side of the toothbrush 10. In addition to photographing the toothbrush 10 from the tip side, the image acquisition unit 130 may also acquire an image of the toothbrush 10 photographed from above the brush head 22, as in the first and second embodiments.

[0136] As shown in FIG. 16 , the bristle opening degree estimation unit 228 estimates the bristle opening degree based on an image taken from the tip side. For example, if the image recognition unit 224 measures the width d1 of the brush head 22 and the width d2 of the head substrate 20 shown in FIG. 16 , the bristle opening degree estimation unit 228 can use d1-d2 as the bristle opening degree based on the width of the head substrate 20, as in the first and second embodiments. If only an image taken from above the brush head 22 is used, the head substrate 20 will be located behind the brush head 22, and the measurement accuracy of the width d2 of the head substrate 20 may be insufficient. However, using an image taken from the tip side of the toothbrush 10 as shown in FIG. 16 can improve the measurement accuracy of d1 and d2, thereby improving the determination accuracy of the determination unit 232.

[0137] Alternatively, as shown in FIG. 16 , the bristle opening degree estimation unit 228 can generate a straight line L connecting the tip of the bristle bundle B at the right end of the brush unit 22 and the base of the bristle bundle B on the bristle implantation surface 24. The bristle opening degree estimation unit 228 can estimate the angle α between the line L and a line perpendicular to the bristle implantation surface 24 as the "bristle opening degree." The bristle opening threshold table 246 can include a value corresponding to α as the "bristle opening threshold." In this case, the "bristle opening threshold" is a value that indicates to what extent the bristle bundle B at the right end of the brush unit 22 is allowed to tilt to the right with respect to the line perpendicular to the bristle implantation surface 24.

[0138] As in the second embodiment, the user terminal 100 may request the user to take an image of the toothbrush 10 before using the toothbrush 10. In this case, the bristle opening degree estimation unit 228 can estimate the bristle opening degree by comparing first image data 244a obtained by photographing the toothbrush 10 before use with second image data 244b obtained by photographing the toothbrush 10 currently being used, as shown in Figures 17 and 18.

[0139] FIG. 17 illustrates a case where the bristles constituting the brush head 22 are straight bristles. The position of the bristle bundle B1 in the first image data 244a is indicated by a dashed line, and the position of the bristle bundle B2 in the second image data 244b is indicated by a solid line. As in the second embodiment, the bristle opening degree estimation unit 228 can compare the first image data 244a and the second image data 244b after aligning the position, orientation, and size of the toothbrush 10. For example, as shown in FIG. 17, the bristle opening degree estimation unit 228 generates a straight line L1 connecting the tip of the rightmost bristle bundle B1 in the first image data 244a to the base of the bristle bundle B on the bristle implantation surface 24. The bristle opening degree estimation unit 228 also generates a straight line L2 connecting the tip of the rightmost bristle bundle B1 in the second image data 244b to the base of the bristle bundle B on the bristle implantation surface 24. The bristle opening degree estimation unit 228 can estimate the angle β between the lines L1 and L2 as the "bristle opening degree." The bristle opening threshold table 246 can include a value corresponding to β as the "bristle opening threshold." In this case, the "bristle opening threshold" is a value indicating to what extent the bristle bundle B at the right end of the brush head 22 is allowed to tilt to the right from its position before use.

[0140] FIG. 18 shows a case where the bristles constituting the brush unit 22 are super tapered bristles. Compared to straight bristles, super tapered bristles are tapered, and the tip of the hair bundle B is more likely to bend than the base. Therefore, in the example shown in FIG. 18, unlike FIG. 17, the hair opening degree estimation unit 228 represents the hair opening degree as the inclination γ of the tip of the hair bundle B. Specifically, when the height of hair bundle B is h, the hair opening degree estimation unit 228 can estimate the hair opening degree based on, for example, only the region h / 3 from the tip of hair bundle B. As shown in FIG. 18, the hair opening degree estimation unit 228 generates a straight line L1' connecting the tip of the rightmost hair bundle B1 in the first image data 244a to the right end position of the hair bundle B at a height of 2h / 3. The hair opening degree estimation unit 228 also generates a straight line L2' that connects the tip of the rightmost hair bundle B1 in the second image data 244b to the right end position of the hair bundle B at a height of 2h / 3. The hair opening degree estimation unit 228 can estimate the angle γ formed by the lines L1' and L2' as the "hair opening degree."

[0141] In this way, the bristle opening degree estimation unit 228 can estimate the degree of bristle opening based on an image of the toothbrush 10 viewed from the tip side of the toothbrush 10. This makes it possible to determine the position at which the bristle bundle B begins to bend. Therefore, the determination system 1 according to this embodiment is very effective for determining the bristle opening of super tapered bristles, which tend to bend primarily at the tip. In contrast, when making a determination based only on an image from above, such super tapered bristles are relatively difficult to bend in the region near the base, so the amount of bristle opening may be underestimated.

[0142] In Figures 17 and 18, an example is described in which the inclination of bristle bundle B is used as the bristle opening degree, but as in the second embodiment, the bristle opening degree may also be estimated based on the distance traveled by bristle bundle B before and after use, or the area traveled by the group of bristle bundles B that make up the right edge of the brush part 22.

[0143] The bristle opening degree estimation unit 228 may estimate the bristle opening degree by combining an image taken from above the brush head 22 and an image taken from the tip side of the toothbrush 10. For example, the accuracy of the bristle opening degree estimation can be improved by estimating and comparing both the bristle opening degree based on an image taken from above and an image taken from the tip side. In particular, when there is information that cannot be clearly understood from one image alone, for example, because the head base material 20 and the brush head 22 are similar colors and difficult to distinguish, estimation based on images taken from multiple angles is effective. This can improve the accuracy of the determination by the determination unit 232.

[0144] The bristle opening degree estimation unit 228 may use an image captured from another direction (for example, from the side of the brush unit 22) to estimate the bristle opening degree.

[0145] Each functional unit such as the input unit, processing unit, memory unit, and output unit described in the above example is realized by the cooperation of a hardware configuration including a processor, memory, storage, input / output interface, communication interface, and buses that interconnect these, which are provided in the information processing device.

[0146] In the above embodiment, the processing is described as being performed by a single server 200, but the processing may be distributed among a plurality of servers 200. Also, part of the processing described as being performed by the user terminal 100 may be performed by the server 200, and conversely, part of the processing described as being performed by the server 200 may be performed by the user terminal 100.

[0147] In the above embodiment, the determination unit 232 makes a determination by comparing the bristle opening degree with a bristle opening threshold. However, the determination method is not limited to this. For example, the determination unit 232 may determine whether the toothbrush 10 should be replaced by comparing an image of the toothbrush 10 with a predetermined reference image corresponding to the toothbrush 10, without estimating the bristle opening degree and / or determining the bristle opening threshold. For example, images of various types of toothbrushes with various bristle opening degrees may be prepared in advance as reference images, and a determination result as to whether the toothbrush 10 should be replaced may be associated with each image. The determination unit 232 may compare the image of the toothbrush 10 with the reference images with various bristle opening degrees corresponding to the toothbrush 10 and select the most similar reference image. The determination unit 232 may output the determination result associated with the most similar reference image as the determination result as to whether the toothbrush 10 should be replaced.

[0148] Although several embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included within the scope and spirit of the invention, as well as within the scope of the invention described in the claims and their equivalents. [Explanation of symbols]

[0149] 1...determination system, 10...toothbrush, 12...head, 20...head base material, 22...brush unit, 100...user terminal, 110...processing unit, 112...memory unit, 130...image acquisition unit, 132...feature information acquisition unit, 134...guide frame determination unit, 136...display control unit, 138...image recognition unit, 140...estimation unit, 142...determination unit, 144...shooting guide unit, 146...analysis area designation unit, 200...server, 210...processing unit, 212...memory unit, 220...image acquisition unit, 222...feature information acquisition unit, 224...image recognition unit, 226...image correction unit, 228...bristle opening degree estimation unit, 230...bristle opening threshold acquisition unit, 232...determination unit, 234...usable period estimation unit, 236...guidance information generation unit.

Claims

1. A determination system, comprising: an image acquisition unit that acquires an image of the toothbrush; a feature information acquisition unit that acquires feature information of the toothbrush, including information indicating attributes of the toothbrush head, based on the image; a reference information acquisition unit that acquires reference information corresponding to the toothbrush based on the feature amount information, the reference information being information regarding criteria for determining whether or not the toothbrush should be replaced; a determination unit that determines whether the toothbrush should be replaced based on the image and the reference information; an output unit that outputs information including a determination result by the determination unit; Equipped with the information indicating the attributes of the toothbrush head includes information regarding one or more of the attributes of a base material of the head and the attributes of bristles of the head; a bristle opening degree estimation unit that estimates a bristle opening degree indicating a degree of opening of the bristles of the toothbrush based on the image, the determination unit determines whether the toothbrush should be replaced by comparing the degree of bristle opening with the reference information; The criteria to be used can be changed depending on at least one of the width of the toothbrush head, the shape of the bristles, and the material of the bristles. Judging system.

2. The bristle opening degree is a value calculated based on geometric information of the bristles of the toothbrush in the image. The determination system according to claim 1 .

3. the hair opening degree estimation unit selects the geometric information used to calculate the hair opening degree in accordance with the feature amount information. The determination system according to claim 2 .

4. the image acquisition unit acquires an image of the toothbrush before use as a first image and an image of the toothbrush after use as a second image; The hair opening degree estimation unit estimates the hair opening degree by comparing the first image with the second image. The determination system according to any one of claims 1 to 3.

5. The bristle opening degree estimation unit estimates the bristle opening degree for each different portion of the toothbrush. The determination system according to any one of claims 1 to 4.

6. the reference information acquisition unit acquires, as the reference information, a hair opening threshold value according to the feature amount information; the determination unit determines whether the toothbrush should be replaced by comparing the degree of bristle opening with the bristle opening threshold value. The determination system according to any one of claims 1 to 5.

7. The toothbrush further includes an instruction information generating unit that generates instruction information including a recommended tooth brushing method based on the bristle opening degree. The determination system according to any one of claims 1 to 6.

8. The image acquisition unit acquires an image of the toothbrush as seen from at least the tip side of the toothbrush. The determination system according to any one of claims 1 to 7.

9. a guide frame determination unit that determines a type of guide frame for assisting a user terminal, which has an imaging unit that photographs the toothbrush, in photographing the toothbrush according to the feature amount information; a display control unit that displays the guide frame on a display unit provided in the user terminal; Further provided with The determination system according to any one of claims 1 to 8.

10. the determination unit determines whether the orientation of the toothbrush in the image of the toothbrush is appropriate. The determination system according to any one of claims 1 to 8.

11. A program, The processor of the information processing device an image acquisition step of acquiring an image of the toothbrush; a feature information acquisition step of acquiring feature information of the toothbrush, including information indicating attributes of the toothbrush head, based on the image; a reference information acquiring step of acquiring reference information corresponding to the toothbrush as information on a criterion for determining whether or not the toothbrush should be replaced based on the feature amount information; a determining step of determining whether the toothbrush should be replaced based on the image and the reference information; an output step of outputting information including the determination result of the determination step; Execute the information indicating the attributes of the toothbrush head includes information regarding one or more of the attributes of a base material of the head and the attributes of bristles of the head; The determining step estimating a bristle opening degree indicating a degree of opening of the bristles of the toothbrush based on the image; determining whether the toothbrush should be replaced by comparing the degree of bristle opening with the reference information; Including, The criteria to be used can be changed depending on at least one of the width of the toothbrush head, the shape of the bristles, and the material of the bristles. program.

12. A determination system, comprising: an image acquisition unit that acquires an image of the toothbrush; a feature amount information acquisition unit that acquires feature amount information of the toothbrush; a reference information acquisition unit that acquires reference information corresponding to the toothbrush based on the feature amount information, the reference information being information regarding criteria for determining whether or not the toothbrush should be replaced; a determination unit that determines whether the toothbrush should be replaced based on the image and the reference information; an output unit that outputs information including a determination result by the determination unit; Equipped with When the product ID of the toothbrush is unknown, the feature information acquisition unit acquires feature information including information indicating attributes of a head of the toothbrush, the information indicating the attributes of the toothbrush head includes information regarding one or more of the attributes of a base material of the head and the attributes of bristles of the head; a bristle opening degree estimation unit that estimates a bristle opening degree indicating a degree of opening of the bristles of the toothbrush based on the image, the determination unit determines whether the toothbrush should be replaced by comparing the degree of bristle opening with the reference information; The criteria to be used can be changed depending on at least one of the width of the toothbrush head, the shape of the bristles, and the material of the bristles. Judging system.

13. The bristle opening degree is a value calculated based on geometric information of the bristles of the toothbrush in the image. The determination system according to claim 12.

14. the hair opening degree estimation unit selects the geometric information used to calculate the hair opening degree in accordance with the feature amount information. The determination system according to claim 13.

15. the image acquisition unit acquires an image of the toothbrush before use as a first image and an image of the toothbrush after use as a second image; The hair opening degree estimation unit estimates the hair opening degree by comparing the first image with the second image. The determination system according to any one of claims 12 to 14.

16. The bristle opening degree estimation unit estimates the bristle opening degree for each different portion of the toothbrush. The determination system according to any one of claims 12 to 15.

17. the reference information acquisition unit acquires, as the reference information, a hair opening threshold value according to the feature amount information; the determination unit determines whether the toothbrush should be replaced by comparing the degree of bristle opening with the bristle opening threshold value. The determination system according to any one of claims 12 to 16.

18. The toothbrush further includes an instruction information generating unit that generates instruction information including a recommended tooth brushing method based on the bristle opening degree. The determination system according to any one of claims 12 to 17.

19. The image acquisition unit acquires an image of the toothbrush as seen from at least the tip side of the toothbrush. The determination system according to any one of claims 12 to 18.

20. a guide frame determination unit that determines a type of guide frame for assisting a user terminal, which has an imaging unit that photographs the toothbrush, in photographing the toothbrush according to the feature amount information; a display control unit that displays the guide frame on a display unit provided in the user terminal; Further provided with The determination system according to any one of claims 12 to 19.

21. the determination unit determines whether the orientation of the toothbrush in the image of the toothbrush is appropriate. The determination system according to any one of claims 12 to 20.

22. A program, The processor of the information processing device an image acquisition step of acquiring an image of the toothbrush; a feature amount information acquisition step of acquiring feature amount information of the toothbrush; a reference information acquiring step of acquiring reference information corresponding to the toothbrush as information on a criterion for determining whether or not the toothbrush should be replaced based on the feature amount information; a determining step of determining whether the toothbrush should be replaced based on the image and the reference information; an output step of outputting information including the determination result of the determination step; Execute When the product ID of the toothbrush is unknown, the feature information acquisition step includes acquiring feature information including information indicating attributes of a head of the toothbrush; the information indicating the attributes of the toothbrush head includes information regarding one or more of the attributes of a base material of the head and the attributes of bristles of the head; The determining step estimating a bristle opening degree indicating a degree of opening of the bristles of the toothbrush based on the image; determining whether the toothbrush should be replaced by comparing the degree of bristle opening with the reference information; Including, The criteria to be used can be changed depending on at least one of the width of the toothbrush head, the shape of the bristles, and the material of the bristles. program.

Images