Intraoral camera system and signal processing method

Abstract

This intraoral camera system comprises: an image capture unit that generates image data by capturing an image of teeth in an oral cavity, and that includes a handle part (10b), a head part (10a) including an image capturing element that generates image data, and a neck part (10c) connecting the handle part (10b) and the head part (10a); a position sensor (90) that detects the posture of the image capture unit; a signal processing unit that performs signal processing using the posture of the image capture unit detected by the position sensor (90); and a display unit for displaying the image data. The signal processing unit acquires an initial posture, which is a prescribed posture, of the image capture unit (S231), and performs signal processing where a first direction from the handle part toward the head part in the initial posture is considered a second direction oriented vertically upward along the vertical axis (S232, S233).

Description

[Technical Field]

[0001] The present disclosure relates to an intraoral camera system and a signal processing method. [Background technology]

[0002] For oral care devices such as intraoral cameras and toothbrushes, there are known techniques for performing signal processing and control using the posture of the device. For example, Patent Document 1 discloses an electric toothbrush device that performs signal processing using the posture of the device. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2017-6338 Summary of the Invention [Problem to be solved by the invention]

[0004] In such signal processing, it is desirable to be able to improve the accuracy of the signal processing.

[0005] Therefore, an object of the present disclosure is to provide an intraoral camera system and a signal processing method that can improve the accuracy of signal processing. [Means for solving the problem]

[0006] An intraoral camera system according to one embodiment of the present disclosure includes a handle portion, a head portion including an imaging element that generates image data, and a neck portion that connects the handle portion and the head portion, and is equipped with an imaging unit that photographs teeth in the oral cavity and generates image data, a sensor that detects the attitude of the imaging unit, a signal processing unit that performs signal processing using the attitude of the imaging unit detected by the sensor, and a display unit that displays the image data, wherein the signal processing unit acquires an initial attitude, which is a predetermined attitude of the imaging unit, and performs the signal processing by regarding a first direction from the handle portion to the head portion in the initial attitude as a second direction vertically upward along a vertical axis. [Effects of the Invention]

[0007] The present disclosure can provide an intraoral camera system and a signal processing method that can improve the accuracy of signal processing. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a perspective view of an intraoral camera in an intraoral camera system according to an embodiment. [Figure 2] FIG. 2 is a schematic configuration diagram of an intraoral camera system according to an embodiment. [Figure 3] FIG. 3 is a diagram showing the flow of an intraoral photographing operation in the intraoral camera system according to the embodiment. [Figure 4] FIG. 4 is a diagram showing teeth in an oral cavity according to an embodiment. [Figure 5] FIG. 5 is a functional block diagram of the mobile terminal according to the embodiment. [Figure 6] FIG. 6 is a diagram showing an example of an area in the oral cavity according to the embodiment. [Figure 7] FIG. 7 is a diagram illustrating an example of classification of reference data according to the embodiment. [Figure 8] FIG. 8 is a diagram illustrating an example of reference data according to the embodiment. [Figure 9] FIG. 9 is a flowchart of the type identification process according to the embodiment. [Figure 10] FIG. 10 is a diagram showing an example of a tooth image according to the embodiment. [Figure 11] FIG. 11 is a flowchart illustrating another example of the type identification process according to the embodiment. [Figure 12] FIG. 12 is a diagram showing the state of the oral cavity of a user with missing teeth according to an embodiment. [Figure 13] FIG. 13 is a diagram showing the relationship of the projection plane with respect to a user in a standing position according to an embodiment. [Figure 14] FIG. 14 is a diagram illustrating an example of a user's posture when using an intraoral camera according to an embodiment. [Figure 15] FIG. 15 is a diagram illustrating an example of a user's posture when using an intraoral camera according to an embodiment. [Figure 16] FIG. 16 is a flowchart of image processing according to the embodiment. [Figure 17] FIG. 17 is a diagram illustrating an example of an initial posture according to the embodiment. [Figure 18] FIG. 18 is a diagram illustrating an example of a setting screen for the initial attitude according to the embodiment. [Figure 19] FIG. 19 is a diagram illustrating an example of a setting screen for the initial attitude according to the embodiment. [Figure 20] FIG. 20 is a diagram illustrating an example of an initial posture according to the embodiment. [Figure 21] FIG. 21 is a diagram illustrating an example of posture correction according to the embodiment. [Figure 22] FIG. 22 is a diagram showing an example of an initial posture when using a dental treatment chair according to an embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] An intraoral camera system according to one embodiment of the present disclosure includes a handle portion, a head portion including an imaging element that generates image data, and a neck portion that connects the handle portion and the head portion, and is equipped with an imaging unit that photographs teeth in the oral cavity and generates image data, a sensor that detects the attitude of the imaging unit, a signal processing unit that performs signal processing using the attitude of the imaging unit detected by the sensor, and a display unit that displays the image data, wherein the signal processing unit acquires an initial attitude, which is a predetermined attitude of the imaging unit, and performs the signal processing by regarding a first direction from the handle portion to the head portion in the initial attitude as a second direction vertically upward along a vertical axis.

[0010] This allows the intraoral camera system to correct the posture of the imaging unit according to the posture of the user, for example, thereby improving the accuracy of signal processing.

[0011] For example, the predetermined posture may be a posture of the image capturing unit that has a predetermined relationship between the posture of the user and the posture of the image capturing unit.

[0012] For example, the specified posture may be a state in which the frontal plane of the user and the imaging plane of the photographing unit are parallel, and in a planar view of the imaging plane, the vertical axis of the user and the first direction are coincident or perpendicular to each other.

[0013] This allows the user to easily obtain the initial attitude, and the accuracy of the correction can be improved by improving the accuracy of the initial attitude.

[0014] For example, the specified posture may be a state in which a predetermined tooth and an imaging surface of the photographing unit are parallel and facing each other, and the first direction and the height direction of the predetermined tooth are aligned or perpendicular to each other in a planar view of the imaging surface.

[0015] This allows the user to easily obtain the initial attitude, and the accuracy of the correction can be improved by improving the accuracy of the initial attitude.

[0016] For example, the signal processing unit may determine the initial posture based on the angle of the backrest of a chair on which the user is sitting. This simplifies the initial setup process, improving convenience. Furthermore, reducing user operations may improve the accuracy of correction.

[0017] In addition, a signal processing method according to one aspect of the present disclosure includes an imaging unit including a handle unit, a head unit including an imaging element that generates image data, and a neck unit that connects the handle unit and the head unit, which images teeth in the oral cavity to generate image data, detects the attitude of the imaging unit, performs signal processing using the detected attitude of the imaging unit, and displays the image data, and in the signal processing, obtains an initial attitude that is a predetermined attitude of the imaging unit, and performs the signal processing by regarding a first direction from the handle unit toward the head unit in the initial attitude as a second direction vertically upward along a vertical axis.

[0018] According to this, the signal processing method can correct the posture of the imaging unit in accordance with the posture of the user, for example, and therefore can improve the accuracy of signal processing.

[0019] These comprehensive or specific aspects may be realized as a system, a method, an integrated circuit, a computer program, or a computer-readable recording medium such as a CD-ROM, or may be realized as any combination of a system, a method, an integrated circuit, a computer program, and a recording medium.

[0020] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. However, more detailed explanation than necessary may be omitted. For example, detailed explanation of well-known matters or redundant explanation of substantially the same configuration may be omitted. This is to avoid unnecessary redundancy in the following explanation and to facilitate understanding by those skilled in the art.

[0021] The inventors have provided the accompanying drawings and the following description to enable those skilled in the art to fully understand the present disclosure, and do not intend for them to limit the subject matter described in the claims.

[0022] (Embodiment) Fig. 1 is a perspective view of an intraoral camera in an intraoral camera system according to the present embodiment. As shown in Fig. 1, the intraoral camera 10 has a toothbrush-shaped housing that can be handled with one hand, and the housing includes a head 10a that is placed in the user's oral cavity when photographing the dentition, a handle 10b that the user holds, and a neck 10c that connects the head 10a and the handle 10b.

[0023] The photographing optical system 12 is incorporated into the head portion 10a and the neck portion 10c. The photographing optical system 12 includes an image sensor 14 and a lens (not shown in FIG. 1) arranged on its optical axis LA.

[0024] The imaging element 14 is a photographing device such as a C-MOS sensor or a CCD element, and an image of the teeth is formed by a lens. The imaging element 14 outputs a signal (image data) corresponding to the formed image to the outside.

[0025] The intraoral camera 10 is also equipped with a plurality of first to fourth LEDs 26A to 26D as lighting devices that irradiate light onto the teeth to be photographed during photography. The first to fourth LEDs 26A to 26D are, for example, white LEDs.

[0026] Fig. 2 is a schematic diagram of the intraoral camera system according to the present embodiment. As shown in Fig. 2, the intraoral camera system according to the present embodiment is generally configured to photograph the row of teeth using an intraoral camera 10 and perform image processing on the photographed image.

[0027] As shown in Fig. 2, the intraoral camera system includes an intraoral camera 10, a mobile terminal 70, and a cloud server 80. The mobile terminal 70 is, for example, a smartphone or tablet terminal capable of wireless communication. The mobile terminal 70 includes, as an input device and an output device, for example, a touch screen 72 capable of displaying a dentition image. The mobile terminal 70 functions as a user interface for the intraoral camera system.

[0028] The cloud server 80 is a server that can communicate with the mobile terminal 70 via the Internet or the like, and provides the mobile terminal 70 with an application for using the intraoral camera 10. For example, a user downloads the application from the cloud server 80 and installs it on the mobile terminal 70. The cloud server 80 also acquires dentition images captured by the intraoral camera 10 via the mobile terminal 70.

[0029] The intraoral camera system includes a central control unit 50 as the main part that controls the system, an image processing unit 52 that processes the dentition image from the imaging element 14, an LED control unit 54 that controls multiple LEDs 26A to 26D, a lens driver 56 that controls the actuator 36 of the composition adjustment mechanism and the actuator 40 of the focus adjustment mechanism, and a position sensor 90.

[0030] The intraoral camera system also includes a wireless communication module 58 that communicates wirelessly with the mobile terminal 70, and a power supply control unit 60 that supplies power to the central control unit 50 and the like.

[0031] The central control unit 50 of the intraoral camera system is mounted, for example, on the handle unit 10b of the intraoral camera 10. For example, the central control unit 50 also includes a controller 62 such as a CPU or MPU that executes various processes described below, and a memory 64 such as a RAM or ROM that stores programs for causing the controller 62 to execute the various processes. In addition to the programs, the memory 64 also stores images of rows of teeth (image data) captured by the image sensor 14, various setting data, and the like.

[0032] The image processing unit 52 is mounted, for example, on the handle unit 10b of the intraoral camera 10, and acquires the row-of-teeth image (image data) captured by the imaging element 14 based on a control signal from the controller 62 of the central control unit 50. It then performs image processing on the acquired row-of-teeth image and outputs the processed row-of-teeth image to the central control unit 50. The image processing unit 52 is composed of, for example, a circuit, and performs image processing on the row-of-teeth image, such as noise removal and AWB (Automatic White Balance) processing. The controller 62 transmits the row-of-teeth image output from the image processing unit 52 to the mobile terminal 70 via the wireless communication module 58. The mobile terminal 70 displays the transmitted row-of-teeth image on the touch screen 72, thereby presenting the row-of-teeth image to the user.

[0033] The LED control unit 54 is mounted on, for example, the handle portion 10b of the intraoral camera 10, and turns on and off the first to fourth LEDs 26A to 26D based on a control signal from the controller 62. The LED control unit 54 is configured, for example, by a circuit. For example, when a user performs an operation on the touch screen 72 of the mobile terminal 70 to activate the intraoral camera 10, a corresponding signal is transmitted from the mobile terminal 70 to the controller 62 via the wireless communication module 58. Based on the received signal, the controller 62 transmits a control signal to the LED control unit 54 to turn on the first to fourth LEDs 26A to 26D.

[0034] The lens driver 56 is mounted, for example, on the handle portion 10b of the intraoral camera 10 and controls the actuator 36 of the composition adjustment mechanism and the actuator 40 of the focus adjustment mechanism based on a control signal from a controller 62 of the central control unit 50. The lens driver 56 is composed of, for example, a circuit. For example, when a user performs an operation related to composition adjustment or focus adjustment on the touch screen 72 of the mobile terminal 70, a corresponding signal is transmitted from the mobile terminal 70 to the central control unit 50 via the wireless communication module 58. Based on the received signal, the controller 62 of the central control unit 50 transmits a control signal to the lens driver 56 to adjust the composition or focus. Furthermore, for example, the controller 62 calculates the control amounts of the actuators 36, 40 required for the composition adjustment or focus adjustment based on the row-of-teeth image from the image processing unit 52, and transmits a control signal corresponding to the calculated control amount to the lens driver 56.

[0035] The wireless communication module 58 is mounted on, for example, the handle portion 10b of the intraoral camera 10, and performs wireless communication with the mobile terminal 70 based on a control signal from the controller 62. The wireless communication module 58 performs wireless communication with the mobile terminal 70 in accordance with an existing communication standard such as WiFi (registered trademark) or Bluetooth (registered trademark). Via the wireless communication module 58, the intraoral camera 10 transmits a row-of-dentition image showing the teeth D to the mobile terminal 70, and operation signals are transmitted from the mobile terminal 70 to the intraoral camera 10.

[0036] In this embodiment, the power supply control unit 60 is mounted on the handle unit 10b of the intraoral camera 10, and distributes power from a battery 66 to the central control unit 50, the image processing unit 52, the LED control unit 54, the lens driver 56, and the wireless communication module 58. The power supply control unit 60 is configured, for example, by a circuit. In this embodiment, the battery 66 is a rechargeable secondary battery, and is wirelessly charged by an external charger 69 connected to a commercial power source via a coil 68 mounted on the intraoral camera 10.

[0037] The position sensor 90 is a sensor for detecting the posture and position of the intraoral camera 10, and is, for example, a multi-axis (here, three axes: x, y, and z) acceleration sensor. For example, the position sensor 90 may be a six-axis sensor having a three-axis acceleration sensor and a three-axis gyro sensor. For example, as shown in FIG. 1 , the z-axis coincides with the optical axis LA. The y-axis is parallel to the imaging surface and extends in the longitudinal direction of the intraoral camera 10. The x-axis is parallel to the imaging surface and perpendicular to the y-axis. The output of each axis (sensor data) of the position sensor 90 is transmitted to the mobile terminal 70 via the central control unit 50 and the wireless communication module 58.

[0038] The position sensor 90 may be a piezo-resistive, capacitance, or thermal-detection MEMS sensor. Although not shown, a compensation circuit may be provided to compensate for the balance of sensitivity of the sensors for each axis, the temperature characteristics of sensitivity, temperature drift, etc. A band-pass filter (low-pass filter) may also be provided to remove dynamic acceleration components and noise. Noise may also be reduced by smoothing the output waveform of the acceleration sensor.

[0039] Next, an intraoral imaging operation in the intraoral camera system will be described. Fig. 3 is a diagram showing the flow of the intraoral imaging operation in the intraoral camera system.

[0040] A user uses the intraoral camera 10 to capture images of the teeth and gums in their oral cavity, thereby generating image data (S101). Next, the intraoral camera 10 transmits the captured image data and sensor data obtained by the position sensor 90 at the time of capture to the mobile terminal 70 (S102). Note that the image data may be a video or one or more still images. If the image data is a video or multiple still images, the sensor data is transmitted for each frame of the video or for each still image. If the image data is a video, the sensor data may be transmitted for each multiple frame.

[0041] The image data and sensor data may be transmitted in real time, or may be transmitted all at once after a series of photographs (for example, photographs of all teeth in the oral cavity) have been taken.

[0042] The mobile terminal 70 acquires reference data from the cloud server 80 (S103), and uses the received image data and sensor data and the acquired reference data to identify the type and position of each of the multiple teeth included in the image data (S104).

[0043] Fig. 4 is a diagram showing teeth in an oral cavity. The types of teeth identified by the mobile terminal 70 are, for example, central incisors, lateral incisors, canines, etc. shown in Fig. 4, and the positions of the teeth identified by the mobile terminal 70 are upper jaw, lower jaw, right side, left side, etc. In other words, identifying the type and position of a tooth means identifying which of the multiple teeth shown in Fig. 4 the target tooth is.

[0044] Furthermore, the mobile terminal 70 may generate a three-dimensional model of multiple teeth in the oral cavity from the captured image data using, for example, the type and position of the identified teeth, and display an image based on the generated three-dimensional model.

[0045] By using such an intraoral camera system, a user can take an image of the inside of their own oral cavity with the intraoral camera 10 and check the condition of the inside of their oral cavity displayed on the mobile terminal 70. This allows the user to easily check the health condition of their own teeth.

[0046] Here, an example will be described in which the mobile terminal 70 identifies the type of tooth, etc., but some or all of the processing performed by the mobile terminal 70 may be performed by the intraoral camera 10 or the cloud server 80.

[0047] 5 is a functional block diagram of the mobile terminal 70. The mobile terminal 70 includes an area detection unit 101, a user information acquisition unit 102, and an identification unit 103. The functions of each of these processing units are realized, for example, by a program execution unit such as a CPU or a processor reading and executing a software program recorded on a recording medium such as a hard disk or semiconductor memory.

[0048] The area detection unit 101 uses the sensor data to detect an area within the oral cavity corresponding to each image data, and generates area information indicating the detected area.

[0049] The user information acquisition unit 102 acquires user information indicating user attributes. For example, the user information acquisition unit 102 acquires user information input by the user via a user interface of the mobile terminal 70. Alternatively, the user information acquisition unit 102 may acquire user information stored in the mobile terminal 70 or another device (for example, the cloud server 80). Specifically, the user information indicates at least one of the user's gender, age group (or age), and race.

[0050] The identification unit 103 identifies the type and position of each tooth included in the image data using the image data, region information, user information, and reference data. The identification unit 103 includes a tooth image generation unit 104 and a type identification unit 105. From the image data, tooth images each representing one tooth are generated. The type identification unit 105 identifies the type and position of the tooth included in the tooth image using the region information, user information, reference data, and estimation model 106.

[0051] The estimation model 106 is a model for estimating the type and position of the tooth included in the tooth image from the tooth image and reference data. For example, the estimation model 106 may include a neural network.

[0052] Next, an example of a region detected by the region detection unit 101 will be described. FIG. 6 is a diagram showing an example of a region within the oral cavity. In this figure, for example, multiple teeth within the oral cavity are divided into six regions: maxillary left, maxillary front, maxillary right, mandibular left, mandibular front, and mandibular right. Note that while an example of division into six regions is shown here, any number of regions may be used. Alternatively, the region may be divided into two regions: maxillary and mandibular. Each region may also be further divided based on the imaging direction. For example, as shown in FIG. 6, each region may be divided into two imaging directions: the buccal side and the lingual side. Although an example in which each tooth does not belong to multiple regions has been shown here, some teeth may belong to two or more regions. For example, a tooth near the boundary between two adjacent regions may belong to both the two regions. For example, the third canine tooth on the left edge of the maxillary front in FIG. 6 may belong to both the maxillary front and the maxillary left.

[0053] A specific example of a method for determining the region and imaging direction from sensor data will be described below. First, the region detection unit 101 determines whether it is the upper jaw or the lower jaw based on the output Az of the z-direction acceleration sensor. Here, when imaging the row of teeth of the upper jaw, the imaging surface faces upward to a certain extent, and when imaging the row of teeth of the lower jaw, the imaging surface faces downward to a certain extent. Therefore, the region detection unit 101 determines that the region corresponding to the image data is the lower jaw if Az>0, and that the region corresponding to the image data is the upper jaw if Az≦0.

[0054] Next, a method for determining which region of the upper jaw it is when it is determined to be the upper jaw will be described. The region detection unit 101 determines whether it is a front tooth or not based on the output Ay of the y-direction acceleration sensor. Here, when photographing the front teeth, the intraoral camera 10 is relatively horizontal, but when photographing the molars, the intraoral camera 10 must be at an angle due to interference with the lips. Therefore, the region detection unit 101 determines that it is the front of the upper jaw when Ay≦threshold a.

[0055] Furthermore, if the region detection unit 101 determines that the area is the front of the maxilla, it determines whether it is the cheek side or the tongue side based on the output Ax of the x-direction acceleration sensor. Here, the orientation of the imaging plane is reversed for the cheek side and the tongue side. Therefore, the region detection unit 101 determines that the area is the "front of the maxilla cheek side" if Ax>0, and determines that the area is the "front of the maxilla tongue side" if Ax≦0.

[0056] On the other hand, if the region detection unit 101 determines that the area is not in front of the upper jaw, it determines the orientation of the imaging plane based on the output Ax of the x-direction acceleration sensor. Specifically, if Ax>0, the region detection unit 101 determines that the area is "upper jaw right cheek side or upper jaw left tongue side," and if Ax≦0, it determines that the area is "upper jaw left cheek side or upper jaw right tongue side."

[0057] Furthermore, the region detection unit 101 narrows down the region based on the region determined in the previous process. Specifically, when determining whether the region is the upper right cheek side or the upper left lingual side, if the previous region was any of "upper anterior cheek side, upper right cheek side, upper right lingual side, lower anterior cheek side, lower right cheek side, and lower right lingual side," the region detection unit 101 estimates that the current region is the "upper right cheek side," and if the previous region was any of "upper anterior lingual side, upper left cheek side, upper left lingual side, lower anterior lingual side, lower left cheek side, and lower left lingual side," the region detection unit 101 estimates that the current region is the "upper left lingual side."

[0058] Furthermore, when determining whether the area is the upper jaw left cheek side or the upper jaw right tongue side, if the previous area was any of "upper jaw anterior cheek side, upper jaw left cheek side, upper jaw left tongue side, lower jaw anterior cheek side, lower jaw left cheek side, or lower jaw left tongue side," the area detection unit 101 estimates the current area to be "upper jaw left cheek side," and if the previous area was any of "upper jaw anterior tongue side, upper jaw right cheek side, upper jaw right tongue side, lower jaw anterior tongue side, lower jaw right cheek side, or lower jaw right tongue side," the area detection unit 101 estimates the current area to be "upper jaw right tongue side." This takes advantage of the fact that the imaging plane is likely to be moved so as to minimize the amount of movement and change in direction of the imaging plane.

[0059] A similar determination is also made for the lower jaw. Specifically, the region detection unit 101 determines whether or not a tooth is a front tooth based on the output Ay of the y-direction acceleration sensor. Specifically, if Ay≦threshold b, the region detection unit 101 determines that the tooth is in front of the lower jaw.

[0060] If it is determined to be the front of the mandible, the region detection unit 101 determines whether it is the cheek side or the tongue side based on the output Ax of the x-direction acceleration sensor. Specifically, the region detection unit 101 determines the region as the "front of the mandible cheek side" if Ax<0, and determines the region as the "front of the mandible tongue side" if Ax≧0.

[0061] On the other hand, if the region detection unit 101 determines that the area is not in front of the lower jaw, it determines the orientation of the imaging plane based on the output Ax of the x-direction acceleration sensor. Specifically, if Ax>0, the region detection unit 101 determines that the area is "the right cheek side or the left tongue side of the lower jaw," and if Ax≦0, it determines that the area is "the left cheek side or the right tongue side of the lower jaw."

[0062] In addition, when determining whether the area is the right cheek side of the lower jaw or the left tongue side of the lower jaw, the area detection unit 101 determines whether the previous area is "the front cheek side of the lower jaw, the right cheek side of the lower jaw, the right tongue side of the lower jaw, above If the previous area is either "anterior lingual side of the mandible, right cheek side of the maxilla, or right lingual side of the maxilla," the current area is estimated to be "right cheek side of the mandible," and if the previous area is either "anterior lingual side of the mandible, left cheek side of the mandible, left lingual side of the mandible, anterior lingual side of the maxilla, left cheek side of the maxilla, or left lingual side of the maxilla," the current area is estimated to be "left lingual side of the mandible."

[0063] Furthermore, when determining whether the area is the left cheek side of the lower jaw or the right tongue side of the lower jaw, if the previous area was any of "anterior cheek side of the lower jaw, left cheek side of the lower jaw, left tongue side of the lower jaw, anterior cheek side of the upper jaw, left cheek side of the upper jaw, or left tongue side of the upper jaw," the area detection unit 101 estimates that the current area is "left cheek side of the lower jaw," and if the previous area was any of "anterior tongue side of the lower jaw, right cheek side of the lower jaw, right tongue side of the lower jaw, anterior tongue side of the upper jaw, right cheek side of the upper jaw, or right tongue side of the upper jaw," the area detection unit 101 estimates that the current area is "right tongue side of the lower jaw."

[0064] Through the above processing, the current area is identified as one of the following: "anterior cheek side of the maxilla," "anterior lingual side of the maxilla," "right cheek side of the maxilla," "left lingual side of the maxilla," "left cheek side of the maxilla," "right lingual side of the maxilla," "anterior cheek side of the mandible," "anterior lingual side of the mandible," "right cheek side of the mandible," "left lingual side of the mandible," "left cheek side of the mandible," or "right lingual side of the mandible."

[0065] The above-described determination algorithm is merely an example, and any determination algorithm may be used as long as it can identify the region from the acceleration sensor outputs Ax, Ay, and Az. For example, instead of using the Ax, Ay, and Az values ​​directly as determination variables, a secondary variable obtained by appropriately combining Ax, Ay, and Az may be used for the determination. The secondary variable may be set arbitrarily, such as Ay / Az, Ax·Ax+Ay·Ay, or Az-Ax. Alternatively, the region may be determined after converting the acceleration information Ax, Ay, and Az of each axis into angle information (attitude angles) α, β, and γ. For example, the angle of the x-axis relative to the direction of gravitational acceleration may be defined as the roll angle α, the angle of the y-axis relative to the direction of gravitational acceleration may be defined as the pitch angle β, and the angle of the z-axis relative to the direction of gravitational acceleration may be defined as the yaw angle γ. Furthermore, the threshold values ​​used for each determination may be determined based on the results of clinical experiments, etc.

[0066] In the above description, two imaging directions, the cheek side and the tongue side, are determined as imaging directions, but three directions, including the crown side, may also be determined. For example, when imaging the crown side, the imaging plane becomes more horizontal than when imaging the cheek side or tongue side, making it possible to determine whether the imaging direction is the crown side.

[0067] Although the above describes an example in which the area of ​​the subject to be photographed and the photographing direction are determined using a three-axis acceleration sensor included in the position sensor 90, the area of ​​the subject to be photographed and the photographing direction may also be determined using a three-axis gyro sensor. The three-axis gyro sensor outputs, for example, the amount of change in angle due to movement around the x-axis, the amount of change in angle due to movement around the y-axis, and the amount of change in angle due to movement around the z-axis. That is, in the case of a three-axis gyro sensor, the amount of change in each axis is added with the initial states of the x-axis, y-axis, and z-axis arbitrarily set, and the area of ​​the subject to be photographed and the orientation (photography direction) of the imaging surface of the intraoral camera 10 are determined.

[0068] In addition, the area of ​​the subject to be photographed and the orientation of the imaging surface of the intraoral camera 10 may be determined by combining both a three-axis acceleration sensor and a three-axis gyro sensor.

[0069] Next, a detailed description will be given of the operation of the identification unit 103. Note that the following describes processing for one piece of image data (one frame included in a moving image, or one still image).

[0070] First, the tooth image generating unit 104 generates tooth images, each representing one tooth, from the image data. Specifically, the tooth image generating unit 104 detects interdental positions from the image data by image analysis or the like, and extracts tooth images using the detected interdental positions. For example, the tooth image generating unit 104 generates tooth images by extracting images using the interdental positions as boundaries.

[0071] Next, the type identifying unit 105 identifies the type and position of the tooth included in the tooth image using the region information, user information, reference data, and the estimation model 106 .

[0072] The reference data is data that is referenced when identifying the type and position of teeth included in a tooth image. For example, the reference data is data on teeth whose type and position are known. Specifically, the reference data may be a group of image data of teeth photographed in advance, a group of image data of the tooth row, or a panoramic image of the tooth row. Alternatively, the reference data may be information indicating the standard shape or feature values ​​of each tooth.

[0073] In addition to the type and position, the reference data may be classified by shooting direction and by user attribute. The user attribute is one or a combination of two or more of the user's gender, age group (or age), and race. In other words, the user attribute is uniquely determined according to the user's gender, age group, and race.

[0074] FIG. 7 is a diagram showing an example of reference data classification. Note that while the diagram shows hierarchical classification of reference data, hierarchical classification is not necessarily required. The reference data used for identification is represented as A(n). Furthermore, n is uniquely associated with a set of tooth type, position, and imaging direction. FIG. 8 is a diagram showing an example of reference data. As an example, reference data for the buccal, lingual, and crown sides of maxillary incisors, canines, and first molars are shown.

[0075] As shown in Figure 8, teeth vary in shape and size depending on the type. For example, maxillary central incisors have the following characteristics: The buccal outer shape is generally a vertically elongated trapezoid, and the incisal edge is almost straight. The cervical line is convex toward the root, and the mesial and distal edges are slightly curved. The apex of the curve of the mesial edge is near the mesial incisal corner. The apex of the curve of the distal edge is 1 / 3 the height of the incisal edge. The lingual outer shape is triangular, and the mesial and distal marginal ridges and the lingual cervical ridge form a marginal ridge, forming a lingual fossa.

[0076] In addition, maxillary canines have the following characteristics: The outer shape on the buccal side is generally pentagonal, with the incisal edge protruding in the center to form a cusp. The cervical line is convex toward the root. The mesial edge is straight or slightly convex outward, and the distal edge is straight or slightly concave. The outer shape on the lingual side is diamond-shaped, with the mesial and distal marginal ridges and the lingual cervical ridge forming a marginal ridge.

[0077] The maxillary first molar also has the following characteristics: The buccal outer shape is generally trapezoidal, with the mesiodistal margin being nearly straight. The cervical line is horizontal and convex toward the furcation in the central part. The contact point is 1 / 3 of the height on the occlusal surface mesially and 1 / 2 of the height distally. The lingual outer shape is trapezoidal, with a lingual lateral groove running vertically through nearly the center. The coronal outer shape is a parallelogram, with the buccolingual diameter being larger than the mesiodistal diameter.

[0078] The tooth image to be processed is represented as B(m). Therefore, the tooth images of the teeth on both sides of the tooth image to be processed are represented as B(m-1) and B(m+1).

[0079] Furthermore, region information corresponding to the tooth image (B(m)) to be processed, detected by region detection unit 101, is represented as C(m). For example, region information is generated for each image data. Therefore, if one image data contains multiple teeth and multiple tooth images are generated, the same region information corresponding to the single image data is associated with the multiple tooth images.

[0080] 9 is a flowchart of the type identification process by the type identification unit 105. First, the type identification unit 105 performs initialization (S111). Specifically, the type identification unit 105 sets n to 0, Err to Err_Max, and N to 0. Here, Err is an evaluation value described later, and the smaller the Err value, the higher the evaluation. Furthermore, Err_Max is the maximum value that Err can theoretically take. Furthermore, N indicates the n value of the smallest Err.

[0081] Next, the type identification unit 105 selects reference data to be used based on the user information and the region information (S112). Specifically, the type identification unit 105 selects reference data to which the user attributes indicated by the user information are assigned and which are also assigned to the type, position, and imaging direction of the tooth corresponding to the region indicated by the region information. For example, if the region indicated by the region information is the left lingual side of the maxilla, a total of five reference data sets, namely, the lingual sides of five teeth included in the left maxilla, are selected as the reference data to be used. Furthermore, n_max, which is the maximum value of n, is set according to the number of selected reference data sets. For example, if the number of reference data sets is five, n=0 to 4 are assigned to these five reference data sets, and n_max is set to 4.

[0082] Next, the type identification unit 105 calculates Err(n) from the tooth image B(m) and the reference data (A(n)) (S113). For example, the type identification unit 105 calculates Err(n) using Err(n) = f(A(n)) - f(B(m)). Here, f(A(n)) and f(B(m)) are values ​​obtained by inputting A(n) and B(m) to the function f(). The function f() is a function that extracts the feature quantities of A(n) and B(m). Note that f() may be expressed as a vector rather than a scalar.

[0083] As shown in Fig. 8, each tooth has a characteristic shape and size according to its type. The type identification unit 105 extracts these characteristic shapes and sizes as feature quantities using the above-mentioned function f.

[0084] The feature extracted by the function f will be described with reference to FIG. 10. FIG. 10 is a diagram showing an image of a first molar in the maxillary right region photographed from the crown side. The occlusal surface of the first molar is shaped like a parallelogram, and the lines AB and DC connecting the mesiodistal buccal cusp tip and the lingual cusp tip, respectively, are nearly parallel, and the lines AD and BC are also nearly parallel. The distances between the cusps are also approximately equal (AB = DC, AD = BC). As an example, the two intercusp distances mentioned above can be used as feature amounts.

[0085] Also, Err(n) is a value that indicates the difference (distance in the case of vectors) between f(A(n)) and f(B(m)). In other words, the closer B(m) is to A(n), the smaller "f1(A(n))-f1(B(m))" becomes, and Err(n) takes a minimum value when n=m.

[0086] If the calculated Err(n) is smaller than Err, the type identifying unit 105 sets Err to Err(n) and sets N to n (S114).

[0087] If n=n_max is not satisfied (No in S115), the type identification unit 105 increments n by 1 (S116) and performs the processes from step S113 onwards again. That is, steps S113 and S114 are performed for all reference data to be used.

[0088] If n=n_max (Yes in S115), the type identifying unit 105 outputs the type, position, and imaging direction corresponding to N as the type, position, and imaging direction of the tooth included in the tooth image (S117).

[0089] Through the above process, the type identification unit 105 can identify the type, position, and photographing direction of the tooth image. Furthermore, in step S112, the user information and region information can be used to reduce candidates for the tooth type, position, and photographing direction. This reduces the amount of processing and improves the identification accuracy.

[0090] Fig. 11 is a flowchart showing another example of the type identification process by the type identification unit 105. In the process shown in Fig. 11, step S112 is changed to step S112A and step S118 is added to the process shown in Fig. 9.

[0091] In step S112A, the type identification unit 105 selects reference data to be used based on the user information. Specifically, the type identification unit 105 selects reference data to which the user attribute indicated by the user information is assigned.

[0092] In step S118, the type identification unit 105 weights Err(n) calculated in step S113 based on the region information. Specifically, the type identification unit 105 multiplies Err(n) by w according to the region information. For example, if the region indicated by the region information includes a tooth corresponding to n, Err(n) is multiplied by w0. On the other hand, if the region indicated by the region information does not include a tooth corresponding to n, Err(n) is multiplied by w1, which is larger than w0. This reduces Err for teeth included in the region indicated by the region information, making it easier to determine that teeth included in the tooth image are teeth included in that region.

[0093] Furthermore, weighting does not have to be based on whether or not a tooth is included in a region. For example, weighting may be set according to distance from the region indicated by the region information. For example, the weight of a tooth located near the region indicated by the region information may be set smaller than the weight of a tooth located far from the region.

[0094] Furthermore, instead of using the user information to select reference data, it may be used to weight Err(n) in the same way as the region information.

[0095] Furthermore, the selection of reference data based on the region information as described in Fig. 9 may be combined with the weighting described in Fig. 11. For example, teeth located far from the region indicated by the region information may be excluded from the target, and weights may be used for teeth located close to the region.

[0096] Furthermore, if a user regularly takes intraoral photographs and has previously acquired tooth images of the user, the tooth images may be used as reference data. In this case, the selection of reference data based on the user information is not performed, and only processing based on the region information is performed.

[0097] In addition, in the above, the tooth image to be processed is compared with reference data, but multiple tooth images corresponding to the tooth row including the tooth to be processed may also be compared with multiple reference data.

[0098] For example, the type identification unit 105 may calculate Err(n) by the following formula: Err(n) = f(A(n)) - f(B(m)) + f'(A(n-1)) - f'(B(m-1)) + f'(A(n+1)) - f'(B(m+1)). Here, A(n-1) and A(n+1) are reference data of teeth adjacent to the tooth of A(n), and B(m-1) and B(m+1) are tooth images of teeth adjacent to the tooth of B(m). Furthermore, f'() is a function that extracts features for evaluating the teeth on both sides of the tooth of interest. In this way, the accuracy of identification can be improved by using information on adjacent teeth.

[0099] Furthermore, in the above example, an image of a tooth is used as reference data, but a feature amount (that is, the value of f(A(n)) above) may also be used as reference data.

[0100] The estimation model 106 used by the type identification unit 105 for identification may include a trained model such as a neural network. For example, the above-mentioned function f or f' may be a trained model. The method using a neural network is not limited to this. For example, the entire estimation model 106 that performs type estimation may be configured using a neural network. In this case, for example, an estimation model 106 may be provided for each user attribute. Each estimation model 106 is a trained model generated by machine learning using multiple pairs of tooth images, area information, tooth type, position, and imaging direction for the corresponding user attribute as training data (learning data). The estimation model 106 receives the tooth image and area information as input and outputs the tooth type, position, and imaging direction. In this case, the type identification unit 105 selects a corresponding estimation model 106 using user information and inputs the tooth image and area information into the selected estimation model 106 to acquire the tooth type, position, and imaging direction.

[0101] Alternatively, an estimation model 106 may be provided for each pair of user attributes and region information. In this case, each estimation model 106 is a trained model generated by machine learning using multiple pairs of tooth images and tooth type, position, and imaging direction for the corresponding user attributes and region information as training data. The estimation model 106 receives a tooth image as input and outputs the tooth type, position, and imaging direction. In this case, the type identification unit 105 selects a corresponding estimation model 106 using the user information and region information, and acquires the tooth type, position, and imaging direction by inputting the tooth image into the selected estimation model 106.

[0102] In the above description, an example in which both user information and area information are used has been described, but only one of them may be used.

[0103] In the above description, the area information indicates both the area and the imaging direction, but it may indicate only one of them.

[0104] Next, the operation performed by the identification unit 103 in the intraoral camera system when a user has lost some teeth (for example, the "second premolar on the left side of the upper jaw") due to treatment for dental caries or the like will be described in detail. Note that the following describes processing for one image data (one frame included in a moving image, or one still image). Figure 12 is a diagram showing the state of the oral cavity of a user who has lost the second premolar on the left side of the upper jaw.

[0105] The region detection unit 101 determines that the intraoral camera 10 is capturing an image of the "upper left side" region, which includes the second premolar. The intraoral camera 10 then captures an image B(m') of the area corresponding to the second premolar, and detects the absence of a tooth by image analysis or the like.

[0106] Next, the type identification unit 105 calculates Err(n) from the tooth images B(m'-1) and B(m'+1) and the reference data (A(n)), identifies the type and position of the teeth in the tooth images B(m'-1) and B(m'+1), identifies that B(m') is an image of the area sandwiched between the first premolar and first molar on the left side of the upper jaw, determines that the second premolar is missing, and outputs the determination result.

[0107] In addition, if a third molar is missing, it can be determined that the third molar is missing by identifying the adjacent second molar.

[0108] Furthermore, if a user regularly takes intraoral photographs and tooth images of the user have been obtained in the past, the tooth images may be used as reference data. In this case, information about the user's missing teeth can be obtained from the results of the past intraoral photographs.

[0109] The above-described determination of the tooth region and imaging direction based on the posture of the intraoral camera 10 (generation of region information) assumes that the user is facing forward, such as standing upright or sitting in a chair. On the other hand, when a dentist or other professional photographs a patient's teeth, the photograph may be taken with the user (patient) lying on their back. In such cases, the relationship between the vertical axis and the teeth differs from that when the user is facing forward, and therefore accurate determination may not be possible. Below, a method for making accurate determinations even in such cases will be described.

[0110] 13 is a diagram showing the relationship of projection planes with respect to user BD in a standing state. Here, the projection planes are imaginary planes based on user BD, and consist of three planes: a frontal plane 110, a sagittal plane 111, and a horizontal plane 112. The frontal plane 110 is a plane that bisects the body of user BD into front and back and is perpendicular to the floor. The sagittal plane 111 is a plane that passes through the body of user BD from front to back, bisects the body of user BD into left and right, and is perpendicular to the floor. The horizontal plane 112 is a plane that is parallel to the floor, bisects the body of user BD into top and bottom, and is perpendicular to both the frontal plane 110 and the sagittal plane 111.

[0111] Furthermore, the axes of movement are divided into the vertical axis, the sagittal-horizontal axis, and the forehead-horizontal axis. The x-axis shown in FIG. 13 is the "forehead-horizontal axis." The sagittal-horizontal axis is an axis in the left-right direction, and is the axis of rotation for movements such as anterior-posterior bending and flexion-extension in the sagittal plane 111. The y-axis shown in FIG. 13 is the "sagittal-horizontal axis." The sagittal-horizontal axis is an axis in the frontal plane 110, and is the axis of rotation for movements such as lateral bending and abduction / abduction. The z-axis shown in FIG. 13 is the "vertical axis." The vertical axis is an axis in the vertical direction, and is the axis of rotation for movements such as rotation in the horizontal plane 112.

[0112] 14 and 15 are diagrams showing examples of the posture of the user BD when using the intraoral camera 10. FIG.

[0113] 14, when the user BD uses the intraoral camera 10 in an upright position or while sitting in a chair, the user can be considered to be in a standing position. In this case, the vertical axis Z0 (z-axis) of the body of the user BD is perpendicular to the floor surface, and the vertical axis Z0 of the body of the user BD and the direction of gravitational acceleration are aligned.

[0114] 15, for example, when a dentist uses the intraoral camera 10 with a user BD resting on a dental treatment table, the frontal plane 110 of the upper half of the body of the user BD tilts along the backrest of the treatment table. That is, as a result of the tilt of the frontal plane 110 of the user BD, the vertical axis Z1 of the user with the upper half of the body tilted along the backrest tilts relative to the vertical axis Z0 of the body when the user BD is standing upright.

[0115] FIG. 16 is a flowchart of the area detection process in the area detection unit 101 when the posture of the user BD changes as described above. First, the area detection unit 101 acquires and holds the initial posture of the intraoral camera 10 (S231). Specifically, based on a user operation, the posture of the intraoral camera 10 in a state in which the user operation is performed is acquired as the initial posture. For example, the initial posture is acquired based on a user operation on the mobile terminal 70. Alternatively, the initial posture is acquired by pressing a button or the like provided on the intraoral camera 10. FIG. 17 is a diagram showing an example of the initial posture. For example, as shown in FIG. 17, three-axis posture information based on the vertical direction LV obtained by the position sensor 90, which is a six-axis sensor, is acquired as the initial posture. This initial posture is held in the mobile terminal 70 or the intraoral camera 10.

[0116] FIG. 18 is a diagram showing an example of an initial posture setting screen on the mobile terminal 70. As shown in FIG. 18, for example, the posture of the intraoral camera 10 in a state where the relationship between the teeth and the intraoral camera 10 is a predetermined relationship is acquired as the initial posture. In the example shown in FIG. 18, the initial posture is a state where the imaging surface S of the intraoral camera 10 is parallel to the front surfaces of the front teeth, and the axial direction of the intraoral camera 10 coincides with the height direction of the front teeth in a plan view of the imaging surface S. Here, the axial direction is a direction passing through the center of the longitudinal direction of the intraoral camera 10 from the handle portion 10b to the head portion 10a of the intraoral camera 10. Also, for example, the axial direction is a direction passing through the center of the vertical direction (column direction) of the imaging surface S (image data).

[0117] The state in which the initial posture is acquired is not limited to this example, and may be any state based on one or more teeth. For example, teeth other than the front teeth may be used. Also, here, "a state in which the axial direction of the intraoral camera 10 coincides with the height (vertical) direction of the front teeth" is specified, but "a state in which the axial direction of the intraoral camera 10 is perpendicular to the height direction of the front teeth (a state in which the axial direction of the intraoral camera 10 coincides with the width (horizontal) direction of the front teeth)" may also be used.

[0118] Furthermore, the terms "parallel," "coincident," and "orthogonal" mentioned here do not need to be strict, and may be "approximately parallel," "approximately coincident," and "approximately orthogonal." In other words, the mobile terminal 70 may instruct the user to take the above states, and the state used as the initial posture may be the posture of the intraoral camera 10 taken by the user based on the instruction.

[0119] Fig. 19 is a diagram showing another example of the setting screen for the initial posture on the mobile terminal 70. As shown in Fig. 19, for example, the posture of the intraoral camera 10 in a state where the relationship between the posture of the user and the posture of the intraoral camera 10 is a predetermined relationship is acquired as the initial posture. In the example shown in Fig. 19, the initial posture is a state where the frontal plane 110 of the user BD and the imaging plane S of the imaging unit are parallel, and the vertical axis Z1 of the user BD and the second direction LB coincide in a planar view on the imaging plane S.

[0120] Note that the state in which the initial posture is acquired is not limited to this example, and may be any posture that can associate the posture of the user BD with the posture of the intraoral camera 10. Furthermore, the posture of the user BD may be defined using one or more of the frontal plane 110, the sagittal plane 111, the horizontal plane 112, the vertical axis, the sagittal-horizontal axis, and the forehead-horizontal axis. For example, here, "a state in which the axial direction LB of the intraoral camera 10 coincides with the vertical axis Z1" is specified, but "a state in which the axial direction LB of the intraoral camera 10 is perpendicular to the vertical axis Z1 (a state in which the axial direction LB coincides with the forehead-horizontal axis)" may also be used.

[0121] Next, the teeth are photographed as described above. Specifically, the area detection unit 101 corrects the orientation of the intraoral camera 10 obtained during the photographing of the teeth using the initial orientation (S232). That is, the area detection unit 101 corrects the orientation of the intraoral camera 10 using the initial orientation so that the orientation of the intraoral camera 10 is the same as when the user is facing forward.

[0122] FIG. 20 is a diagram showing an example of the initial attitude. FIG. 21 is a diagram showing an example of attitude correction. Note that although an example of correcting information on the y-axis is shown here, the same applies when correcting information on other axes. When an angle δ between the vertical direction LV and the y-axis is obtained as the initial attitude as shown in FIG. 20, the area detection unit 101 corrects the vertical direction LV to a vertical direction LV0 as shown in FIG. 21. Furthermore, the corrected vertical direction LV0 is used instead of the vertical direction LV to perform processing such as determining the shooting direction. For example, as shown in FIG. 21, the angle between the corrected vertical direction LV0 and the imaging plane S is calculated as the angle α between the vertical direction LV and the imaging plane S (y-axis).

[0123] The area detection unit 101 may correct the posture itself obtained by the position sensor 90, or may correct values ​​being calculated (such as angles used for determination) instead of correcting the vertical direction LV. A part or all of this correction process may be performed by the area detection unit 101 (mobile terminal 70) or may be performed within the intraoral camera 10.

[0124] Finally, the area detection unit 101 determines the tooth area, the imaging direction, etc. (generates area information) as described above based on the corrected posture (S233).

[0125] As described above, the area detection unit 101 can improve the accuracy of determination by correcting the posture of the intraoral camera 10 according to the posture of the user.

[0126] Although the example of determining the area of ​​the photographed tooth and the photographing direction has been described above as an example of signal processing using the attitude of the intraoral camera 10, signal processing using the attitude of the intraoral camera 10 is not limited to this and may be any processing. For example, the signal processing may be image processing according to the attitude of the intraoral camera 10. Specifically, the signal processing may be processing that uses the attitude of the intraoral camera 10 to rotate image data so that the top of the displayed image is the top of the real space.

[0127] In the above description, an example has been described in which the initial attitude is acquired based on the user's operation or the like in step S231 shown in FIG. 16, but the following method may also be used.

[0128] 15, when a dentist uses the intraoral camera 10 while a user BD is sitting on a dental treatment table, the frontal plane 110 of the upper half of the body of the user BD is tilted along the backrest of the treatment table. That is, as a result of the tilt of the frontal plane 110 of the user BD, the vertical axis Z1 of the user with the upper half of the body tilted along the backrest is tilted relative to the vertical axis Z0 of the body when the user BD is standing upright.

[0129] Therefore, the initial posture can be set based on the tilt angle of the backrest of the dental treatment chair, and in this case, it is not necessary to obtain the initial posture based on the user's operation or the like.

[0130] Fig. 22 is a diagram showing an example of the initial posture in this case. For example, as shown in Fig. 22, the angle θ between the vertical axis Z0, which is the vertical direction (the direction in which gravity acts), and the backrest 121 (vertical axis Z1) of the dental treatment chair 120 on which the user sits may be acquired as the initial posture.

[0131] For example, the angle θ is sent from the dental treatment chair 120 (or its control device, etc.) to the area detection unit 101 provided in the mobile terminal 70 via wireless communication, etc. Note that the method of transmitting this information is not particularly limited, and any method may be used.

[0132] Furthermore, information indicating the angle θ itself may be sent, or information that can identify the angle θ may be sent. For example, if the tilt of the backrest is set in stages, information indicating the tilt stage may be sent. In this case, the area detection unit 101 identifies the angle θ from the acquired tilt stage using a correspondence relationship between a plurality of tilt stages and a plurality of angles θ that is stored in advance. Furthermore, the angle θ or the tilt stage may be specified (input) by a dentist.

[0133] In this way, the area detection unit 101 may determine the initial posture based on the angle of the backrest of the chair (dental treatment chair 120) on which the user (the person whose teeth are to be photographed by the intraoral camera 10) sits. This simplifies the initial setup work, improving convenience. Furthermore, reducing the user's operations may improve the accuracy of correction.

[0134] As described above, the intraoral camera system includes an imaging unit (e.g., intraoral camera 10) that includes handle unit 10b, head unit 10a including an imaging element that generates image data, and neck unit 10c connecting handle unit 10b and head unit 10a and that captures images of teeth in the oral cavity and generates image data, a position sensor 90 that detects the orientation of the imaging unit (e.g., intraoral camera 10), a signal processing unit (e.g., area detection unit 101) that performs signal processing using the orientation of the imaging unit (e.g., intraoral camera 10) detected by the position sensor 90, and a display unit (e.g., touch screen 72) that displays the image data. The signal processing unit acquires an initial orientation, which is a predetermined orientation of the imaging unit (S231), and performs signal processing by regarding a first direction from the handle unit toward the head unit in the initial orientation as a second direction vertically upward along the vertical axis (S232, S233).

[0135] This allows the intraoral camera system to correct the posture of the imaging unit according to the user's posture, for example, thereby improving the accuracy of signal processing.

[0136] For example, the predetermined posture is a posture of the imaging unit in which the posture of the user BD and the posture of the imaging unit have a predetermined relationship.

[0137] For example, the predetermined posture is a state in which the frontal plane 110 of the user BD and the imaging plane S of the imaging unit are parallel, and the vertical axis Z1 of the user BD and the first direction are aligned or perpendicular to each other in a plan view of the imaging plane S. This allows the user to easily obtain the initial posture. Furthermore, improving the accuracy of the initial posture improves the accuracy of correction.

[0138] For example, the predetermined posture is a state in which a predetermined tooth (e.g., a front tooth) is parallel to and directly opposed to the imaging surface S of the imaging unit, and the first direction coincides with or is perpendicular to the height direction of the predetermined tooth in a plan view of the imaging surface S. This allows the user to easily obtain the initial posture. Furthermore, improving the accuracy of the initial posture can improve the accuracy of correction.

[0139] For example, the signal processing unit determines the initial posture based on the angle (e.g., angle θ) of the backrest of a chair (e.g., dental treatment chair 120) on which the user sits. This simplifies the initial setting work, improving convenience.

[0140] Although the intraoral camera system according to the embodiment of the present disclosure has been described above, the present disclosure is not limited to this embodiment.

[0141] For example, although the above description has been given of an example in which the intraoral camera 10 is primarily intended to photograph teeth, the intraoral camera 10 may also be an oral care device equipped with a camera. For example, the intraoral camera 10 may also be an oral irrigator equipped with a camera.

[0142] Furthermore, each processing unit included in the intraoral camera system according to the above embodiment is typically realized as an LSI, which is an integrated circuit. These may be individually implemented as single chips, or some or all of them may be integrated into a single chip.

[0143] Furthermore, the integration is not limited to LSI, but may be realized by dedicated circuits or general-purpose processors. FPGAs (Field Programmable Gate Arrays), which can be programmed after LSI fabrication, or reconfigurable processors, which allow the connections and settings of circuit cells within LSIs to be reconfigured, may also be used.

[0144] In each of the above embodiments, each component may be configured with dedicated hardware, or may be realized by executing a software program suitable for each component. Each component may be realized by a program execution unit such as a CPU or processor reading and executing a software program recorded on a recording medium such as a hard disk or semiconductor memory.

[0145] The present disclosure may also be realized as a signal processing method or the like executed by an intraoral camera system. The present disclosure may also be realized as an intraoral camera, a mobile terminal, or a cloud server included in the intraoral camera system.

[0146] The division of functional blocks in the block diagram is an example, and multiple functional blocks may be realized as a single functional block, one functional block may be divided into multiple blocks, or some functions may be moved to another functional block.Furthermore, the functions of multiple functional blocks having similar functions may be processed in parallel or in time-sharing by a single piece of hardware or software.

[0147] The order in which the steps in the flowchart are executed is merely an example for specifically explaining the present disclosure, and other orders may be used. Some of the steps may be executed simultaneously (in parallel) with other steps.

[0148] Although the intraoral camera system according to one or more aspects has been described based on the embodiments, the present disclosure is not limited to these embodiments. As long as it does not deviate from the spirit of the present disclosure, various modifications conceivable by a person skilled in the art to the present embodiments and forms constructed by combining components of different embodiments may also be included within the scope of one or more aspects. [Industrial Applicability]

[0149] The present disclosure is applicable to intraoral camera systems. [Explanation of symbols]

[0150] 10 Intraoral Camera 10a head part 10b Handle 10c neck 12. Shooting optical system 14 Image sensor 26A 1st LED 26B Second LED 26C Third LED 26D 4th LED 36, 40 Actuator 50 Central Control Unit 52 Image processing section 54 LED control unit 56 Lens Driver 58 Wireless Communication Module 60 Power supply control unit 62 Controller 64 memory 66 Batteries 68 Coil 69 Charger 70 Mobile Devices 72 Touchscreen 80 Cloud Servers 90 Position Sensor 101 Area detection unit 102 User information acquisition unit 103 Identification unit 104 Tooth image generation unit 105 Type Identification Unit 106 Estimation Model 110 Frontal 111 sagittal plane 112 Horizontal plane 120 Dental Treatment Chair 121 Backrest

Claims

1. an imaging unit including a handle unit, a head unit including an imaging element for generating image data, and a neck unit connecting the handle unit and the head unit, the imaging unit capturing images of teeth in the oral cavity and generating image data; a sensor for detecting the attitude of the imaging unit; a signal processing unit that performs signal processing using the attitude of the imaging unit detected by the sensor; a display unit that displays the image data, The signal processing unit acquire an initial posture of the imaging unit in which the frontal plane of the user and an imaging surface of the imaging unit are parallel, and a vertical axis of the user and a first direction from the handle unit toward the head unit are aligned or perpendicular to each other in a plan view of the imaging surface; The signal processing is performed by regarding the first direction in the initial attitude as a second direction vertically upward along a vertical axis. Intraoral camera system.

2. An imaging unit including a handle portion, a head portion including an image sensor for generating image data, and a neck portion connecting the handle portion and the head portion, and for capturing images of teeth in the oral cavity and generating image data; a sensor for detecting the attitude of the imaging unit; a signal processing unit that performs signal processing using the attitude of the imaging unit detected by the sensor; a display unit that displays the image data, The signal processing unit an initial posture of the photographing unit in which a predetermined tooth and an imaging surface of the photographing unit are placed parallel to each other and a first direction from the handle unit toward the head unit and a height direction of the predetermined tooth are aligned or perpendicular to each other in a plan view of the imaging surface; The signal processing is performed by regarding the first direction in the initial attitude as a second direction vertically upward along a vertical axis. Intraoral camera system.

3. An imaging unit including a handle portion, a head portion including an image sensor for generating image data, and a neck portion connecting the handle portion and the head portion, and for capturing images of teeth in the oral cavity and generating image data; a sensor for detecting the attitude of the imaging unit; a signal processing unit that performs signal processing using the attitude of the imaging unit detected by the sensor; a display unit that displays the image data, The signal processing unit determining an initial posture, which is a predetermined posture of the imaging unit, based on an angle of the back of a chair on which the user is sitting; The signal processing is performed by regarding a first direction from the handle portion toward the head portion in the initial posture as a second direction vertically upward along a vertical axis. Intraoral camera system.

4. an imaging unit including a handle, a head including an imaging element for generating image data, and a neck connecting the handle and the head, which images teeth in the oral cavity and generates image data; Detecting the posture of the imaging unit; performing signal processing using the detected attitude of the imaging unit; Displaying the image data; In the signal processing, acquire an initial posture of the imaging unit in which the frontal plane of the user and an imaging surface of the imaging unit are parallel, and a vertical axis of the user and a first direction from the handle unit toward the head unit are aligned or perpendicular to each other in a plan view of the imaging surface; The signal processing is performed by regarding the first direction in the initial attitude as a second direction vertically upward along a vertical axis. Signal processing methods.

5. An imaging unit including a handle portion, a head portion including an imaging element that generates image data, and a neck portion that connects the handle portion and the head portion, images teeth in the oral cavity and generates image data, Detecting the posture of the imaging unit; performing signal processing using the detected attitude of the imaging unit; Displaying the image data; In the signal processing, an initial posture of the photographing unit in which a predetermined tooth and an imaging surface of the photographing unit are placed parallel to each other and a first direction from the handle unit toward the head unit and a height direction of the predetermined tooth are aligned or perpendicular to each other in a plan view of the imaging surface; The signal processing is performed by regarding the first direction in the initial attitude as a second direction vertically upward along a vertical axis. Signal processing methods.

6. An imaging unit including a handle portion, a head portion including an imaging element for generating image data, and a neck portion connecting the handle portion and the head portion, images the teeth in the oral cavity and generates image data, Detecting the posture of the imaging unit; performing signal processing using the detected attitude of the imaging unit; Displaying the image data; In the signal processing, determining an initial posture, which is a predetermined posture of the imaging unit, based on an angle of the back of a chair on which the user is sitting; The signal processing is performed by regarding a first direction from the handle portion toward the head portion in the initial posture as a second direction vertically upward along a vertical axis. Signal processing methods.

Images