Data processing device, data processing method, and data processing program

Abstract

To provide a technique capable of easily and accurately identifying a tooth position.SOLUTION: The data processing device 10 includes: an input unit 101 configured to receive three-dimensional data including three-dimensional positional information corresponding to each of a plurality of points representing a surface shape of an object in an oral cavity including teeth; a computation unit 102 configured to execute processing for identifying each of a plurality of portions of a tooth on the basis of the three-dimensional data input from the input unit 101; and an output unit 103 configured to output a processing result by the computation unit 102.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present disclosure relates to a data processing apparatus, a data processing method, and a data processing program for processing three-dimensional data.

Background Art

[0002] It is known that keeping teeth in a healthy state helps to extend the healthy life span. Therefore, the introduction of so-called national dental checkups, which obligate all citizens to have an annual dental examination, is being considered. In a dental examination, a practitioner such as a dentist examines the health conditions of a patient's teeth and gums and records the examination results in a medical record. At this time, the practitioner needs to examine the health conditions of the teeth while identifying the types and positions (surface positions) of the teeth and record the examination results in the medical record for each type and position of the teeth.

[0003] In this regard, Japanese Unexamined Patent Application Publication No. 2020-96691 discloses a data processing apparatus that identifies the types of teeth based on three-dimensional data including three-dimensional position information corresponding to each of a plurality of points constituting the teeth.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] According to the data processing device disclosed in Patent Document 1, the operator can accurately identify the type of tooth. However, when performing a dental examination, it is necessary to identify not only the type of tooth being examined, but also the location of the tooth being examined. However, currently, the operator must identify the location of the tooth themselves, and depending on the individual operator's skill and ability, this may lead to the dental examination taking a long time, or the operator mistakenly identifying the location of a tooth where caries or other problems have occurred, resulting in incorrect information being recorded in the medical record.

[0006] This disclosure was made to solve such problems and aims to provide a technology that can easily and accurately identify the location of teeth. [Means for solving the problem]

[0007] According to one example of this disclosure, a data processing device for processing three-dimensional data is provided. The data processing device is specific An input unit receives three-dimensional data, which includes three-dimensional positional information corresponding to each of several points that represent the surface shape of objects in the oral cavity, including teeth, and based on the three-dimensional data input from the input unit, specific teeth surface odor and were divided It comprises a calculation unit that performs processing to identify each of multiple parts, and an output unit that outputs the processing results from the calculation unit.

[0008] According to an example of this disclosure, a data processing method for processing three-dimensional data by a computer is provided. The data processing method includes the following processes performed by the computer: specific A step of acquiring three-dimensional data including three-dimensional positional information corresponding to each of multiple points that represent the surface shape of an object in the oral cavity, including teeth, and based on the three-dimensional data acquired in the acquisition step, specific teeth surface odor and were divided The process includes the step of outputting the processing result obtained by performing a process to identify each of multiple parts.

[0009] According to one example of this disclosure, a data processing program is provided for processing three-dimensional data using a computer. The data processing program allows the computer to: specific A step of acquiring three-dimensional data including three-dimensional positional information corresponding to each of multiple points that represent the surface shape of an object in the oral cavity, including teeth, and based on the three-dimensional data acquired in the acquisition step, specific teeth surface odor and were divided The system performs the steps of: executing a process to identify each of multiple parts, and outputting the processing results from the first step. [Effects of the Invention]

[0010] According to this disclosure, based on three-dimensional data including three-dimensional positional information corresponding to each of a plurality of points representing the surface shape of an object in the oral cavity, including a tooth, processing can be performed to identify each of a plurality of parts of the tooth, and the processing result can be output, thereby enabling easy and highly accurate identification of the parts of the tooth. [Brief explanation of the drawing]

[0011] [Figure 1] This figure shows an example of the application of the data processing device according to the embodiment. [Figure 2] This is a block diagram showing the hardware configuration of a data processing device according to an embodiment. [Figure 3] This is a diagram illustrating an example of a location on a tooth. [Figure 4] This diagram illustrates variations in the location of each tooth: the first molar, second molar, third molar, first premolar, or second premolar. [Figure 5] This diagram illustrates variations in the location of each tooth, specifically the canine, lateral incisor, or central incisor. [Figure 6] This is a block diagram showing the functional configuration of a data processing device according to an embodiment. [Figure 7] This figure shows an example of the tooth location identification result obtained by the data processing device according to the embodiment. [Figure 8] It is a diagram showing an example of an identification result of a part in a tooth obtained by the data processing device according to the embodiment. [Figure 9] It is a diagram showing an example of an identification result of a part in a tooth obtained by the data processing device according to the embodiment. [Figure 10] It is a diagram showing an example of an identification result of a part in a tooth obtained by the data processing device according to the embodiment. [Figure 11] It is a diagram showing an example of data processing using an estimation model by the data processing device according to the embodiment. [Figure 12] It is a diagram showing an example of data processing using an estimation model by the data processing device according to the embodiment. [Figure 13] It is a diagram showing an example of data processing using an estimation model by the data processing device according to the embodiment. [Figure 14] It is a diagram showing a modified example of the identification process of a part in a tooth by the data processing device according to the embodiment. [Figure 15] It is a diagram showing an example of an identification process using tooth surface extraction logic by the data processing device according to the embodiment. [Figure 16] It is a diagram showing an example of an identification process using tooth surface extraction logic by the data processing device according to the embodiment. [Figure 17] It is a diagram showing an example of an identification process using tooth surface extraction logic by the data processing device according to the embodiment. [Figure 18] It is a diagram showing an example of an identification process using tooth surface extraction logic by the data processing device according to the embodiment. [Figure 19] It is a diagram showing an example of an identification process using tooth surface extraction logic by the data processing device according to the embodiment. [Figure 20] It is a diagram showing an example of an identification process using tooth surface extraction logic by the data processing device according to the embodiment. [Figure 21] It is a diagram showing an example of an identification process using tooth surface extraction logic by the data processing device according to the embodiment. [Figure 22] This figure shows an example of identification processing using tooth surface extraction logic by a data processing device according to the embodiment. [Figure 23] This figure shows an example of identification processing using tooth surface extraction logic by a data processing device according to the embodiment. [Figure 24] This figure shows an example of identification processing using tooth surface extraction logic by a data processing device according to the embodiment. [Figure 25] This figure shows an example of identification processing using tooth surface extraction logic by a data processing device according to the embodiment. [Figure 26] This figure shows an example of identification processing using tooth surface extraction logic by a data processing device according to the embodiment. [Figure 27] This figure shows an example of identification processing using tooth surface extraction logic by a data processing device according to the embodiment. [Figure 28] This figure shows an example of identification processing using tooth surface extraction logic by a data processing device according to the embodiment. [Figure 29] This is a flowchart that defines the data processing performed by the data processing device according to the embodiment. [Figure 30] This figure shows an example of an electronic medical record generated by the data processing device according to the embodiment. [Figure 31] This figure shows an example of an electronic medical record generated by the data processing device according to the embodiment. [Figure 32] This figure shows an example of an electronic medical record generated by the data processing device according to the embodiment. [Figure 33] This figure shows an example of an electronic medical record generated by the data processing device according to the embodiment. [Figure 34] This figure shows an example of an electronic medical record generated by the data processing device according to the embodiment. [Figure 35] This figure shows an example of an electronic medical record generated by the data processing device according to the embodiment. [Modes for carrying out the invention]

[0012] Embodiments of this disclosure will be described in detail with reference to the drawings. Parts identical or corresponding to those shown in the drawings are denoted by the same reference numerals, and their descriptions will not be repeated.

[0013] [Examples of application] An example of the application of the data processing device 10 according to the embodiment will be described with reference to Figure 1. Figure 1 is a diagram showing an example of the application of the data processing device 10 according to the embodiment.

[0014] The user can acquire three-dimensional data of objects within the oral cavity by scanning the subject's oral cavity using a three-dimensional scanner 2 (for example, an optical scanner). "Objects within the oral cavity" include biological tissues such as teeth and gums, and processed parts such as implants, prepared teeth, abutment teeth, and prostheses. The three-dimensional data includes three-dimensional positional information corresponding to each of multiple points (point cloud) that represent the surface shape of the objects within the oral cavity. Specifically, the three-dimensional data includes the coordinates (X, Y, Z) of each point in the point cloud that represents the surface shape of the objects within the oral cavity in predetermined horizontal (X-axis direction), vertical (Y-axis direction), and height direction (Z-axis direction). Furthermore, the three-dimensional data includes color information that represents the actual color of the part (surface portion of the object) corresponding to each point in the point cloud that represents the surface shape of the objects within the oral cavity.

[0015] Furthermore, the user may use a CT (Computed Tomography) imaging device (not shown) to image the subject's oral cavity, not limited to the three-dimensional scanner 2. The CT imaging device includes a CBCT (Cone Beam Coherence Tomography) device that uses a cone-shaped cone beam (X-ray beam) to perform computed tomography of the subject's maxilla and mandible. By imaging the subject's maxilla and mandible using the CT imaging device, the user can obtain three-dimensional volume (voxel) data of the hard tissue parts (bone, teeth, etc.) other than the soft tissue parts (skin, gums, etc.) around the subject's maxilla and mandible. Alternatively, the user may use an OCT (Optical Coherence Tomography) device that uses optical coherence tomography to perform computed tomography of the subject's maxilla and mandible. The user can generate tomographic images or external images of the object by tomographic processing of the volume data of the object obtained by the CT imaging device or OCT device. Furthermore, by processing the volume (voxel) data of the tooth obtained by a CT imaging device or OCT device, such as changing the voxels to dots, the user can generate three-dimensional data that includes three-dimensional positional information corresponding to each of the multiple points (point cloud) representing the surface shape of the tooth.

[0016] As described above, the "three-dimensional data" includes at least one of the following: three-dimensional scanner data (IOS (Intra Oral Scanner) data) obtained by scanning the teeth using three-dimensional scanner 2, CBCT data obtained by imaging the teeth using cone-beam CT, and OCT data obtained by imaging the teeth using optical coherence tomography.

[0017] "Users" include practitioners (such as doctors) or assistants (such as dental assistants, dental hygienists, dental technicians, and nurses) in various fields such as dentistry, oral surgery, orthopedics, plastic surgery, and cosmetic surgery. "Subjects" include patients in dentistry, oral surgery, orthopedics, plastic surgery, and cosmetic surgery. The 3D scanner 2 is a so-called intraoral scanner (IOS) capable of optically imaging the inside of a patient's oral cavity using methods such as confocal or triangulation, and can acquire positional information for each point in a point cloud representing the surface shape of an object to be scanned (for example, teeth and gums in the oral cavity) placed in a certain coordinate space.

[0018] The data processing device 10 generates a rendering image (appearance image) that shows the surface shape of an object in the oral cavity in three dimensions, based on the three-dimensional data acquired by the three-dimensional scanner 2. A "rendering image" is an image generated by processing or editing certain data. For example, a user can generate a rendering image that shows a two-dimensional object (a part of the object that can be shown by IOS data) viewed from a predetermined viewpoint by processing or editing the three-dimensional data of the object acquired by the three-dimensional scanner 2. Furthermore, by changing the predetermined viewpoint in multiple directions, the user can generate multiple rendering images that show the two-dimensional object viewed from multiple directions.

[0019] The data processing device 10 displays a rendering image generated based on the three-dimensional data acquired by the three-dimensional scanner 2 on the display 20. The user can acquire three-dimensional data of objects in the oral cavity step by step while viewing the rendering image displayed on the display 20.

[0020] In a dental check-up, the user, such as a dentist or dental hygienist, examines the health of the patient's teeth and gums and records the results in the patient's medical record. At this time, the user must examine the health of the teeth while identifying the type and location (surface area) of each tooth, and record the results in the medical record for each type and location. Therefore, the user must conduct the dental check-up while identifying the type and location of the tooth being examined. However, if the user identifies the type and location of the tooth themselves, depending on the user's individual skills and abilities, the dental check-up may take too long, or they may mistakenly identify a tooth with caries or other problems, leading to incorrect information being recorded in the medical record.

[0021] Therefore, the data processing device 10 according to the embodiment uses AI (Artificial Intelligence) technology to identify the type and location of the tooth to be scanned, based on three-dimensional data showing the surface shape of an object in the oral cavity, including teeth.

[0022] The "types of teeth" include the central incisors, lateral incisors, canines, first premolars, second premolars, first molars, second molars, and third molars on the upper right side. The "types of teeth" include the central incisors, lateral incisors, canines, first premolars, second premolars, first molars, second molars, and third molars on the upper left side. The "types of teeth" include the central incisors, lateral incisors, canines, first premolars, second premolars, first molars, second molars, and third molars on the lower right side. The "types of teeth" include the central incisors, lateral incisors, canines, first premolars, second premolars, first molars, second molars, and third molars on the lower left side.

[0023] "The part of the tooth" includes each of the parts when the surface of each tooth is divided into multiple parts as described above. As will be explained in more detail later, for example, in the case of the second molar on the left side of the mandible, "the part of the tooth" includes each of the parts when the surface of the second molar on the left side of the mandible is divided into multiple parts (occlusal surface, distal surface, mesial surface, lingual surface, buccal surface), such as the occlusal surface, distal surface, mesial surface, lingual surface, and buccal surface.

[0024] The data processing device 10 generates image data for displaying a diagram or table showing the identification results of the tooth type and tooth location. The image data includes data for displaying the tooth in at least one of two or three dimensions, along with the identification results. By outputting the image data to the display 20, the data processing device 10 can display the diagram or table showing the identification results on the display 20.

[0025] Furthermore, the data processing device 10 may identify the processing area on a tooth from among the objects in the oral cavity that were scanned, based on the three-dimensional data. The "processing area" includes implants, prepared teeth, abutment teeth, and prostheses. An implant includes the implant body corresponding to the root portion, the scan body, and the abutment corresponding to the support portion. In addition, the implant body, abutment, abutment tooth, and prepared tooth may be fitted with inlays, onlays, crowns, bridges, veneers, and dentures, etc.

[0026] The data processing device 10 may identify lesion sites from the scanned oral cavity objects (teeth, gums) based on the three-dimensional data. A "lesion site" is an area in the oral cavity that has an abnormality, and includes, for example, areas where caries has occurred, areas where periodontal disease has occurred, areas where gingivitis has occurred, and areas where teeth are missing. Furthermore, a "lesion site" is not limited to areas that are actually diagnosed as diseased, but may also include areas that are likely to cause disease, such as areas where tartar has accumulated or areas where plaque has attached.

[0027] The data processing device 10 may estimate the depth of the periodontal pocket in the tooth based on the three-dimensional data. Furthermore, the data processing device 10 may estimate the position of premature contact in the tooth based on the three-dimensional data.

[0028] Hereinafter, the various processes performed by the data processing device 10, such as the process of identifying the type of tooth, the process of identifying the location of the tooth, the process of identifying the processed area of ​​the tooth, the process of identifying the lesion site of the tooth, the process of estimating the depth of the periodontal pocket of the tooth, and the process of estimating the premature contact position of the tooth, will be collectively referred to as "data processing." The results obtained from the data processing (identification results, estimation results) will also be collectively referred to as "processing results."

[0029] The data processing device 10 may further add at least one piece of information from among the processed area, lesion area, periodontal pocket depth, and early contact location to the figure or table showing the identification results of the tooth type and tooth location. By outputting the image data showing these processing results to the display 20, the data processing device 10 can display on the display 20 images showing the processing results such as lesion area, processed area, periodontal pocket depth, or early contact location obtained for each tooth type and tooth location.

[0030] The data processing device 10 can record the processing results obtained for each type and location of tooth, such as lesion sites, processing sites, periodontal pocket depth, or premature contact locations, in the storage device 13 (described later) as an electronic medical record, or transmit them to an external server device via the communication device 18. In addition to electronic medical records, the data processing device 10 may also record the processing results obtained for each type and location of tooth, such as lesion sites, processing sites, periodontal pocket depth, or premature contact locations, in the storage device 13, or transmit them to an external server device via the communication device 18, as a dental hygienist's work record for recording the work of a dental hygienist, or as a document related to dental hygienist's practical guidance for recording the content of dental hygienist instruction.

[0031] For example, as shown in Figure 1, the data processing device 10 displays an image showing the surface shape of a part of the dentition on the display 20, and in the image displayed on the display 20, it color-codes the areas of teeth where caries has occurred, as identified by data processing, and color-codes the areas of gums where periodontal disease has occurred, as identified by data processing. The data processing device 10 also records the type of tooth and the location of the tooth where a lesion such as caries or periodontal disease has occurred, as identified by data processing, as an electronic medical record, and displays the processing results on the display 20 using a figure or table.

[0032] In this way, the data processing device 10 can automatically diagnose and / or suggest various examinations such as tooth type, tooth location, processing location, lesion location, periodontal pocket depth, and premature contact location based on three-dimensional data. As a result, the user does not need to identify the tooth type or tooth location themselves, or perform various examinations such as processing location, lesion location, periodontal pocket depth, or premature contact location themselves, but can view the image showing the processing results displayed on the display 20, or view the electronic medical record in which the processing results are recorded via the display 20. Therefore, the user can avoid spending a lot of time on dental checkups, or mistakenly identifying the location of a tooth where caries or other problems have occurred and recording incorrect information in the medical record.

[0033] [Hardware configuration of data processing unit] The hardware configuration of the data processing device 10 according to the embodiment will be described with reference to Figure 2. Figure 2 is a block diagram showing the hardware configuration of the data processing device 10 according to the embodiment. The data processing device 10 may be implemented, for example, as a general-purpose computer, or as a dedicated computer for the three-dimensional scanner 2.

[0034] As shown in Figure 2, the data processing system 1 according to this embodiment comprises a data processing device 10, a three-dimensional scanner 2, a display 20, and input devices 30 such as a keyboard 31 and a mouse 32. The data processing device 10 comprises, as its main hardware elements, an arithmetic unit 11, a memory 12, a storage device 13, a scanner interface 14, a display interface 15, an input device interface 16, a storage medium interface 17, and a communication device 18.

[0035] The arithmetic unit 11 is an arithmetic entity (computer) that performs various processes by executing various programs, and is an example of an "arithmetic unit". The arithmetic unit 11 is composed of a processor such as a CPU (Central Processing Unit), MPU (Micro Processing Unit), TPU (Tensor Processing Unit), or GPU (Graphics Processing Unit). A processor, which is an example of the arithmetic unit 11, has the function of performing predetermined processes by executing predetermined programs, but some or all of these functions may be implemented using dedicated hardware circuits such as an ASIC (Application Specific Integrated Circuit) or FPGA (Field Programmable Gate Array). The term "processor" is not limited to processors in the narrow sense that perform processing in a stored-program manner, such as a CPU, MPU, TPU, or GPU, but may also include hardwired circuits such as ASICs or FPGAs. Furthermore, the arithmetic unit 11 is not limited to von Neumann type computers such as CPUs or GPUs, but may also be composed of non-von Neumann type computers such as quantum computers or optical computers. The arithmetic unit 11 described above can also be read as a processing circuitry that performs predetermined processes. The arithmetic unit 11 may consist of one chip or multiple chips. Furthermore, the processor and associated processing circuits may consist of multiple computers interconnected by wired or wireless connections via a local area network or wireless network. The processor and associated processing circuits may also consist of a cloud computer that remotely performs calculations based on input data and outputs the calculation results to other devices located at a distance.

[0036] Memory 12 includes a volatile storage area (for example, a working area) for temporarily storing program code, work memory, etc., when the arithmetic unit 11 executes various programs. Examples of storage units include volatile memory such as DRAM (dynamic random access memory) and SRAM (static random access memory), or non-volatile memory such as ROM (read-only memory) and flash memory.

[0037] The storage device 13 stores various programs or data executed by the arithmetic unit 11. The storage device 13 may be one or more non-transitory computer-readable media or one or more computer-readable storage media. Examples of storage devices 13 include HDDs (Hard Disk Drives) and SSDs (Solid State Drives).

[0038] The storage device 13 stores the data processing program 100 and the estimation model 50. The data processing program 100 describes the data processing that the computing device 11 uses to identify the type of tooth and the location of the tooth based on three-dimensional data showing the surface shape of objects in the oral cavity, using the estimation model 50.

[0039] The estimated model 50 includes a neural network 51 and parameters 52 used by the neural network 51.

[0040] The neural network 51 is trained by machine learning to perform various data processing operations based on three-dimensional data of objects in the oral cavity. Specifically, the neural network 51 is trained by machine learning (e.g., supervised learning) to identify the type of tooth when three-dimensional data is directly input. The neural network 51 is trained by machine learning (e.g., supervised learning) to identify each of multiple parts of a tooth when three-dimensional data is directly input. Here, "three-dimensional data is directly input" is not limited to the pattern in which the three-dimensional data acquired by the three-dimensional scanner 2 is input to the neural network 51 without modification, but also includes the pattern in which the data input to the neural network 51 is substantially the same as the original three-dimensional data acquired by the three-dimensional scanner 2, even if the three-dimensional data acquired by the three-dimensional scanner 2 has been slightly modified by preprocessing. The neural network 51 is trained by machine learning (e.g., supervised learning) to identify the processed area of ​​a tooth when three-dimensional data is directly input. The neural network 51 is trained by machine learning (e.g., supervised learning) to identify lesion sites in teeth by directly inputting three-dimensional data. The neural network 51 is also trained by machine learning (e.g., supervised learning) to estimate the depth of periodontal pockets in teeth by directly inputting three-dimensional data.

[0041] The neural network 51 may be any algorithm applicable to the neural network 51 of this embodiment, such as an autoencoder, convolutional neural network (CNN), recurrent neural network (RNN), transformer, or generative adversarial network (GAN). The estimation model 50 is not limited to the neural network 51; it may also include other known algorithms such as Bayesian estimation or support vector machines (SVM).

[0042] Parameter 52 includes weighting coefficients used in calculations performed by the neural network 51 and judgment values ​​used for making decisions during calculations.

[0043] The scanner interface 14 is an example of an "input unit." The scanner interface 14 acquires three-dimensional data showing the surface shape of objects in the oral cavity. For example, the scanner interface 14 is communicatively connected to the three-dimensional scanner 2 and acquires three-dimensional data from the three-dimensional scanner 2. Alternatively, the scanner interface 14 may be communicatively connected to a CT imaging device (not shown) and acquire three-dimensional data showing the surface shape of teeth generated based on the volume (voxel) data of teeth obtained by the CT imaging device. The three-dimensional data input from the scanner interface 14 is stored in the memory 12 or storage device 13 and used when the arithmetic unit 11 performs data processing.

[0044] The display interface 15 is an interface for connecting the display 20 and is an example of an "output unit". The display interface 15 enables data input and output between the data processing device 10 and the display 20. For example, the data processing device 10 outputs image data to the display 20 via the display interface 15 for displaying a figure or table showing the identification results of tooth type and tooth location. Based on the image data received via the display interface 15, the display 20 displays a figure or table showing the identification results of tooth type and tooth location.

[0045] The input device interface 16 is an interface for connecting input devices 30, such as a keyboard 31 and a mouse 32. The input device interface 16 enables data input and output between the data processing device 10 and the input devices 30. For example, a user can use the input device 30 to input command signals to move a cursor in a figure or table displayed on the display 20, or to process a figure or table. The data processing device 10 performs processing based on the command signals received via the input device interface 16.

[0046] The storage medium interface 17 reads various types of data stored on the removable disk 40, which is the storage medium, and writes various types of data to the removable disk 40. For example, the storage medium interface 17 may obtain a data processing program 100 from the removable disk 40, or it may write electronic medical record data generated by the arithmetic unit 11 to the removable disk 40. The removable disk 40 may be one or more non-transitory computer-readable media or one or more computer-readable storage media. When the arithmetic unit 11 obtains three-dimensional data showing the surface shape of an object in the oral cavity from the removable disk 40 via the storage medium interface 17, the storage medium interface 17 may be an example of an "input unit". Also, when the arithmetic unit 11 outputs electronic medical record data to the removable disk 40 via the storage medium interface 17, the storage medium interface 17 may be an example of an "output unit".

[0047] The communication device 18 transmits and receives data to and from an external device via wired or wireless communication. For example, the communication device 18 acquires three-dimensional data showing the surface shape of an object in the oral cavity from an external device. The three-dimensional data acquired by the communication device 18 is stored in the memory 12 or storage device 13 and used by the arithmetic unit 11 when it performs data processing. The communication device 18 may also output electronic medical record data generated by the arithmetic unit 11 to an external device. When the arithmetic unit 11 acquires three-dimensional data showing the surface shape of an object in the oral cavity from an external device via the communication device 18, the communication device 18 can be an example of an "input unit". Also, when the arithmetic unit 11 outputs electronic medical record data to an external device via the communication device 18, the communication device 18 can be an example of an "output unit".

[0048] [Location on the tooth] Referring to Figures 3 to 5, each of the multiple regions constituting the tooth surface identified by the data processing device 10 will be described. Figure 3 is a diagram illustrating an example of a region on a tooth.

[0049] As shown in Figure 3, the surface of each tooth in the oral cavity is virtually divided into multiple regions depending on the type of tooth. For the maxillary first molar, second molar, third molar, first premolar, or second premolar, the region of each tooth includes at least one of the occlusal surface, distal surface, mesial surface, palatal surface, and buccal surface. For the maxillary canine, lateral incisor, or central incisor, the region of each tooth includes at least one of the distal surface, mesial surface, palatal surface, and labial surface. For the mandibular first molar, second molar, third molar, first premolar, or second premolar, the region of each tooth includes at least one of the occlusal surface, distal surface, mesial surface, lingual surface, and buccal surface. For mandibular canines, lateral incisors, or central incisors, the site for each tooth includes at least one of the following: distal surface, mesial surface, lingual surface, and labial surface.

[0050] Specifically, in the example shown in Figure 3, for the first molar, second molar, third molar, first premolar, and second premolar on the upper right or upper left side, the surface of each tooth is divided into five surfaces: occlusal, distal, mesial, palatal, and buccal. For the canine, lateral incisor, and central incisor on the upper right or upper left side, the surface of each tooth is divided into four surfaces: distal, mesial, palatal, and labial. For the first molar, second molar, third molar, first premolar, and second premolar on the lower right or lower left side, the surface of each tooth is divided into five surfaces: occlusal, distal, mesial, lingual, and buccal. For the canines, lateral incisors, and central incisors on the right or left side of the mandible, the surface of each tooth is divided into four surfaces: the distal surface, the mesial surface, the lingual surface, and the labial surface.

[0051] Here, the surfaces that come into contact with each of the maxillary first premolars, second premolars, first molars, second molars, and third molars and each of the mandibular first premolars, second premolars, first molars, second molars, and third molars are called occlusal surfaces. The direction away from the apex of the dental arch (between the left and right central incisors) is called distal, and the surface facing distal is called the distal surface. The direction towards the apex of the dental arch is called mesial, and the surface facing mesial is called the mesial surface. In the maxillary dental arch, the side closer to the palate is called the palatal side, and the surface facing palate is called the palatal surface. In the mandibular dental arch, the side closer to the tongue is called the lingual side, and the surface facing lingual is called the lingual surface. The side of the upper or lower jaw dentition closest to the lips is called the labial side, and the surface facing the lips is called the labial surface. The side of the upper or lower jaw dentition closest to the cheek is called the buccal side, and the surface facing the cheek is called the buccal surface.

[0052] Furthermore, regarding the first molar, second molar, third molar, first premolar, or second premolar, the parts of each tooth may be divided as shown in Figure 4, not limited to the example shown in Figure 3. Figure 4 is a diagram illustrating variations in the parts of each tooth: the first molar, second molar, third molar, first premolar, or second premolar.

[0053] For the maxillary first molar, second molar, third molar, first premolar, or second premolar, the region of each tooth includes the multiple cusps that make up the tooth. Similarly, for the mandibular first molar, second molar, third molar, first premolar, or second premolar, the region of each tooth includes the multiple cusps that make up the tooth.

[0054] For example, as shown in Figure 4, the surface of the second molar on the left side of the mandible is divided into four cusps: a distal buccal cusp, a distal lingual cusp, a mesiobuccal cusp, and a mesiobuccal cusp. Similarly, the surface of the second molar on the right side of the mandible is divided into four cusps: a distal buccal cusp, a distal lingual cusp, a mesiobuccal cusp, and a mesiobuccal cusp. Although not shown in the illustration, the surface of the first molar on the left side of the maxilla is also divided into four cusps: a distal buccal cusp, a distal palatal cusp, a mesiobuccal cusp, and a mesiopalatal cusp. Similarly, the surface of the first molar on the right side of the maxilla is also divided into four cusps: a distal buccal cusp, a distal palatal cusp, a mesiobuccal cusp, and a mesiopalatal cusp.

[0055] Here, the cusp that is distal and closer to the buccal side is called the distal-buccal cusp. The cusp that is distal and closer to the lingual side is called the distal-lingual cusp. The cusp that is mesial and closer to the buccal side is called the mesiobuccal cusp. The cusp that is mesial and closer to the lingual side is called the mesiolar cusp. The cusp that is distal and closer to the palatal side is called the distal-palatal cusp. The cusp that is mesial and closer to the palatal side is called the mesiopalatal cusp.

[0056] Furthermore, the term "cusp" may be used not only for the first and second molars, but also for other teeth such as the third molar, first premolar, or second premolar. For example, the tip of the canine tooth may be referred to as a cusp. Also, the number of cusps in a molar is not limited to four, but may be, for example, two to five. In addition, for canines, lateral incisors, or central incisors, the parts of each tooth may be divided as shown in Figure 5, not limited to the examples shown in Figure 3. Figure 5 is a diagram illustrating variations in the parts of each tooth, such as canines, lateral incisors, or central incisors.

[0057] As shown in Figure 5, for the maxillary canines, lateral incisors, or central incisors, the region of each tooth includes multiple surfaces that constitute the labial surface. Similarly, for the mandibular canines, lateral incisors, or central incisors, the region of each tooth includes multiple surfaces that constitute the labial surface.

[0058] In the example shown in Figure 5, for maxillary canines, lateral incisors, or central incisors, the labial surface of each tooth is divided into three surfaces: the cervical region, the central region, and the incisal region. Similarly, for mandibular canines, lateral incisors, or central incisors, the labial surface of each tooth is divided into three surfaces: the cervical region, the central region, and the incisal region.

[0059] Here, the part of the labial surface closest to the gums is called the cervical region. The part of the labial surface closest to the tip of the tooth is called the incisal edge. The part of the labial surface located between the cervical region and the incisal edge is called the central region.

[0060] As illustrated in Figures 3 to 5, the user can set the location of the tooth using the input device 30 or the like. Note that the division of the tooth location illustrated in Figures 3 to 5 is just one example, and the user may divide the tooth surface into multiple locations in other ways, or combine the tooth locations illustrated in Figures 3 to 5.

[0061] [Functional Configuration of Data Processing Devices] The functional configuration of the data processing device 10 will be explained with reference to Figure 6. Figure 6 is a block diagram showing the functional configuration of the data processing device 10 according to an embodiment.

[0062] As shown in Figure 6, the data processing device 10 includes an input unit 101, an arithmetic unit 102, and an output unit 103 as functional units for data processing. Each of these functions is realized when the arithmetic unit 11 of the data processing device 10 executes a data processing program 100.

[0063] The input unit 101 is a functional unit implemented by at least one of the scanner interface 14, the storage medium interface 17, and the communication device 18. Three-dimensional data showing the surface shape of objects in the oral cavity is input to the input unit 101.

[0064] The calculation unit 102 is a functional unit implemented by the arithmetic device 11. The calculation unit 102 performs predetermined data processing on the three-dimensional data input from the input unit 101.

[0065] For example, the calculation unit 102 identifies each of the multiple locations in a tooth based on the three-dimensional data input from the input unit 101 and an estimation model 50 that has been machine-learned to identify each of the multiple locations in the tooth from which the three-dimensional data is to be acquired. Specifically, the calculation unit 102 identifies whether each of the multiple locations in the tooth from which the three-dimensional data is to be acquired belongs to an upper jaw tooth or a lower jaw tooth.

[0066] More specifically, if the tooth to be identified is the maxillary first molar, second molar, third molar, first premolar, or second premolar, the calculation unit 102 estimates, using the estimation model 50, which of the multiple parts constituting the surface of the tooth to be identified corresponds to: the occlusal surface, distal surface, mesial surface, palatal surface, or buccal surface. If the tooth to be identified is the maxillary canine, lateral incisor, or central incisor, the calculation unit 102 estimates, using the estimation model 50, which of the multiple parts constituting the surface of the tooth to be identified corresponds to: the distal surface, mesial surface, palatal surface, or labial surface. If the tooth to be identified is the mandibular first molar, second molar, third molar, first premolar, or second premolar, the calculation unit 102 estimates, using the estimation model 50, which of the multiple parts constituting the surface of the tooth to be identified corresponds to: the occlusal surface, distal surface, mesial surface, lingual surface, or buccal surface. If the tooth to be identified is the mandibular canine, lateral incisor, or central incisor, the calculation unit 102 estimates, using the estimation model 50, which of the multiple parts constituting the surface of the tooth to be identified corresponds to: the distal surface, mesial surface, lingual surface, or labial surface.

[0067] Furthermore, the calculation unit 102 identifies the type of tooth based on the three-dimensional data input from the input unit 101 and an estimation model 50 that has been machine-learned to identify the type of tooth from which the three-dimensional data is to be acquired.

[0068] The calculation unit 102 identifies the processed area on a tooth based on the three-dimensional data input from the input unit 101 and an estimation model 50 that has been machine-learned to identify the processed area on the tooth from which the three-dimensional data is to be acquired.

[0069] The calculation unit 102 identifies the lesion site in the tooth or gum based on the three-dimensional data input from the input unit 101 and an estimation model 50 that has been machine-learned to identify the lesion site in the tooth or gum targeted for acquisition of the three-dimensional data.

[0070] The calculation unit 102 estimates the depth of the periodontal pocket in a tooth based on the three-dimensional data input from the input unit 101 and an estimation model 50 that has been machine-learned to estimate the depth of the periodontal pocket in the tooth from which the three-dimensional data is to be acquired.

[0071] The calculation unit 102 estimates the premature contact position where the upper and lower teeth make premature contact when they occlude, based on the three-dimensional data input from the input unit 101.

[0072] The output unit 103 is a functional unit realized by at least one of the display interface 15, the storage medium interface 17, and the communication device 18. The output unit 103 outputs image data to the display 20 for displaying a figure or table showing the results of each data processing obtained by the calculation unit 102, such as the identification result of the tooth location, the identification result of the tooth type, the identification result of the processing site, the identification result of the lesion site, the estimation result of the periodontal pocket depth, and the estimation result of the early contact position. It also records the results of each data processing as an electronic medical record, and outputs the recorded electronic medical record to the removable disk 40 or an external device.

[0073] The data processing device 10, having the various functional units described above, can identify the tooth location using the calculation unit 102 based on the three-dimensional data input to the input unit 101, and output the identification result using the output unit 103. This allows the user to easily and accurately identify the tooth location using the data processing device 10.

[0074] Furthermore, the data processing device 10 can also calculate, based on the three-dimensional data input to the input unit 101, the results of tooth type identification, processing site identification, lesion site identification, periodontal pocket depth estimation, or premature contact location estimation using the calculation unit 102, and the output unit 103 can output these data processing results. As a result, the user does not need to identify the tooth type, processing site, lesion site, periodontal pocket depth, or premature contact location themselves, in addition to the tooth location, thus shortening the time required for dental examinations and improving the accuracy of dental examinations.

[0075] Furthermore, three-dimensional data may be input to the input unit 101 each time a tooth is scanned using the three-dimensional scanner 2. The calculation unit 102 may, each time three-dimensional data is input to the input unit 101, use the three-dimensional data to calculate the tooth location identification result, tooth type identification result, processed area identification result, lesion area identification result, periodontal pocket depth estimation result, or premature contact location estimation result. The output unit 103 may output the results of this data processing obtained during scanning by the three-dimensional scanner 2 in real time. This allows the user to obtain the results of data processing in real time each time a tooth is scanned using the three-dimensional scanner 2 during a dental checkup.

[0076] [An example of the identification result of a tooth location] Referring to Figures 7 to 10, the results of tooth location identification obtained by the data processing device 10 according to the embodiment will be explained. Figures 7 to 10 are diagrams showing examples of tooth location identification results obtained by the data processing device 10 according to the embodiment.

[0077] As shown in Figures 7 to 10, the data processing device 10 directly inputs the three-dimensional data of each of the multiple teeth included in the maxillary and mandibular dentition into the neural network 51 of the estimation model 50. The neural network 51 is machine-trained to identify each of the multiple teeth by the input of the three-dimensional data. In training the estimation model 50 using supervised learning, training data is used that includes three-dimensional data showing the surface shape of each tooth and colors pre-associated with multiple parts that constitute the tooth surface shown by the three-dimensional data. Such coloring is not necessarily required; any identification label that can identify the multiple parts that constitute the tooth surface shown by the three-dimensional data may be used. For example, numerical values ​​that can identify the multiple parts that constitute the tooth surface shown by the three-dimensional data may be associated with those multiple parts.

[0078] For example, as shown in Figure 9, in the case of the second molar on the left mandible, yellow data is associated with the occlusal surface, blue data with the distal surface, red data with the buccal surface, and green data with the mesial surface. On the other hand, although not shown in the illustration, in the case of the second molar on the right mandible, orange data is associated with the occlusal surface, dark blue data with the distal surface, pink data with the buccal surface, and yellow-green data with the mesial surface. In other words, in the training data, different colors are associated with each part of the 3D data corresponding to each part of each tooth, which is the input data, as the correct answer data.

[0079] Furthermore, for the first molar, second molar, third molar, first premolar, and second premolar, a predetermined color may be associated with the three-dimensional data corresponding to each of the multiple cusps that make up each tooth. For example, as shown in Figure 10, in the case of the second molar on the left side of the mandible, data showing red may be associated with the three-dimensional data corresponding to the distal buccal cusp, data showing purple may be associated with the three-dimensional data corresponding to the distal lingual cusp, data showing yellow may be associated with the three-dimensional data corresponding to the mesiobuccal cusp, and data showing blue may be associated with the three-dimensional data corresponding to the mesiobuccal cusp. On the other hand, although not shown in the figure, in the case of the second molar on the right side of the mandible, data showing orange may be associated with the three-dimensional data corresponding to the distal buccal cusp, data showing pink may be associated with the three-dimensional data corresponding to the distal lingual cusp, data showing yellow-green may be associated with the three-dimensional data corresponding to the mesiobuccal cusp, and data showing navy blue may be associated with the three-dimensional data corresponding to the mesiobuccal cusp. In other words, in the training data, the three-dimensional data corresponding to each cusp of each tooth, which is the input data, is associated with different colors for each part as the ground truth data.

[0080] During the training phase, the estimation model 50 identifies the surface shape features of an object indicated by the three-dimensional data input to the neural network 51, estimates which type of tooth and which part of the tooth the three-dimensional data corresponds to, and outputs the color associated with the estimated part as the identification result. The estimation model 50 determines whether the outputted identification result color matches the color of the ground truth data, and if they do not match, optimizes the parameters 52 so that they match. As a result, during the operational phase, the estimation model 50 can accurately estimate the color of the part of the tooth corresponding to the three-dimensional data based on the three-dimensional data input to the neural network 51, and output the estimated color as the identification result.

[0081] For example, as shown in Figure 7, when the data processing device 10 displays the surface shape of the upper and lower dental arches, which are the target of three-dimensional data acquisition, on the display 20 using a plurality of points (point cloud) that have three-dimensional data, it uses an estimation model 50 to estimate the color of each part based on the three-dimensional data, assigns the estimated color to the points corresponding to each part, and then displays the surface shape of the upper and lower dental arches indicated by the colored points on the display 20.

[0082] For example, as shown in Figure 8, when the data processing device 10 displays the surface shape of the upper and lower dental arches, which are the target of three-dimensional data acquisition, on the display 20 using a polygon mesh containing the three-dimensional data, it uses an estimation model 50 to estimate the color of each part based on the three-dimensional data, assigns the estimated color to the polygon mesh corresponding to each part, and then displays the surface shape of the upper and lower dental arches shown by the colored polygon mesh on the display 20.

[0083] Furthermore, the data processing device 10 is not limited to identifying the location of each tooth based on three-dimensional data of multiple teeth, such as a dental arch; it may also identify the location of a single tooth based on three-dimensional data of that single tooth.

[0084] For example, as shown in Figures 9 and 10, when the data processing device 10 displays the surface shape of a single tooth, which is the target of three-dimensional data acquisition, on the display 20, it uses the estimation model 50 to estimate the color of each part that makes up the surface of the single tooth based on the three-dimensional data, and displays an image of the single tooth on the display 20 with the estimated colors applied to each part.

[0085] In this way, the data processing device 10 can use the estimation model 50 to identify the tooth location based on three-dimensional data and display the identification result in color. This allows the user to easily and accurately identify the tooth location using the data processing device 10.

[0086] [Example of data processing] An example of data processing performed by the data processing device 10 will be explained with reference to Figures 11 to 13. Figures 11 to 13 show an example of data processing using the estimation model 50 by the data processing device 10 according to the embodiment.

[0087] As shown in Figure 11, the data processing device 10 may use a single estimation model 50 to estimate the type and location of teeth based on three-dimensional data. In this case, the neural network 51 of the estimation model 50 is trained by machine learning (e.g., supervised learning) to identify both the type and location of teeth by directly inputting three-dimensional data of objects in the oral cavity.

[0088] As shown in Figure 12, the data processing device 10 may include, as the estimation model 50, a first estimation model 50A which includes a first neural network 51A for identifying the type of tooth, and a second estimation model 50B which includes a second neural network 51B for identifying the part of the tooth.

[0089] Specifically, the data processing device 10 may identify the type of tooth using a first estimation model 50A based on three-dimensional data showing the surface shape of objects in the oral cavity. In this case, the first neural network 51A of the first estimation model 50A is trained by machine learning (e.g., supervised learning) to identify the type of tooth by directly inputting three-dimensional data of objects in the oral cavity.

[0090] For example, in the training phase, the first estimation model 50A identifies the surface shape features of an object indicated by the three-dimensional data input to the first neural network 51A, estimates the type of tooth corresponding to the three-dimensional data, and outputs the estimation result. The first estimation model 50A determines whether the outputted estimation result matches the correct data for the type of tooth, and if they do not match, it optimizes the parameter 52 so that they match. As a result, in the operational phase, the first estimation model 50A can accurately estimate the type of tooth corresponding to the three-dimensional data based on the three-dimensional data input to the first neural network 51A.

[0091] Furthermore, as a method for identifying the type of tooth using the first estimation model 50A based on three-dimensional data showing the surface shape of objects in the oral cavity, the data processing device 10 adopts the method disclosed in Japanese Patent Application Publication No. 2020-096691 (Japanese Patent No. 6650996). Therefore, this specification incorporates the information disclosed in Japanese Patent Application Publication No. 2020-096691 (Japanese Patent No. 6650996).

[0092] The data processing device 10 may use a second estimation model 50B to identify the location of a tooth based on three-dimensional data showing the surface shape of an oral cavity object to which the tooth type estimated by the first estimation model 50A corresponds. In this case, the second neural network 51B of the second estimation model 50B is trained by machine learning (e.g., supervised learning) to identify the location of a tooth by directly inputting three-dimensional data of an oral cavity object to which the tooth type corresponds.

[0093] For example, in the training phase, the second estimation model 50B identifies the surface shape features of an object represented by three-dimensional data to which the tooth type has been pre-associated, input to the second neural network 51B, estimates the tooth location corresponding to that three-dimensional data, and outputs the estimation result. The second estimation model 50B determines whether the outputted estimation result matches the tooth location data, which is the correct answer, and if they do not match, it optimizes the parameter 52 so that they match. As a result, in the operational phase, the second estimation model 50B can accurately estimate the tooth location corresponding to the three-dimensional data to which the tooth type has been associated, based on the three-dimensional data input to the second neural network 51B. Note that, unlike the estimation model 50 shown in Figure 11, the second estimation model 50B does not need to identify the tooth type because the tooth type has been pre-associated with the input three-dimensional data. For this reason, the second estimation model 50B has improved machine learning efficiency compared to the estimation model 50 shown in Figure 11, and can estimate the tooth location with higher accuracy.

[0094] Note that the first estimation model 50A and the second estimation model 50B shown in Figure 12 may be different estimation models, or, as shown in Figure 11, one estimation model 50 may have the functions of both the first estimation model 50A and the second estimation model 50B.

[0095] As shown in Figure 13, the data processing device 10 may include, as estimation models 50, a first estimation model 50A including a first neural network 51A for identifying the type of tooth, a second estimation model 50B including a second neural network 51B for identifying the area on the tooth, a third estimation model 50C including a third neural network 51C for identifying the processed area on the tooth, a fourth estimation model 50D including a fourth neural network 51D for identifying the lesion area on the tooth, and a fifth estimation model 50E including a fifth neural network 51E for estimating the depth of the periodontal pocket on the tooth.

[0096] Specifically, the data processing device 10 may identify the type of tooth using a first estimation model 50A based on three-dimensional data showing the surface shape of objects in the oral cavity. In this case, as explained in Figure 12, the first neural network 51A of the first estimation model 50A is trained by machine learning (e.g., supervised learning) to identify the type of tooth by directly inputting three-dimensional data of objects in the oral cavity.

[0097] The data processing device 10 may identify the location of a tooth using a second estimation model 50B based on three-dimensional data showing the surface shape of an object in the oral cavity. In this case, as explained in Figures 7 to 10, the second neural network 51B of the second estimation model 50B is trained by machine learning (e.g., supervised learning) to identify the location of a tooth by directly inputting three-dimensional data of an object in the oral cavity.

[0098] The data processing device 10 may identify the processed area on the tooth using a third estimation model 50C based on three-dimensional data showing the surface shape of an object in the oral cavity. In this case, the third neural network 51C of the third estimation model 50C is trained by machine learning (e.g., supervised learning) to identify the processed area on the tooth by directly inputting three-dimensional data of an object in the oral cavity.

[0099] For example, during the training phase, the third estimation model 50C identifies the surface shape features of an object indicated by the three-dimensional data input to the third neural network 51C, estimates the processing area corresponding to the three-dimensional data, and outputs the estimation result. The third estimation model 50C determines whether the outputted estimation result matches the processing area on the tooth, which is the ground truth data. If they do not match, it optimizes the parameter 52 so that they match. As a result, during the operational phase, the third estimation model 50C can accurately estimate the processing area on the tooth corresponding to the three-dimensional data input to the third neural network 51C.

[0100] Furthermore, as a method for identifying the processed area on a tooth using a third estimation model 50C based on three-dimensional data showing the surface shape of an object in the oral cavity, the data processing device 10 adopts the method disclosed in Japanese Patent Application Publication No. 2022-012198 (Japanese Patent No. 7267974). Therefore, this specification will utilize the information disclosed in Japanese Patent Application Publication No. 2022-012198 (Japanese Patent No. 7267974).

[0101] The data processing device 10 may identify lesion sites in teeth using a fourth estimation model 50D based on three-dimensional data showing the surface shape of objects in the oral cavity. In this case, the fourth neural network 51D of the fourth estimation model 50D is trained by machine learning (e.g., supervised learning) to identify lesion sites in teeth by directly inputting three-dimensional data of objects in the oral cavity.

[0102] For example, during the training phase, the fourth estimation model 50D identifies the surface shape features of an object indicated by the three-dimensional data input to the fourth neural network 51D, estimates the lesion site corresponding to the three-dimensional data, and outputs the estimation result. The fourth estimation model 50D determines whether the outputted estimation result matches the lesion in the tooth, which is the ground truth data. If they do not match, it optimizes the parameter 52 so that they match. As a result, during the operational phase, the fourth estimation model 50D can accurately estimate the lesion site in the tooth corresponding to the three-dimensional data input to the fourth neural network 51D.

[0103] Furthermore, as a method for identifying lesion sites in teeth using a fourth estimation model 50D based on three-dimensional data showing the surface shape of objects in the oral cavity, the data processing device 10 adopts the method disclosed in Japanese Patent Application Publication No. 2022-012199 (Japanese Patent No. 7324735). Therefore, this specification incorporates the information disclosed in Japanese Patent Application Publication No. 2022-012199 (Japanese Patent No. 7324735).

[0104] The data processing device 10 may estimate the depth of periodontal pockets in teeth using a fifth estimation model 50E based on three-dimensional data showing the surface shape of objects in the oral cavity. In this case, the fifth neural network 51E of the fifth estimation model 50E is trained by machine learning (e.g., supervised learning) to estimate the depth of periodontal pockets in teeth by directly inputting three-dimensional data of objects in the oral cavity.

[0105] For example, in the training phase, the fifth estimation model 50E identifies the surface shape features of an object indicated by the three-dimensional data input to the fifth neural network 51E, estimates the depth of the periodontal pocket corresponding to the three-dimensional data, and outputs the estimation result. Specifically, the fifth neural network 51E of the fifth estimation model 50E is input with composite image data obtained by combining IOS data and CT data acquired for the same subject, as well as predetermined measurement directions and predetermined measurement points for measuring the depth of the periodontal pocket. Based on this input data, the fifth estimation model 50E estimates the depth of the periodontal pocket. The fifth estimation model 50E determines whether the outputted estimation result matches the ground truth data for the depth of the periodontal pocket, and if they do not match, it optimizes the parameters 52 so that they match. As a result, in the operational phase, the fifth estimation model 50E can accurately estimate the depth of the periodontal pocket in the tooth corresponding to the three-dimensional data input to the fifth neural network 51E.

[0106] Furthermore, the data processing device 10 employs the method disclosed in Japanese Patent Application No. 2023-013834 as a method for estimating the depth of periodontal pockets in teeth using the fifth estimation model 50E based on three-dimensional data showing the surface shape of objects in the oral cavity. Therefore, this specification incorporates the matters disclosed in Japanese Patent Application No. 2023-013834.

[0107] The data processing device 10 may estimate the premature contact positions where the upper and lower teeth make premature contact when they occlude by performing premature contact position estimation processing based on three-dimensional data showing the surface shape of objects in the oral cavity.

[0108] For example, the data processing device 10 receives maxillary dentition data showing the surface shape of the subject's maxillary dentition acquired by the three-dimensional scanner 2, and mandibular dentition data showing the surface shape of the subject's mandibular dentition acquired by the three-dimensional scanner 2. The data processing device 10 calculates the early contact position by performing image processing so that the upper and lower dentitions are displaced from an occlusal state to an open state, using a rendering image showing the maxillary dentition generated based on the maxillary dentition data and a rendering image showing the mandibular dentition generated based on the mandibular dentition data.

[0109] Furthermore, as a method for estimating the premature contact position based on three-dimensional data showing the surface shape of objects in the oral cavity, the data processing device 10 employs the method disclosed in Japanese Patent Application No. 2022-182611. Therefore, this specification incorporates the information disclosed in Japanese Patent Application No. 2022-182611.

[0110] Thus, the data processing device 10 can estimate at least one of the following based on three-dimensional data showing the surface shape of objects in the oral cavity: the type of tooth and its location on the tooth, as well as the processed area on the tooth, the lesioned area on the tooth, the depth of the periodontal pocket on the tooth, and the premature contact position on the tooth. The first estimation model 50A for estimating the type of tooth, the second estimation model 50B for estimating the location on the tooth, the third estimation model 50C for estimating the processed area on the tooth, the fourth estimation model 50D for estimating the lesioned area on the tooth, and the fifth estimation model 50E for estimating the depth of the periodontal pocket all share the common feature of using three-dimensional data to calculate the estimation results. Therefore, the data processing device 10 can estimate the type of tooth, its location on the tooth, the processed area on the tooth, the lesioned area on the tooth, the depth of the periodontal pocket on the tooth, and the premature contact position on the tooth simply by using the common three-dimensional data for each estimation model 50. The data processing device 10 can effectively utilize the estimation results obtained by each estimation model 50, for example, by adding at least one piece of information from the processed area, lesion area, periodontal pocket depth, and early contact location to a figure or table showing the identification results of the tooth type and tooth location. In other words, the systems used in each of the above-mentioned identifications employ the same coordinate system for three-dimensional data, and the position coordinates included in each identification result are relatively correct. Therefore, the data processing device 10 can easily understand which tooth and which part of it has processing and lesions. When the data processing device 10 uses identification results obtained from systems that employ different coordinate systems, it is sufficient to perform a transformation process to make the position coordinates included in the identification results relatively correct. As such a transformation process, a general process of aligning or registering three-dimensional data can be used, and for example, methods such as ICP (Iterative Closest Point) may be employed.

[0111] For example, the data processing device 10 may identify the type and location of the tooth where the processed area is located, and display the results in a figure or table, or output them as an electronic medical record. The data processing device 10 may identify the type and location of the tooth where the lesion is located, and display the results in a figure or table, or output them as an electronic medical record. The data processing device 10 may estimate the depth of the periodontal pocket for each type and location of tooth, and display the results in a figure or table, or output them as an electronic medical record. The data processing device 10 may estimate the premature contact location for each type and location of tooth, and display the results in a figure or table, or output them as an electronic medical record. Furthermore, the data processing device 10 may combine multiple results selected from the processed area, lesion location, periodontal pocket depth, and premature contact location, associate them with the type and location of tooth, and display the results in a figure or table, or output them as an electronic medical record.

[0112] This eliminates the need for users to perform various examinations themselves, such as checking the treatment area, lesion site, periodontal pocket depth, or premature contact location, nor does it require them to identify the type and location of the tooth being examined. Consequently, users can avoid spending time on dental checkups or recording incorrect information in their medical records.

[0113] Note that the first estimation model 50A, second estimation model 50B, third estimation model 50C, fourth estimation model 50D, and fifth estimation model 50E shown in Figure 13 may be different estimation models from each other, or one estimation model 50 may have the functions of the first estimation model 50A, second estimation model 50B, third estimation model 50C, fourth estimation model 50D, and fifth estimation model 50E. Furthermore, the data processing device 10 may be configured to obtain multiple types of identification results by commonizing the multiple estimation models included in the first estimation model 50A, second estimation model 50B, third estimation model 50C, fourth estimation model 50D, and fifth estimation model 50E. For example, one estimation model that commonizes the first estimation model 50A and the second estimation model 50B may estimate the type of tooth and the location on the tooth.

[0114] [Variations of tooth location identification processing] Figure 14 shows a modified example of the tooth location identification process by the data processing device 10 according to the embodiment. As shown in Figure 14, the data processing device 10 may estimate the tooth type using the first estimation model 50A, and then estimate the tooth location according to the tooth surface extraction logic without using the second estimation model 50B as illustrated in Figure 12. An example of the identification process using the tooth surface extraction logic will be described below with reference to Figures 15 to 28.

[0115] Figures 15 to 28 show an example of identification processing using tooth surface extraction logic by the data processing device 10 according to the embodiment. First, with reference to Figures 15 to 21, an example will be described in which the data processing device 10 identifies the location of a tooth for the maxillary or mandibular canine, lateral incisor, and central incisor.

[0116] As shown in Figure 15, the data processing device 10 generates a rendering image of the dentition based on three-dimensional data showing the surface shape of the maxillary or mandibular dentition. Figure 15 shows a rendering image of the mandibular dentition. The data processing device 10 calculates the centroid of each tooth that makes up the dentition. In the rendering image, the data processing device 10 draws a curve (or polyline) that passes through the centroid of each tooth. The data processing device 10 calculates the centroid of the curve (i.e., the centroid of the dental arch). The data processing device 10 only needs to calculate the centroid of the tooth and the dental arch based on the coordinates of each three-dimensional data. The data processing device 10 also sets the tip of the dentition through which the curve passes (between the left central incisor and the right central incisor) as the apex of the curve.

[0117] As shown in Figure 16, the data processing device 10 extracts an image of one tooth from among the teeth included in the dentition shown in Figure 15. Figure 16 shows an image of a lateral incisor included in the mandibular dentition. The data processing device 10 calculates the coordinates of the bounding box surrounding the tooth and generates a bounding box to surround the tooth based on the calculated coordinates. The data processing device 10 calculates the centroid of the tooth enclosed by the bounding box. The data processing device 10 generates an axis of the tooth that passes through the centroid of the tooth and is aligned with the direction of the edges of the bounding box. For example, the data processing device 10 generates a first axis that passes through the centroid of the tooth and is aligned with the direction of the long side of the bounding box, and a second axis that passes through the centroid of the tooth and is aligned with the direction of the short side of the bounding box. The data processing device 10 sets the longest axis (for example, the first axis) of the multiple axes generated as the tooth axis.

[0118] As shown in Figure 17, the data processing device 10 sets the plane P to one of several planes passing through the centroid of the tooth, which is aligned with the direction of the curve of the dental arch (the curve shown in Figure 15).

[0119] As shown in Figure 18, the data processing device 10 generates a plane Q by rotating plane P by a predetermined angle D1 in the first rotational direction, with the tooth axis as the central axis. The angle D1 can be set by the user.

[0120] As shown in Figure 19, the data processing device 10 generates a plane R by rotating plane P by a predetermined angle D2 in a second rotation direction opposite to the first rotation direction, with the tooth axis as the central axis. Note that angle D2 can be set by the user and may be the same angle as angle D1 or a different angle.

[0121] As shown in Figure 20, the data processing device 10 can divide the tooth surface into four parts by dividing the tooth in directions along plane R and plane Q, respectively.

[0122] As shown in Figure 21, the data processing device 10 assigns one of the following surfaces to each of the four surface portions of the divided tooth (in this example, the lateral incisor): distal surface, mesial surface, labial surface, and lingual surface. For example, the data processing device 10 sets the surface that has a large contact area with the adjacent tooth and is close to the apex of the dental arch curve as the mesial surface. The data processing device 10 sets the surface that has a large contact area with the adjacent tooth and is farther from the apex of the dental arch curve as the distal surface. The data processing device 10 sets the surface farther from the centroid of the dental arch as the labial surface. The data processing device 10 sets the surface close to the centroid of the dental arch as the lingual surface.

[0123] Figures 15 to 21 above show examples of identifying locations on the surface of the mandibular lateral incisor, but the data processing device 10 can identify locations on each tooth in the same way as shown in Figures 15 to 21 for any of the mandibular canines and central incisors, or maxillary canines, lateral incisors, and central incisors.

[0124] Next, referring to Figures 22 to 28, we will explain an example in which the data processing device 10 identifies the location of the first premolar, second premolar, first molar, second molar, and third molar of the maxilla or mandible.

[0125] First, using the method shown in Figure 15, the data processing device 10 draws a curve passing through the centroid of each tooth in a rendering image of a dentition generated based on three-dimensional data showing the surface shape of the maxillary or mandibular dentition, and calculates the centroid of the curve. The data processing device 10 also defines the apex of the curve as the tip of the dentition through which the curve passes (between the left central incisor and the right central incisor).

[0126] As shown in Figure 22, the data processing device 10 extracts an image of one tooth from among the teeth included in the dentition shown in Figure 15. Figure 22 shows an image of the first molar included in the mandibular dentition. The data processing device 10 calculates the coordinates of the bounding box surrounding the tooth and generates a bounding box to surround the tooth based on the calculated coordinates. The data processing device 10 calculates the centroid of the tooth enclosed by the bounding box. The data processing device 10 generates an axis of the tooth that passes through the centroid of the tooth and is aligned with the direction of the edges of the bounding box. For example, the data processing device 10 generates a first axis that passes through the centroid of the tooth and is aligned with the direction of the long side of the bounding box, and a second axis that passes through the centroid of the tooth and is aligned with the direction of the short side of the bounding box. Of the multiple axes generated, the data processing device 10 sets the shortest axis (for example, the second axis) as the tooth axis.

[0127] As shown in Figure 23, the data processing device 10 applies a cylinder with a predetermined radius r to the tooth, with the tooth axis as the central axis. The data processing device 10 sets the tooth surface contained within the cylinder as the occlusal surface. The radius r can be set by the user. Furthermore, the data processing device 10 may apply an elliptical cylinder or a rectangular prism to the tooth, not just a cylinder.

[0128] As shown in Figure 24, the data processing device 10 sets the plane P' to one of several planes passing through the centroid of the tooth, which is aligned with the direction of the curve of the dental arch (the curve shown in Figure 15).

[0129] As shown in Figure 25, the data processing device 10 generates a plane Q' by rotating plane P' by a predetermined angle D1' in the first rotational direction, with the tooth axis as the central axis. The angle D1' can be set by the user.

[0130] As shown in Figure 26, the data processing device 10 generates a plane R' by rotating plane P' by a predetermined angle D2' in a second rotation direction opposite to the first rotation direction, with the tooth axis as the central axis. Note that angle D2' can be set by the user and may be the same angle as angle D1' or a different angle.

[0131] As shown in Figure 27, the data processing device 10 can divide the tooth surface into four parts by dividing the tooth in directions along plane R' and plane Q', respectively.

[0132] As shown in Figure 28, the data processing device 10 assigns one of the following surfaces to each of the five surface portions of the divided tooth (in this example, the first molar): the occlusal surface, distal surface, mesial surface, lingual surface, and buccal surface. For example, the data processing device 10 sets the surface determined by the cylinder shown in Figure 23 as the occlusal surface. The data processing device 10 sets the surface that has a large contact area with adjacent teeth and is close to the apex of the dental arch curve as the mesial surface. The data processing device 10 sets the surface that has a large contact area with adjacent teeth and is farther from the apex of the dental arch curve as the distal surface. The data processing device 10 sets the surface farther from the center of gravity of the dental arch as the buccal surface. The data processing device 10 sets the surface close to the center of gravity of the dental arch as the lingual surface.

[0133] Figures 22 to 28 above show examples of identifying locations on the surface of the mandibular first molar, but the data processing device 10 can identify locations on each tooth in any of the mandibular first molar, second molar, first premolar, and second premolar, as well as the maxillary first molar, second molar, third molar, first premolar, and second premolar, using the methods shown in Figures 22 to 28.

[0134] Thus, the data processing device 10 can identify each of the multiple regions based on the multiple regions divided by multiple planes along the tooth axis passing through the centroid of the tooth, and the centroid of the dental arch containing the tooth. Furthermore, the data processing device 10 can identify the occlusal surface of the tooth using at least one of a cylinder, an elliptical cylinder, and a rectangular prism with the tooth axis as its central axis. As a result, the data processing device 10 can estimate the region of the tooth according to the tooth surface extraction logic without using the estimation model 50, and therefore there is no need to train the estimation model 50 by machine learning.

[0135] [Data Processing] The data processing performed by the data processing device 10 according to the embodiment will be explained with reference to Figure 29. Figure 29 is a flowchart that defines the data processing performed by the data processing device 10 according to the embodiment. Each STEP shown in Figure 29 (hereinafter referred to as "S") is realized by the arithmetic unit 11 of the data processing device 10 executing the data processing program 100.

[0136] As shown in Figure 29, the data processing device 10 determines whether or not it has acquired three-dimensional data including three-dimensional positional information corresponding to each of a plurality of points indicating the surface shape of an object in the oral cavity (S1). If the data processing device 10 has not acquired three-dimensional data (NO in S1), it terminates this process.

[0137] On the other hand, if the data processing device 10 acquires three-dimensional data (YES in S1), it identifies the type of tooth based on the three-dimensional data (S2). For example, the data processing device 10 identifies the type of tooth from which the three-dimensional data was acquired by directly inputting the three-dimensional data acquired in S1 into the neural network 51 (or first neural network 51A) of the estimation model 50 (or first estimation model 50A).

[0138] The data processing device 10 identifies each of the multiple parts that make up the surface of the tooth based on the three-dimensional data (S3). For example, the data processing device 10 identifies each of the multiple parts that make up the surface of the tooth from which the three-dimensional data is to be acquired by directly inputting the three-dimensional data acquired in S1 into the neural network 51 (or second neural network 51B) of the estimation model 50 (or second estimation model 50B). Alternatively, the data processing device 10 identifies each of the multiple parts that make up the surface of the tooth from which the three-dimensional data is to be acquired by directly inputting the three-dimensional data with the tooth type identification result acquired in S2 into the neural network 51 (or second neural network 51B) of the estimation model 50 (or second estimation model 50B). Alternatively, the data processing device 10 identifies each of the multiple parts that make up the surface of the tooth from which the three-dimensional data is to be acquired by following the tooth surface extraction logic described in Figures 15 to 28.

[0139] The data processing device 10 identifies the processed area on the tooth based on the three-dimensional data (S4). For example, the data processing device 10 identifies the processed area on the tooth from which the three-dimensional data was acquired by directly inputting the three-dimensional data acquired in S1 into the neural network 51 (or third neural network 51C) of the estimation model 50 (or third estimation model 50C).

[0140] The data processing device 10 identifies the lesion site in the tooth based on the three-dimensional data (S5). For example, the data processing device 10 identifies the lesion site in the tooth from which the three-dimensional data was acquired by directly inputting the three-dimensional data acquired in S1 into the neural network 51 (or fourth neural network 51D) of the estimation model 50 (or fourth estimation model 50D).

[0141] The data processing device 10 estimates the depth of the periodontal pocket in the tooth based on the three-dimensional data (S6). For example, the data processing device 10 directly inputs the three-dimensional data acquired in S1 into the neural network 51 (or fifth neural network 51E) of the estimation model 50 (or fifth estimation model 50E) to estimate the depth of the periodontal pocket in the tooth from which the three-dimensional data was acquired.

[0142] The data processing device 10 estimates the premature contact position in the teeth based on the three-dimensional data (S7). For example, the data processing device 10 divides the three-dimensional data acquired in S1 into maxillary dentition data showing the surface shape of the maxillary dentition and mandibular dentition data showing the surface shape of the mandibular dentition, and estimates the premature contact position by performing image processing using the maxillary dentition data and mandibular dentition data so that the upper and lower dentitions are displaced from an occlusal state to an open state.

[0143] The data processing device 10 may perform only the process in S3, or it may perform the process in S3 plus at least one of the processes in S4 to S7. Alternatively, the data processing device 10 may perform the processes in S2 and S3, or it may perform the processes in S2 and S3 plus at least one of the processes in S4 to S7.

[0144] The data processing device 10 outputs the results of the data processing obtained through the processes in S2 to S7 to the display 20 or reflects them in the electronic medical record (S8). After that, the data processing device 10 terminates this process. The data processing device 10 may execute the processes in S2 to S8 in real time each time it acquires three-dimensional data obtained during scanning by the three-dimensional scanner 2 in S1, or it may execute the processes in S2 to S8 on the condition that a predetermined amount of three-dimensional data has been acquired in S1.

[0145] [An example of an electronic medical record] An example of an electronic medical record generated by the data processing device 10 will be explained with reference to Figures 30 to 35. Figures 30 to 35 are diagrams showing an example of an electronic medical record generated by the data processing device 10 according to the embodiment. The data processing device 10 reflects the processing results obtained by data processing, such as tooth type, tooth location, processed area, lesion location, periodontal pocket depth, or premature contact location, into the electronic medical record, and displays a diagram or table showing the contents of the electronic medical record on the display 20. Here, reflecting each processing result into the electronic medical record means that the data processing device 10 grasps and stores the correspondence between the items corresponding to each part of each tooth type in the electronic medical record format and each recognized part of each tooth type, and further grasps and stores the correspondence between each recognized part of each tooth type and at least one of the following based on the coordinate position of each part: presence or absence or type of processing, presence or absence, degree or type of lesion, periodontal pocket depth, and presence or degree of premature contact. As a result, the data processing device 10 can display on the display 20 a diagram or table representing various conditions in each part of each type of tooth, based on this stored information.

[0146] For example, as shown in Figure 30, the data processing device 10 aligns multiple teeth included in the dentition and displays them on the display 20, and reflects the processing results for each tooth, such as the processed area, lesion area, periodontal pocket depth, or premature contact location. In particular, the data processing device 10 reflects in the electronic medical record which of the multiple areas constituting the surface of the tooth has a processed area or lesion area, and displays an image showing the result of this reflection on the display 20.

[0147] As shown in Figure 31, the data processing device 10 may create a chart summarizing the processing results for the depth of the periodontal pocket for each tooth and display the created chart on the display 20. In particular, the data processing device 10 displays the processing results for the depth of the periodontal pocket for each of the multiple parts that make up the surface of the tooth in a chart on the display 20. The data processing device 10 also displays the presence or absence of plaque attached to the surface of each of the multiple parts that make up the surface of the tooth in a chart on the display 20. Furthermore, for each tooth, the data processing device 10 displays the degree of tooth mobility, which indicates tooth movement, in a chart on the display 20. The degree of tooth mobility can be measured based on the feel of the tooth by a dentist or other professional, and the results can be input to the data processing device 10.

[0148] As shown in Figure 32, the data processing device 10 may display at least one of the maxillary and mandibular dentitions in three dimensions on the display 20. Furthermore, the data processing device 10 may annotate teeth included in the three-dimensional dentition with tooth numbers indicating the type of tooth, annotate processed areas such as inlays (metal) and abutment teeth, annotate lesion areas such as missing teeth, and annotate contact positions with opposing teeth, such as early contact positions or final contact positions.

[0149] As shown in Figure 33, the data processing device 10 may enlarge a portion of the dentition and display some of the teeth in three dimensions on the display 20. Furthermore, the data processing device 10 may indicate lesions such as caries in the teeth displayed in three dimensions using annotations.

[0150] As shown in Figure 34, the data processing device 10 may indicate the positions of multiple identified cusps in the three-dimensional representation of a tooth, specifically for the first molar, second molar, third molar, first premolar, or second premolar.

[0151] As shown in Figure 35, the data processing device 10 may display on the display 20 a three-dimensional image showing the upper and lower jaw dentition and a two-dimensional image showing the dental formula side by side. Furthermore, if the user adds annotations to a predetermined position on a tooth displayed in three dimensions, the data processing device 10 may also add annotations to the same predetermined position on the tooth displayed in two dimensions. Additionally, if the user places a cursor over a predetermined position on a tooth displayed in three dimensions, the data processing device 10 may highlight that predetermined position on the tooth displayed in two dimensions.

[0152] For example, in the case of Figure 35, caries has occurred on the buccal surface of the mandibular second premolar. When the data processing device 10 receives input from the input device 30 to add the annotation "caries" to the buccal surface of the second premolar shown in the three-dimensional image, it also adds the annotation "caries" to the buccal surface of the second premolar shown in the two-dimensional image. Furthermore, when the data processing device 10 receives input from the input device 30 to place the cursor over the buccal surface of the second premolar shown in the three-dimensional image, it also places the cursor over the buccal surface of the second premolar shown in the two-dimensional image and displays the annotation "caries," thereby highlighting the buccal surface of the second premolar where caries has occurred.

[0153] Conversely, if the user adds an annotation to a predetermined position on a tooth displayed in two dimensions, the data processing device 10 may add an annotation to the same predetermined position on the tooth displayed in three dimensions. Furthermore, if the user places the cursor over a predetermined position on a tooth displayed in two dimensions, the data processing device 10 may highlight that predetermined position on the tooth displayed in three dimensions.

[0154] For example, when the data processing device 10 receives input from the input device 30 to add the annotation "caries" to the buccal surface of the second premolar shown in the two-dimensional image, it also adds the annotation "caries" to the buccal surface of the second premolar shown in the three-dimensional image. Furthermore, when the data processing device 10 receives input from the input device 30 to place the cursor over the buccal surface of the second premolar shown in the two-dimensional image, it also places the cursor over the buccal surface of the second premolar shown in the three-dimensional image, enlarges the buccal surface of the second premolar, and displays the annotation "caries," thereby highlighting the buccal surface of the second premolar where caries has occurred. In the example in Figure 35, the data processing device 10 displays an enlarged image of the buccal surface of the second premolar on the display 20, as viewed from the direction of the buccal surface of the second premolar.

[0155] Note that the highlighting in the two-dimensional and three-dimensional images shown in Figure 35 is just one example, and the data processing device 10 may highlight the area where the cursor is placed in other ways. For example, when the data processing device 10 receives input from the input device 30 indicating that the cursor is placed on the buccal surface of the second premolar shown in the two-dimensional image, it may display an image of the buccal surface of the second premolar shown in the three-dimensional image in a predetermined display area. Furthermore, when the cursor is placed on a specific tooth area in one of the two-dimensional or three-dimensional images, the data processing device 10 may highlight the specific tooth area in the other image in various ways, such as changing the color of the specific tooth area, making the specific tooth area blink, enlarging the specific tooth area, or shrinking the specific tooth area.

[0156] Furthermore, the user may scan their teeth with the 3D scanner 2 while irradiating them with visible light, utilizing the QLF (Quantitative Light-induced Fluorescence) method which takes advantage of the autofluorescence of teeth. The user may also scan teeth that have been discolored with a staining agent using the 3D scanner 2. In addition, the user may scan teeth that have been discolored with a sheet of articulating paper on which a red or blue paint has been transferred to the surface using the 3D scanner 2.

[0157] As described above, the data processing device 10 may determine at least one of the presence or absence of abnormalities (e.g., plaque) and the degree of abnormality in a colored area of ​​a tooth based on three-dimensional data of the tooth colored with visible light such as blue or a dye. For example, the estimation model 50 of the data processing device 10 is trained by machine learning to determine at least one of the presence or absence of abnormalities and the degree of abnormality in a colored area of ​​a tooth based on three-dimensional data of the colored tooth. The data processing device 10 determines at least one of the presence or absence of abnormalities and the degree of abnormality in a colored area of ​​a tooth by directly inputting the three-dimensional data of the colored tooth into the neural network 51 of the estimation model 50. Furthermore, the data processing device 10 may display on the display 20, associating an image showing the colored area of ​​the tooth (for example, the enlarged image shown in Figure 35) with an image showing the determination result of at least one of the presence or absence of abnormalities and the degree of abnormality (for example, the "caries" image shown in Figure 35). Furthermore, if the data processing device 10 determines the presence or absence of plaque as an abnormality in a tooth area, it may display an image indicating the presence or absence of plaque on the display 20, for example, as shown in Figure 31. The determination of the presence or absence of an abnormality includes any determination of the presence or absence of an abnormality that can be performed based on three-dimensional data of the tooth colored with visible light such as blue or a staining agent, such as determining whether or not a lesion such as caries has occurred, whether or not tartar has been attached, and whether or not plaque has been attached. The determination of the degree of abnormality also includes any determination of the degree of abnormality that can be performed based on three-dimensional data of the tooth colored with visible light such as blue or a staining agent, such as determining the degree of lesions such as CO and C1 indicating the degree of caries, and determining the degree of tartar attachment. The presence or absence and degree of abnormality in a tooth area can also be determined by the fourth estimation model 50D which identifies the lesion area. For example, when the data processing device 10 determines the presence or absence of plaque, similar to caries, the fourth neural network 51D included in the fourth estimation model 50D can determine the presence or absence of plaque mainly based on the morphological characteristics of the tooth area where plaque has formed.In this process, the fourth neural network 51D can determine the presence or absence of plaque with greater accuracy by including features such as the actual color or lighting conditions of the tooth area in the input data. The data processing device 10 can determine the presence or absence of plaque without coloring the teeth with visible light such as blue or a dye, by using the fourth estimation model 50D which includes the fourth neural network 51D. However, if the data processing device 10 uses three-dimensional data of teeth colored with visible light such as blue or a dye, it can determine the presence or absence of plaque with greater accuracy. Thus, by using three-dimensional data of teeth colored with visible light such as blue or a dye as input data, the data processing device 10 can determine the presence or absence or degree of abnormality of the tooth area with greater accuracy. The estimation model 50 considers not only the colored tooth but also the shape of the abnormal area, the actual color, and the lighting conditions to determine the presence or degree of abnormality, thus increasing the accuracy of the determination.

[0158] As described above, the three-dimensional data includes color information indicating the actual color of each point in the point cloud representing the surface shape of an object in the oral cavity (the surface portion of the object). Therefore, the data processing device 10 may determine the actual color of each of multiple parts based on the three-dimensional data. For example, the estimation model 50 of the data processing device 10 is trained by machine learning to determine the actual color of a tooth based on three-dimensional data of the tooth including color information. The data processing device 10 determines the actual color of a tooth by directly inputting the three-dimensional data of the tooth including color information into the neural network 51 of the estimation model 50. Conventionally, when a dentist or other practitioner replaces a patient's missing tooth with a ceramic prosthesis, they visually determine the actual color of the tooth using a predetermined color sample, but the data processing device 10 can perform such a determination using the estimation model 50. Furthermore, the data processing device 10 may display on the display 20 images showing the parts of each tooth and images showing the determination result of the actual color of each tooth in association. Furthermore, the data processing device 10 may estimate, based on the three-dimensional data, which color on the color chart corresponds to the actual color of each of the multiple parts. This allows, for example, a dentist to appropriately select a ceramic prosthesis that matches the color of the patient's teeth when replacing a patient's missing teeth.

[0159] As described above, the data processing device 10 can use the estimated model 50 to identify the type of tooth included in the dentition and the surface area of ​​each tooth. Therefore, the data processing device 10 may generate a triangle in the mandibular dentition by connecting the distal buccal cusp of the left second molar, the distal buccal cusp of the right second molar, and the midpoints (incisor points) of the proximal surfaces of the left and right central incisors, and consider this triangle as the occlusal plane, outputting image data for displaying the occlusal plane to the display 20. In this way, the data processing device 10 can display on the display 20 a virtual occlusal plane generated by the positions of the left first molar cusp, the right first molar cusp, and the central incisors, respectively, which were identified using the estimated model 50.

[0160] As described above, the data processing device 10 can automatically recognize the type of tooth, the location of the tooth, the processing area, the lesion site, the depth of the periodontal pocket, and the location of early contact simply by acquiring three-dimensional data of the tooth obtained by a three-dimensional scanner 2 or a CT imaging device, and reflect this information in the electronic medical record. This improves the convenience for users such as practitioners and also increases the accuracy of dental examinations.

[0161] The embodiments disclosed herein should be considered in all respects to be illustrative and not restrictive. The scope of this disclosure is indicated by the claims rather than the foregoing description, and all modifications within the meaning and scope of the claims are intended to be included. The configurations illustrated in these embodiments and those illustrated in the variations may be combined as appropriate. [Explanation of Symbols]

[0162] 1 Data processing system, 2 Three-dimensional scanner, 10 Data processing device, 11 Arithmetic unit, 12 Memory, 13 Storage device, 14 Scanner interface, 15 Display interface, 16 Input device interface, 17 Storage medium interface, 18 Communication device, 20 Display, 30 Input device, 31 Keyboard, 32 Mouse, 40 Removable disk, 50 Estimation model, 50A First estimation model, 50B Second estimation model, 50C Third estimation model, 50D Fourth estimation model, 50E Fifth estimation model, 51 Neural network, 51A First neural network, 51B Second neural network, 51C Third neural network, 51D Fourth neural network, 51E Fifth neural network, 52 Parameters, 100 Data processing program, 101 Input unit, 102 Arithmetic unit, 103 Output unit.

Claims

1. A data processing device for processing three-dimensional data, An input unit receives the three-dimensional data, which includes three-dimensional positional information corresponding to each of a plurality of points indicating the surface shape of an object in the oral cavity, including a specific tooth. A calculation unit that performs processing to identify each of the multiple parts divided on the surface of the specific tooth based on the three-dimensional data input from the input unit, A data processing device comprising an output unit that outputs the processing results from the calculation unit.

2. If the specific tooth is the maxillary first molar, second molar, third molar, first premolar, or second premolar, the plurality of regions include at least one of the occlusal surface, distal surface, mesial surface, palatal surface, and buccal surface of the specific tooth. If the specific tooth is the maxillary canine, lateral incisor, or central incisor, the plurality of parts include at least one of the distal surface, mesial surface, palatal surface, and labial surface of the specific tooth. If the specific tooth is the mandibular first molar, second molar, third molar, first premolar, or second premolar, the plurality of regions include at least one of the occlusal surface, distal surface, mesial surface, lingual surface, and buccal surface of the specific tooth. The data processing device according to claim 1, wherein, if the specific tooth is the mandibular canine, lateral incisor, or central incisor, the plurality of parts include at least one of the distal surface, mesial surface, lingual surface, and labial surface of the specific tooth.

3. The data processing device according to claim 1, wherein, when the specific tooth is a first molar, a second molar, a third molar, a first premolar, or a second premolar, the plurality of parts include each of the plurality of cusps constituting the specific tooth.

4. The data processing device according to claim 1, wherein, if the specific tooth is a canine, lateral incisor, or central incisor, the plurality of portions include a plurality of surfaces constituting the labial surface of the specific tooth.

5. The data processing device according to claim 1, wherein the calculation unit identifies whether the specific tooth is an upper jaw tooth or a lower jaw tooth.

6. The data processing device according to any one of claims 1 to 5, wherein the calculation unit identifies the specific type of tooth based on the three-dimensional data input from the input unit and a first estimation model on which machine learning has been performed.

7. The data processing device according to claim 6, wherein the first estimation model includes a first neural network that has been trained to identify the specific type of tooth upon input of the three-dimensional data.

8. The calculation unit identifies each of the plurality of parts based on the three-dimensional data input from the input unit and a second estimation model on which machine learning has been performed, as a data processing device according to any one of claims 1 to 5.

9. The data processing device according to claim 8, wherein the second estimation model includes a second neural network that has been trained to identify each of the plurality of parts when the three-dimensional data is input.

10. The calculation unit identifies the processing area in the specific tooth based on the three-dimensional data input from the input unit and a third estimation model on which machine learning has been performed, as a data processing device according to any one of claims 1 to 5.

11. The data processing apparatus according to claim 10, wherein the third estimation model includes a third neural network that has been trained to identify the processing area when the three-dimensional data is input.

12. The data processing device according to any one of claims 1 to 5, wherein the calculation unit identifies a lesion site in a specific tooth or gum based on the three-dimensional data input from the input unit and a fourth estimation model on which machine learning has been performed.

13. The data processing device according to claim 12, wherein the fourth estimation model includes a fourth neural network that has been trained to identify the lesion site by inputting the three-dimensional data.

14. The data processing device according to any one of claims 1 to 5, wherein the calculation unit estimates the depth of the periodontal pocket in the specific tooth based on the three-dimensional data input from the input unit and a fifth estimation model on which machine learning has been performed.

15. The data processing device according to claim 14, wherein the fifth estimation model includes a fifth neural network that has been trained to estimate the depth of the periodontal pocket by inputting the three-dimensional data.

16. The data processing device according to any one of claims 1 to 5, wherein the calculation unit estimates the premature contact position where the upper and lower dental arches make premature contact when they occlude, based on the three-dimensional data input from the input unit.

17. The calculation unit identifies each of the plurality of parts based on the plurality of parts divided by a plurality of planes along the tooth axis passing through the centroid of the particular tooth and the centroid of the dental arch including the particular tooth, as described in any one of claims 1 to 5.

18. The data processing device according to claim 17, wherein the calculation unit identifies the occlusal surface of the specific tooth using at least one of a cylinder, an elliptical cylinder, and a rectangular prism with the tooth axis as its central axis.

19. The data processing apparatus according to any one of claims 1 to 5, wherein the output unit outputs image data for displaying a figure or table showing the processing results.

20. The processing result is used in an electronic medical record, as described in claim 19.

21. The data processing apparatus according to claim 19, wherein the image data includes data for displaying the specific tooth in at least one of two and three dimensions, together with the processing result.

22. The image data includes data for displaying the specific tooth in two and three dimensions. The data processing device according to claim 21, wherein if an annotation is added by the user to a predetermined position on a specific tooth displayed in three dimensions, the annotation is added to the predetermined position on the specific tooth displayed in two dimensions.

23. The image data includes data for displaying the specific tooth in two and three dimensions. The data processing device according to claim 21, wherein if an annotation is added by a user to a predetermined position on a specific tooth displayed in two dimensions, the annotation is added to the predetermined position on the specific tooth displayed in three dimensions.

24. The image data includes data for displaying the specific tooth in two and three dimensions. The data processing device according to claim 21, wherein when a user places a cursor over a predetermined position on a specific tooth displayed in three dimensions, the predetermined position on the specific tooth displayed in two dimensions is highlighted.

25. The image data includes data for displaying the specific tooth in two and three dimensions. The data processing device according to claim 21, wherein when a user places a cursor over a predetermined position on a specific tooth displayed in two dimensions, the predetermined position on the specific tooth displayed in three dimensions is highlighted.

26. The data processing apparatus according to any one of claims 1 to 5, wherein the processing result includes color information associated with each of the plurality of parts.

27. The aforementioned arithmetic unit, Based on the three-dimensional data input from the input unit, at least one of the presence or absence of an abnormality and the degree of abnormality of the colored specific tooth area is determined. A data processing device according to any one of claims 1 to 5, which associates the colored portion of the specific tooth with at least one determination result of the presence or absence of an abnormality and the degree of abnormality, based on the processing results.

28. The aforementioned arithmetic unit, Based on the three-dimensional data input from the input unit, the actual color or the color in the color sample for each of the multiple parts is determined. A data processing device according to any one of claims 1 to 5, which associates each of the plurality of parts with the actual color or the color determination result in the color sample based on the processing result.

29. The results of the above processing include the distal buccal cusp of the second molar on the left mandibular side, the distal buccal cusp of the second molar on the right mandibular side, and the position of the proximal surfaces of the central incisors on both the left and right mandibular sides. The data processing device according to any one of claims 1 to 5, wherein the output unit outputs image data for displaying an occlusal plane generated based on the distal buccal cusp of the second molar on the left side of the mandible, the distal buccal cusp of the second molar on the right side of the mandible, and the midpoint of the proximity surfaces of the central incisors on the left and right sides of the mandible.

30. The three-dimensional data is three-dimensional scanner data obtained by scanning the specific tooth using a three-dimensional scanner. The data processing apparatus according to any one of claims 1 to 5, wherein the output unit outputs the processing results obtained during scanning by the three-dimensional scanner in real time.

31. The data processing device according to any one of claims 1 to 5, wherein the three-dimensional data includes at least one of three-dimensional scanner data obtained by scanning the specific tooth using a three-dimensional scanner, CBCT data obtained by imaging the specific tooth using a cone-beam CT, and OCT data obtained by imaging the specific tooth using an optical coherence tomography.

32. A data processing method that processes three-dimensional data using a computer, The process to be performed by the aforementioned computer is as follows: Three-dimensional positional information corresponding to each of multiple points that indicate the surface shape of an object in the oral cavity, including a specific tooth. The steps include acquiring the three-dimensional data including the above, A step of performing a process to identify each of the multiple parts divided on the surface of the specific tooth based on the three-dimensional data obtained by the aforementioned acquisition step, A data processing method comprising the step of outputting the processing result obtained by the step of performing the aforementioned processing.

33. A data processing program for processing three-dimensional data using a computer, To the aforementioned computer, The steps include acquiring three-dimensional data which includes three-dimensional positional information corresponding to each of a plurality of points that indicate the surface shape of an object in the oral cavity, including a specific tooth, A step of performing a process to identify each of the multiple parts divided on the surface of the specific tooth based on the three-dimensional data obtained by the aforementioned acquisition step, A data processing program that performs the steps of: executing the aforementioned process and outputting the processing result obtained from that process.

Images