Data processing device, method for data processing, and data processing program

Abstract

To provide a technique capable of easily and accurately identifying a tooth position.SOLUTION: The data processing device 10 includes: an input unit 101 configured to receive three-dimensional data including three-dimensional positional information corresponding to each of a plurality of points representing a surface shape of an object in an oral cavity including teeth; a computation unit 102 configured to execute processing for identifying each of a plurality of portions of a tooth on the basis of the three-dimensional data input from the input unit 101; and an output unit 103 configured to output a processing result by the computation unit 102.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present disclosure relates to a data processing device, a data processing method, and a data processing program for processing three-dimensional data. [Background technology]

[0002] It is known that maintaining healthy teeth helps extend healthy life expectancy. Therefore, the introduction of a so-called universal dental checkup, which would require all citizens to undergo annual dental checkups, is being considered. During a dental checkup, a dentist or other practitioner examines the health of a patient's teeth and gums, and records the examination results in a patient's medical record. During this examination, the practitioner must examine the health of the teeth while identifying the type and location of the tooth (surface area), and record the examination results for each type and location in the patient's medical record.

[0003] In this regard, Japanese Patent Application Laid-Open No. 2020-96691 discloses a data processing device that identifies the type of tooth based on three-dimensional data including three-dimensional position information corresponding to each of multiple points that make up the tooth. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2020-96691 Summary of the Invention [Problem to be solved by the invention]

[0005] According to the data processing device disclosed in Patent Document 1, the practitioner can identify the type of tooth with high accuracy, but when conducting a dental examination, it is necessary to identify not only the type of tooth to be examined but also the part of the tooth to be examined. However, currently, the practitioner must identify the part of the tooth himself, and depending on the practitioner's individual skill and ability, the dental examination may take a long time, or the practitioner may mistakenly identify the part of the tooth where caries or the like has occurred, resulting in recording incorrect information in the patient's chart.

[0006] The present disclosure has been made to solve such problems, and aims to provide a technology that can easily identify tooth locations with high accuracy. [Means for solving the problem]

[0007] According to one example of the present disclosure, there is provided a data processing device for processing three-dimensional data, the data processing device including: an input unit to which three-dimensional data including three-dimensional position information corresponding to each of a plurality of points indicating the surface shape of an object in an oral cavity including a tooth is input; a calculation unit that executes processing to identify each of a plurality of parts of the tooth based on the three-dimensional data input from the input unit; and an output unit that outputs a processing result by the calculation unit.

[0008] According to one example of the present disclosure, there is provided a data processing method for processing three-dimensional data by a computer, the data processing method including, as processing executed by the computer, a step of acquiring three-dimensional data including three-dimensional position information corresponding to each of a plurality of points indicating the surface shape of an object in the oral cavity including a tooth, a step of executing processing for identifying each of a plurality of parts of the tooth based on the three-dimensional data acquired by the step of acquiring, and a step of outputting a processing result obtained by the step of executing processing.

[0009] According to one example of the present disclosure, there is provided a data processing program for processing three-dimensional data by a computer, the data processing program causing the computer to execute the steps of acquiring three-dimensional data including three-dimensional position information corresponding to each of a plurality of points indicating a surface shape of an object in an oral cavity including a tooth, performing processing for identifying each of a plurality of parts of the tooth based on the three-dimensional data acquired by the acquiring step, and outputting a processing result obtained by the processing step. [Effects of the Invention]

[0010] According to the present disclosure, it is possible to perform processing to identify each of multiple parts of a tooth based on three-dimensional data including three-dimensional position information corresponding to each of multiple points indicating the surface shape of an object in the oral cavity, including a tooth, and output the processing results, thereby making it possible to easily and accurately identify the parts of a tooth. [Brief explanation of the drawings]

[0011] [Figure 1] 1 is a diagram illustrating an application example of a data processing device according to an embodiment. [Figure 2] 1 is a block diagram showing a hardware configuration of a data processing device according to an embodiment; [Figure 3] FIG. 2 is a diagram for explaining an example of a part of a tooth. [Figure 4] 10A and 10B are diagrams for explaining modified examples of the site of each tooth, ie, the first molar, the second molar, the third molar, the first premolar, or the second premolar. [Figure 5] 10A and 10B are diagrams for explaining modified examples of portions of each tooth, such as a canine, a lateral incisor, or a central incisor. [Figure 6] 1 is a block diagram showing a functional configuration of a data processing device according to an embodiment; [Figure 7] 10A and 10B are diagrams showing an example of a result of identifying a part of a tooth obtained by the data processing device according to the embodiment. [Figure 8]10A and 10B are diagrams showing an example of a result of identifying a part of a tooth obtained by the data processing device according to the embodiment. [Figure 9] 10A and 10B are diagrams showing an example of a result of identifying a part of a tooth obtained by the data processing device according to the embodiment. [Figure 10] 10A and 10B are diagrams showing an example of a result of identifying a part of a tooth obtained by the data processing device according to the embodiment. [Figure 11] FIG. 10 is a diagram illustrating an example of data processing using an estimation model by the data processing device according to the embodiment. [Figure 12] FIG. 10 is a diagram illustrating an example of data processing using an estimation model by the data processing device according to the embodiment. [Figure 13] FIG. 10 is a diagram illustrating an example of data processing using an estimation model by the data processing device according to the embodiment. [Figure 14] 10A and 10B are diagrams showing a modified example of the process of identifying tooth regions by the data processing device according to the embodiment. [Figure 15] 10A and 10B are diagrams showing an example of an identification process using tooth surface extraction logic by a data processing device according to an embodiment. [Figure 16] 10A and 10B are diagrams showing an example of an identification process using tooth surface extraction logic by a data processing device according to an embodiment. [Figure 17] 10A and 10B are diagrams showing an example of an identification process using tooth surface extraction logic by a data processing device according to an embodiment. [Figure 18] 10A and 10B are diagrams showing an example of an identification process using tooth surface extraction logic by a data processing device according to an embodiment. [Figure 19] 10A and 10B are diagrams showing an example of an identification process using tooth surface extraction logic by a data processing device according to an embodiment. [Figure 20] 10A and 10B are diagrams showing an example of an identification process using tooth surface extraction logic by a data processing device according to an embodiment. [Figure 21] 10A and 10B are diagrams showing an example of an identification process using tooth surface extraction logic by a data processing device according to an embodiment. [Figure 22]10A and 10B are diagrams showing an example of an identification process using tooth surface extraction logic by a data processing device according to an embodiment. [Figure 23] 10A and 10B are diagrams showing an example of an identification process using tooth surface extraction logic by a data processing device according to an embodiment. [Figure 24] 10A and 10B are diagrams showing an example of an identification process using tooth surface extraction logic by a data processing device according to an embodiment. [Figure 25] 10A and 10B are diagrams showing an example of an identification process using tooth surface extraction logic by a data processing device according to an embodiment. [Figure 26] 10A and 10B are diagrams showing an example of an identification process using tooth surface extraction logic by a data processing device according to an embodiment. [Figure 27] 10A and 10B are diagrams showing an example of an identification process using tooth surface extraction logic by a data processing device according to an embodiment. [Figure 28] 10A and 10B are diagrams showing an example of an identification process using tooth surface extraction logic by a data processing device according to an embodiment. [Figure 29] 1 is a flowchart defining data processing executed by a data processing device according to an embodiment. [Figure 30] FIG. 2 is a diagram showing an example of an electronic medical record generated by a data processing device according to an embodiment. [Figure 31] FIG. 2 is a diagram showing an example of an electronic medical record generated by a data processing device according to an embodiment. [Figure 32] FIG. 2 is a diagram showing an example of an electronic medical record generated by a data processing device according to an embodiment. [Figure 33] FIG. 2 is a diagram showing an example of an electronic medical record generated by a data processing device according to an embodiment. [Figure 34] FIG. 2 is a diagram showing an example of an electronic medical record generated by a data processing device according to an embodiment. [Figure 35] FIG. 2 is a diagram showing an example of an electronic medical record generated by a data processing device according to an embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0012] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The present disclosure will be described in detail with reference to the accompanying drawings. In the drawings, the same or corresponding parts are designated by the same reference numerals and the description thereof will not be repeated.

[0013] [Application example] An application example of a data processing device 10 according to an embodiment will be described with reference to Fig. 1. Fig. 1 is a diagram showing an application example of a data processing device 10 according to an embodiment.

[0014] A user can obtain three-dimensional data of intraoral objects by scanning the oral cavity of a subject using a three-dimensional scanner 2 (e.g., an optical scanner). "Intraoral objects" include biological tissues such as teeth and gums, as well as processed areas such as implants, cavity-prepared teeth, abutment teeth, and prostheses. The three-dimensional data includes three-dimensional position information corresponding to each of multiple points (point cloud) that represent the surface shape of the intraoral object. Specifically, the three-dimensional data includes coordinates (X, Y, Z) of each point of the point cloud that represents the surface shape of the intraoral object in predetermined horizontal (X-axis), vertical (Y-axis), and height (Z-axis) directions. Furthermore, the three-dimensional data includes color information that represents the actual color of the portion (surface portion of the object) corresponding to each point of the point cloud that represents the surface shape of the intraoral object.

[0015] The user may also use a CT (Computed Tomography) imaging device (not shown) to image the subject's oral cavity, instead of the three-dimensional scanner 2. CT imaging devices include a CBCT (Cone Beam Coherence Tomography) device that performs computed tomography imaging of the subject's upper and lower jaws using a cone-shaped cone beam (X-ray beam). By using the CT imaging device to image the subject's upper and lower jaws, the user can obtain three-dimensional volume (voxel) data of hard tissues (e.g., bones, teeth) around the subject's upper and lower jaws, but not soft tissues (e.g., skin, gums). The user may also use an OCT (Optical Coherence Tomography) device that performs computed tomography imaging of the subject's upper and lower jaws using an optical coherence tomography (OCT) device. The user can generate cross-sectional images or external images of the subject by performing tomographic processing on the volume data of the subject obtained by the CT imaging device or OCT device. Furthermore, by performing processing such as converting voxels into dots on tooth volume (voxel) data obtained by a CT imaging device or OCT device, the user can generate three-dimensional data containing three-dimensional position information corresponding to each of multiple points (point cloud) that indicate the surface shape of the tooth.

[0016] As described above, the "three-dimensional data" includes at least one of three-dimensional scanner data (IOS (Intra Oral Scanner) data) obtained by scanning teeth using a three-dimensional scanner 2, CBCT data obtained by imaging teeth using a cone beam CT, and OCT data obtained by imaging teeth using an optical coherence tomography.

[0017] The term "user" includes practitioners (such as doctors) or assistants (such as dental assistants, dental hygienists, dental technicians, nurses, etc.) in various fields, such as dentistry, oral surgery, orthopedics, plastic surgery, and cosmetic surgery. The term "subject" also includes patients of dentistry, oral surgery, orthopedics, plastic surgery, and cosmetic surgery. The three-dimensional scanner 2 is a so-called intraoral scanner (IOS) that can optically capture images of the inside of a patient's oral cavity using a confocal method or triangulation method, and can obtain positional information for each point of a point cloud that indicates the surface shape of an object to be scanned (for example, teeth and gums in the oral cavity) placed in a certain coordinate space.

[0018] The data processing device 10 generates a rendering image (appearance image) showing the surface shape of an object in the oral cavity in three dimensions based on the three-dimensional data acquired by the three-dimensional scanner 2. A "rendering image" is an image generated by processing or editing certain data. For example, a user can generate a rendering image showing a two-dimensional object (a portion of an object that can be shown by IOS data) viewed from a specific viewpoint by processing or editing the three-dimensional data of the object acquired by the three-dimensional scanner 2. Furthermore, a user can generate multiple rendering images showing a two-dimensional object viewed from multiple directions by changing the specific viewpoint in multiple directions.

[0019] The data processing device 10 displays a rendering image generated based on the three-dimensional data acquired by the three-dimensional scanner 2 on the display 20. While viewing the rendering image displayed on the display 20, the user can gradually acquire three-dimensional data of the object in the oral cavity.

[0020] In a dental examination, a user such as a dentist or dental hygienist examines the health of a patient's teeth and gums and records the examination results in a medical chart. The user must examine the health of the teeth while identifying the type of tooth and the location (surface location) of the tooth, and record the examination results for each tooth type and location in the medical chart. Therefore, the user must perform the dental examination while identifying the type and location of the tooth being examined. However, if the user identifies the type and location of the tooth themselves, depending on the user's personal skill and ability, the dental examination may take a long time, or the user may mistakenly identify a location of the tooth where caries or other problems have developed, resulting in incorrect information being recorded in the medical chart.

[0021] Therefore, the data processing device 10 according to the embodiment uses AI (Artificial Intelligence) technology to identify the type and location of the tooth to be scanned based on three-dimensional data indicating the surface shape of objects in the oral cavity, including the teeth.

[0022] "Tooth types" include central incisors, lateral incisors, canines, first premolars, second premolars, first molars, second molars, and third molars on the right side of the maxilla. "Tooth types" include central incisors, lateral incisors, canines, first premolars, second premolars, first molars, second molars, and third molars on the left side of the maxilla. "Tooth types" include central incisors, lateral incisors, canines, first premolars, second premolars, first molars, second molars, and third molars on the right side of the mandible. "Tooth types" include central incisors, lateral incisors, canines, first premolars, second premolars, first molars, second molars, and third molars on the left side of the mandible.

[0023] The "part of a tooth" includes each part when the surface of each tooth described above is divided into a plurality of parts. For example, in the case of a second molar on the left side of the mandible, the "part of a tooth" includes each part when the surface of the second molar on the left side of the mandible is divided into a plurality of parts (occlusal surface, distal surface, mesial surface, lingual surface, and buccal surface), as will be described in detail later.

[0024] The data processing device 10 generates image data for displaying a diagram or table showing the identification results of tooth type and tooth location. The image data includes data for displaying the teeth in at least one of two and three dimensions along with the identification results. The data processing device 10 outputs the image data to the display 20, thereby enabling the display 20 to display the diagram or table showing the identification results.

[0025] Furthermore, the data processing device 10 may identify a processed portion of a tooth from among the scanned intraoral objects based on the three-dimensional data. The "processed portion" includes an implant, a cavity-prepared tooth, an abutment, and a prosthesis. The implant includes an implant body corresponding to the root portion, a scan body, and an abutment corresponding to the abutment portion. Furthermore, the implant body, abutment, abutment, and cavity-prepared tooth may be an inlay, an onlay, a crown, a bridge, a veneer, a denture, or the like.

[0026] The data processing device 10 may identify lesions from the scanned intraoral objects (teeth, gums) based on the three-dimensional data. A "lesion" is an area in the oral cavity where there is an abnormality, and includes, for example, areas with caries, areas with periodontal disease, areas with gingivitis, and areas with missing teeth. Furthermore, a "lesion" is not limited to areas that are actually diagnosed as diseased, but may also include areas that are likely to cause disease, such as areas with tartar buildup or plaque.

[0027] The data processing device 10 may estimate the depth of a periodontal pocket in a tooth based on the three-dimensional data. Furthermore, the data processing device 10 may estimate the position of early contact in a tooth based on the three-dimensional data.

[0028] Hereinafter, the various processes executed by the data processing device 10, such as the process of identifying the type of tooth, the process of identifying the part of the tooth, the process of identifying the processed part of the tooth, the process of identifying the diseased part of the tooth, the process of estimating the depth of the periodontal pocket in the tooth, and the process of estimating the early contact position in the tooth, will be collectively referred to as "data processing." Furthermore, the results obtained by data processing (identification results, estimation results) will be collectively referred to as "processing results."

[0029] The data processing device 10 may further add at least one piece of information from among the processed area, the lesion area, the periodontal pocket depth, and the early contact position to the diagram or table showing the identification results of the tooth type and tooth position. The data processing device 10 can display on the display 20 an image showing the processing results such as the lesion area, the processed area, the periodontal pocket depth, or the early contact position obtained for each tooth type and tooth position by outputting image data showing these processing results to the display 20.

[0030] The data processing device 10 can record the processing results, such as the lesion location, processed location, periodontal pocket depth, or early contact location, obtained for each tooth type and tooth location, as an electronic medical record in the storage device 13 described below, or transmit them to an external server device via the communication device 18. Note that the data processing device 10 may also record the processing results, such as the lesion location, processed location, periodontal pocket depth, or early contact location, obtained for each tooth type and tooth location, in the storage device 13 or transmit them to an external server device via the communication device 18, not limited to electronic medical records, as a hygienist work record for recording the work of a dental hygienist or a document related to dental hygienist practical training for recording the content of training provided by a dental hygienist.

[0031] 1, the data processing device 10 displays an image showing the surface shape of a portion of the dentition on the display 20, and also indicates by color the areas of the teeth where caries have occurred as identified by data processing, and the areas of the gums where periodontal disease has occurred as identified by data processing, in the image displayed on the display 20. The data processing device 10 also records the types and areas of the teeth where lesions such as caries or periodontal disease have occurred as identified by data processing as an electronic medical record, and displays the processing results on the display 20 using a diagram or table.

[0032] In this way, the data processing device 10 can automatically diagnose and / or suggest various tests, such as tooth type, tooth location, processed location, lesion location, periodontal pocket depth, and early contact location, based on the three-dimensional data. This allows the user to view an image showing the processed results displayed on the display 20 or to view an electronic medical record in which the processed results are recorded via the display 20, eliminating the need for the user to identify the tooth type or tooth location themselves or to perform various tests, such as the processed location, lesion location, periodontal pocket depth, and early contact location, themselves. This allows the user to avoid time-consuming dental examinations and the risk of incorrectly identifying tooth locations where caries or the like have developed and recording incorrect information in the medical record.

[0033] [Hardware configuration of data processing device] The hardware configuration of a data processing device 10 according to an embodiment will be described with reference to Fig. 2. Fig. 2 is a block diagram showing the hardware configuration of the data processing device 10 according to an embodiment. The data processing device 10 may be realized, for example, by a general-purpose computer or by a computer dedicated to the three-dimensional scanner 2.

[0034] 2, the data processing system 1 according to the embodiment includes a data processing device 10, a three-dimensional scanner 2, a display 20, and an input device 30 such as a keyboard 31 and a mouse 32. The data processing device 10 includes, as main hardware elements, an arithmetic unit 11, a memory 12, a storage device 13, a scanner interface 14, a display interface 15, an input device interface 16, a storage medium interface 17, and a communication device 18.

[0035] The arithmetic device 11 is a computing entity (computer) that executes various processes by executing various programs, and is an example of a "computing unit." The arithmetic device 11 is configured with a processor such as a CPU (Central Processing Unit), an MPU (Micro Processing Unit), a TPU (Tensor Processing Unit), or a GPU (Graphics Processing Unit). Note that a processor, which is an example of the arithmetic device 11, has the function of executing a predetermined process by executing a predetermined program, but some or all of these functions may be implemented using a dedicated hardware circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array). The "processor" is not limited to a processor in the narrow sense that executes processes using a stored program, such as a CPU, an MPU, a TPU, or a GPU, but may also include a hardwired circuit such as an ASIC or an FPGA. Furthermore, the arithmetic device 11 is not limited to a von Neumann computer such as a CPU or a GPU, but may also be configured with a non-von Neumann computer such as a quantum computer or an optical computer. The arithmetic device 11 described above can also be interpreted as a processing circuitry that executes a predetermined process. The computing device 11 may be configured as a single chip or multiple chips. Furthermore, the processor and related processing circuits may be configured as multiple computers interconnected by wire or wirelessly via a local area network or a wireless network. The processor and related processing circuits may be configured as a cloud computer that performs remote calculations based on input data and outputs the calculation results to other devices in remote locations.

[0036] The memory 12 includes a volatile storage area (for example, a working area) that temporarily stores program code, work memory, etc. when the arithmetic unit 11 executes various programs. Examples of the storage unit include volatile memories such as DRAM (dynamic random access memory) and SRAM (static random access memory), and non-volatile memories such as ROM (read only memory) and flash memory.

[0037] The storage device 13 stores various programs and various data executed by the arithmetic device 11. The storage device 13 may be one or more non-transitory computer readable media, or one or more computer readable storage media. Examples of the storage device 13 include a hard disk drive (HDD) and a solid state drive (SSD).

[0038] The storage device 13 stores a data processing program 100 and an estimation model 50. The data processing program 100 describes the contents of data processing that the arithmetic device 11 performs to identify the type of tooth or the location of the tooth using the estimation model 50 based on three-dimensional data indicating the surface shape of an object in the oral cavity.

[0039] The estimation model 50 includes a neural network 51 and parameters 52 used by the neural network 51 .

[0040] The neural network 51 is trained by machine learning to perform various data processing operations based on three-dimensional data of intraoral objects. Specifically, the neural network 51 is trained by machine learning (e.g., supervised learning) to identify tooth types by directly inputting the three-dimensional data. The neural network 51 is trained by machine learning (e.g., supervised learning) to identify each of multiple tooth regions by directly inputting the three-dimensional data. Here, "direct input of three-dimensional data" does not necessarily mean that the three-dimensional data acquired by the three-dimensional scanner 2 is directly input to the neural network 51 without modification, but also means that even if the three-dimensional data acquired by the three-dimensional scanner 2 is slightly modified by preprocessing, the data input to the neural network 51 is substantially unchanged from the original three-dimensional data acquired by the three-dimensional scanner 2. The neural network 51 is trained by machine learning (e.g., supervised learning) to identify processed tooth regions by directly inputting the three-dimensional data. The neural network 51 is trained by machine learning (e.g., supervised learning) to identify lesion sites in teeth by directly inputting three-dimensional data. The neural network 51 is trained by machine learning (e.g., supervised learning) to estimate periodontal pocket depths in teeth by directly inputting three-dimensional data.

[0041] The neural network 51 may be any algorithm that can be applied to the neural network 51 of the embodiment, such as an autoencoder, a convolutional neural network (CNN), a recurrent neural network (RNN), a transformer, or a generative adversarial network (GAN). Note that the estimation model 50 is not limited to the neural network 51, and may include other known algorithms such as Bayesian estimation or a support vector machine (SVM).

[0042] The parameters 52 include weighting coefficients used in the calculations by the neural network 51 and judgment values ​​used in the judgments during the calculations.

[0043] Scanner interface 14 is an example of an "input unit." Scanner interface 14 acquires three-dimensional data indicating the surface shape of an object in the oral cavity. For example, scanner interface 14 is communicatively connected to three-dimensional scanner 2 and acquires three-dimensional data from three-dimensional scanner 2. Scanner interface 14 may also be communicatively connected to a CT imaging device (not shown) and acquire three-dimensional data indicating the surface shape of the tooth generated based on tooth volume (voxel) data obtained by the CT imaging device. The three-dimensional data input from scanner interface 14 is stored in memory 12 or storage device 13 and is used when computing device 11 executes data processing.

[0044] The display interface 15 is an interface for connecting the display 20 and is an example of an "output unit." The display interface 15 realizes input and output of data between the data processing device 10 and the display 20. For example, the data processing device 10 outputs image data for displaying a diagram or table showing the identification results of tooth types and tooth locations to the display 20 via the display interface 15. The display 20 displays a diagram or table showing the identification results of tooth types and tooth locations based on the image data received via the display interface 15.

[0045] The input device interface 16 is an interface for connecting an input device 30 such as a keyboard 31 and a mouse 32. The input device interface 16 realizes input and output of data between the data processing device 10 and the input device 30. For example, a user can use the input device 30 to input command signals for moving a cursor in a figure or table displayed on the display 20 or for editing the figure or table. The data processing device 10 performs processing based on the command signals received via the input device interface 16.

[0046] The storage medium interface 17 reads various data stored on the removable disk 40, which is a storage medium, and writes various data to the removable disk 40. For example, the storage medium interface 17 may acquire the data processing program 100 from the removable disk 40 or write electronic medical record data generated by the arithmetic device 11 to the removable disk 40. The removable disk 40 may be one or more non-transitory computer-readable media or one or more computer-readable storage media. When the arithmetic device 11 acquires three-dimensional data indicating the surface shape of an object in the oral cavity from the removable disk 40 via the storage medium interface 17, the storage medium interface 17 may be an example of an "input unit." When the arithmetic device 11 outputs electronic medical record data to the removable disk 40 via the storage medium interface 17, the storage medium interface 17 may be an example of an "output unit."

[0047] The communication device 18 transmits and receives data to and from an external device via wired or wireless communication. For example, the communication device 18 acquires three-dimensional data indicating the surface shape of an object in the oral cavity from the external device. The three-dimensional data acquired by the communication device 18 is stored in the memory 12 or the storage device 13 and is used when the calculation device 11 executes data processing. The communication device 18 may also output electronic medical record data generated by the calculation device 11 to the external device. When the calculation device 11 acquires three-dimensional data indicating the surface shape of an object in the oral cavity from the external device via the communication device 18, the communication device 18 can be an example of an "input unit." When the calculation device 11 outputs electronic medical record data to the external device via the communication device 18, the communication device 18 can be an example of an "output unit."

[0048] [Location in teeth] Each of the multiple regions constituting the tooth surface identified by the data processing device 10 will be described with reference to Figures 3 to 5. Figure 3 is a diagram for explaining an example of a region on a tooth.

[0049] As shown in Figure 3, the surface of each tooth in the oral cavity is virtually divided into multiple regions according to the type of tooth. For maxillary first molars, second molars, third molars, first premolars, or second premolars, the region of each tooth includes at least one of the occlusal surface, distal surface, mesial surface, palatal surface, and buccal surface. For maxillary canines, lateral incisors, or central incisors, the region of each tooth includes at least one of the distal surface, mesial surface, palatal surface, and labial surface. For mandibular first molars, second molars, third molars, first premolars, or second premolars, the region of each tooth includes at least one of the occlusal surface, distal surface, mesial surface, lingual surface, and buccal surface. For mandibular canines, lateral incisors, or central incisors, the sites on each tooth include at least one of the distal surface, mesial surface, lingual surface, and labial surface.

[0050] Specifically, in the example shown in Figure 3, for the first molar, second molar, third molar, first premolar, and second premolar on the right or left maxilla, the surface of each tooth is divided into five surfaces: occlusal, distal, mesial, palatal, and buccal. For the canine, lateral incisor, and central incisor on the right or left maxilla, the surface of each tooth is divided into four surfaces: distal, mesial, palatal, and labial. For the first molar, second molar, third molar, first premolar, and second premolar on the right or left mandible, the surface of each tooth is divided into five surfaces: occlusal, distal, mesial, lingual, and buccal. For the canines, lateral incisors, and central incisors on the right or left mandible, the surface of each tooth is divided into four surfaces: distal, mesial, lingual, and labial.

[0051] Here, the surfaces where the maxillary first premolar, second premolar, first molar, second molar, and third molar respectively meet with the mandibular first premolar, second premolar, first molar, second molar, and third molar respectively are referred to as occlusal surfaces. The direction away from the tip of the dental arch (between the left central incisor and the right central incisor) is referred to as distal, and the surface facing distally is referred to as distal surface. The direction approaching the tip of the dental arch is referred to as mesial, and the surface facing mesial is referred to as mesial surface. The side of the maxillary dental arch closer to the palate is referred to as palatal side, and the surface facing the palate is referred to as palatal surface. The side of the mandibular dental arch closer to the tongue is referred to as lingual side, and the surface facing the tongue is referred to as lingual surface. The side of the maxillary or mandibular dentition that is closest to the lips is called the labial side, and the surface that faces the lips is called the labial surface.The side of the maxillary or mandibular dentition that is closest to the cheek is called the buccal side, and the surface that faces the cheek is called the buccal surface.

[0052] Note that the first molar, second molar, third molar, first premolar, or second premolar is not limited to the example shown in Fig. 3, and the regions of each tooth may be divided as shown in Fig. 4. Fig. 4 is a diagram for explaining modified examples of the regions of each tooth, the first molar, second molar, third molar, first premolar, or second premolar.

[0053] For maxillary first molars, second molars, third molars, first premolars, or second premolars, each tooth site includes multiple cusps that make up the tooth. Similarly, for mandibular first molars, second molars, third molars, first premolars, or second premolars, each tooth site includes multiple cusps that make up the tooth.

[0054] For example, as shown in Figure 4, the surface of the left mandibular second molar is divided into four cusps: the distal buccal cusp, the distal lingual cusp, the mesio-buccal cusp, and the mesio-lingual cusp. Similarly, the surface of the right mandibular second molar is divided into four cusps: the distal buccal cusp, the distal lingual cusp, the mesio-buccal cusp, and the mesio-lingual cusp. Although not shown, the surface of the left maxillary first molar is divided into four cusps: the distal buccal cusp, the distal palatal cusp, the mesio-buccal cusp, and the mesio-palatal cusp. Similarly, the surface of the right maxillary first molar is divided into four cusps: the distal buccal cusp, the distal palatal cusp, the mesio-buccal cusp, and the mesio-palatal cusp.

[0055] Here, a cusp on the distal side and close to the cheek side is called a distobuccal cusp. A cusp on the distal side and close to the lingual side is called a distolingual cusp. A cusp on the mesial side and close to the cheek side is called a mesiobuccal cusp. A cusp on the mesial side and close to the lingual side is called a mesiolingual cusp. A cusp on the distal side and close to the palate side is called a distopalatal cusp. A cusp on the mesial side and close to the palate side is called a mesiopalatal cusp.

[0056] Note that cusps may be included not only in first molars and second molars, but also in other teeth such as third molars, first premolars, or second premolars. For example, the tip of a canine may be referred to as a cusp. The number of cusps in a molar is not limited to four, but may be, for example, two to five. Note that the canine, lateral incisor, or central incisor is not limited to the example shown in FIG. 3, and each tooth may be divided into sections as shown in FIG. 5. FIG. 5 is a diagram for explaining modified examples of the sections of a canine, lateral incisor, or central incisor.

[0057] As shown in Figure 5, for maxillary canines, lateral incisors, or central incisors, the region of each tooth includes multiple surfaces that make up the labial surface. Similarly, for mandibular canines, lateral incisors, or central incisors, the region of each tooth includes multiple surfaces that make up the labial surface.

[0058] In the example shown in Figure 5, for maxillary canines, lateral incisors, or central incisors, the labial surface of each tooth is divided into three planes: cervical, central, and incisal. Similarly, for mandibular canines, lateral incisors, or central incisors, the labial surface of each tooth is divided into three planes: cervical, central, and incisal.

[0059] Here, the part of the labial surface that is closest to the gum is called the cervical part, the part of the labial surface that is closest to the tip of the tooth is called the incisal part, and the part of the labial surface that is located between the cervical part and the incisal part is called the central part.

[0060] As shown in Figures 3 to 5, the user can set tooth regions using the input device 30. The division of tooth regions shown in Figures 3 to 5 is an example, and the user may divide the tooth surface into multiple regions in other ways, or may combine the tooth regions shown in Figures 3 to 5.

[0061] [Functional configuration of data processing device] The functional configuration of the data processing device 10 will be described with reference to Fig. 6. Fig. 6 is a block diagram showing the functional configuration of the data processing device 10 according to the embodiment.

[0062] 6, the data processing device 10 includes, as functional units related to data processing, an input unit 101, a calculation unit 102, and an output unit 103. Each of these functions is realized by the calculation device 11 of the data processing device 10 executing a data processing program 100.

[0063] The input unit 101 is a functional unit realized by at least one of the scanner interface 14, the storage medium interface 17, and the communication device 18. Three-dimensional data indicating the surface shape of an object in the oral cavity is input to the input unit 101.

[0064] The calculation unit 102 is a functional unit realized by the calculation device 11. The calculation unit 102 executes predetermined data processing on the three-dimensional data input from the input unit 101.

[0065] For example, the calculation unit 102 identifies each of the multiple parts of the tooth based on the three-dimensional data input from the input unit 101 and the estimation model 50 that has been machine-learned to identify each of the multiple parts of the tooth from which the three-dimensional data was obtained based on the three-dimensional data. Specifically, the calculation unit 102 identifies whether each of the multiple parts of the tooth from which the three-dimensional data was obtained belongs to an upper jaw tooth or a lower jaw tooth.

[0066] More specifically, when the tooth to be identified is a maxillary first molar, second molar, third molar, first premolar, or second premolar, the calculation unit 102 estimates whether each of the multiple sites constituting the surface of the tooth to be identified corresponds to the occlusal surface, distal surface, mesial surface, palatal surface, or buccal surface using the estimation model 50. When the tooth to be identified is a maxillary canine, lateral incisor, or central incisor, the calculation unit 102 estimates whether each of the multiple sites constituting the surface of the tooth to be identified corresponds to the distal surface, mesial surface, palatal surface, or labial surface using the estimation model 50. When the tooth to be identified is a mandibular first molar, second molar, third molar, first premolar, or second premolar, the calculation unit 102 estimates whether each of the multiple sites constituting the surface of the tooth to be identified corresponds to the occlusal surface, distal surface, mesial surface, lingual surface, or buccal surface using the estimation model 50. When the tooth to be identified is a mandibular canine, lateral incisor, or central incisor, the calculation unit 102 estimates whether each of the multiple sites constituting the surface of the tooth to be identified corresponds to the distal surface, mesial surface, lingual surface, or labial surface using the estimation model 50.

[0067] Furthermore, the calculation unit 102 identifies the type of tooth based on the three-dimensional data input from the input unit 101 and an estimation model 50 that has undergone machine learning to identify the type of tooth from which the three-dimensional data was obtained.

[0068] The calculation unit 102 identifies the processed part of the tooth based on the three-dimensional data input from the input unit 101 and an estimation model 50 that has been machine-learned to identify the processed part of the tooth from which the three-dimensional data was obtained.

[0069] The calculation unit 102 identifies the lesion site in the tooth or gum based on the three-dimensional data input from the input unit 101 and an estimation model 50 that has been machine-learned to identify the lesion site in the tooth or gum from which the three-dimensional data was obtained.

[0070] The calculation unit 102 estimates the depth of the periodontal pocket in the tooth based on the three-dimensional data input from the input unit 101 and an estimation model 50 that has been machine-learned to estimate the depth of the periodontal pocket in the tooth from which the three-dimensional data was obtained.

[0071] The calculation unit 102 estimates, based on the three-dimensional data input from the input unit 101, a premature contact position where premature contact occurs when the upper and lower teeth occlude.

[0072] The output unit 103 is a functional unit realized by at least one of the display interface 15, the storage medium interface 17, and the communication device 18. The output unit 103 outputs image data to the display 20 to display a diagram or table showing the results of each data processing operation obtained by the calculation unit 102, such as the tooth location identification result, tooth type identification result, processed location identification result, lesion location identification result, periodontal pocket depth estimation result, and early contact position estimation result, and records the results of each data processing operation as an electronic medical record and outputs the recorded electronic medical record to the removable disk 40 or an external device.

[0073] The data processing device 10 having the various functional units described above can identify tooth regions using the calculation unit 102 based on three-dimensional data input to the input unit 101, and output the identification results using the output unit 103. This allows the user to use the data processing device 10 to easily and accurately identify tooth regions.

[0074] Furthermore, the data processing device 10 can calculate the results of tooth type identification, processed area identification, lesion area identification, periodontal pocket depth estimation, or early contact position estimation using the calculation unit 102 based on the three-dimensional data input to the input unit 101, and can output these data processing results using the output unit 103. This eliminates the need for the user to identify not only the tooth area but also the tooth type, processed area, lesion area, periodontal pocket depth, or early contact position, thereby shortening the time required for dental checkups and increasing the accuracy of the dental checkups.

[0075] Alternatively, three-dimensional data may be input to the input unit 101 each time teeth are scanned using the three-dimensional scanner 2, and the calculation unit 102 may use the three-dimensional data to calculate a tooth location identification result, tooth type identification result, processed location identification result, lesion location identification result, periodontal pocket depth estimation result, or early contact position estimation result each time three-dimensional data is input to the input unit 101, and the output unit 103 may output in real time these data processing results obtained during scanning by the three-dimensional scanner 2. This allows the user to obtain the data processing results in real time each time teeth are scanned using the three-dimensional scanner 2 during a dental checkup.

[0076] [An example of tooth part identification results] The results of identifying tooth regions obtained by the data processing device 10 according to the embodiment will be described with reference to Figures 7 to 10. Figures 7 to 10 are diagrams showing examples of the results of identifying tooth regions obtained by the data processing device 10 according to the embodiment.

[0077] As shown in FIGS. 7 to 10 , the data processing device 10 directly inputs three-dimensional data for each of the multiple teeth included in the maxillary dentition and the mandibular dentition to a neural network 51 of an estimation model 50. The neural network 51 is trained by machine learning to identify each of the multiple teeth based on the input three-dimensional data. Training the estimation model 50 using supervised learning uses learning data including three-dimensional data indicating the surface shape of each tooth and colors previously associated with multiple regions constituting the tooth surface shown by the three-dimensional data. Such coloring is not necessarily required, and any identification labels may be used as long as they can identify the multiple regions constituting the tooth surface shown by the three-dimensional data. For example, numerical values ​​that can identify the multiple regions constituting the tooth surface shown by the three-dimensional data may be associated with the multiple regions.

[0078] 9, for example, in the case of a second molar on the left side of the mandible, data indicating yellow is associated with three-dimensional data corresponding to the occlusal surface, data indicating blue is associated with three-dimensional data corresponding to the distal surface, data indicating red is associated with three-dimensional data corresponding to the buccal surface, and data indicating green is associated with three-dimensional data corresponding to the mesial surface. Meanwhile, although not shown, in the case of a second molar on the right side of the mandible, data indicating orange is associated with three-dimensional data corresponding to the occlusal surface, data indicating navy blue is associated with three-dimensional data corresponding to the distal surface, data indicating pink is associated with three-dimensional data corresponding to the buccal surface, and data indicating yellow-green is associated with three-dimensional data corresponding to the mesial surface. In other words, in the training data, a different color is associated as correct answer data for each of the three-dimensional data corresponding to each region of each tooth, which is the input data.

[0079] Furthermore, for the first molar, second molar, third molar, first premolar, and second premolar, a predetermined color may be associated with the three-dimensional data corresponding to each of the multiple cusps constituting each tooth. For example, as shown in Fig. 10, for the second molar on the left mandibular side, red data may be associated with the three-dimensional data corresponding to the distal buccal cusp, purple data may be associated with the three-dimensional data corresponding to the distal lingual cusp, yellow data may be associated with the three-dimensional data corresponding to the mesial buccal cusp, and blue data may be associated with the three-dimensional data corresponding to the mesial lingual cusp. On the other hand, although not shown, orange data may be associated with the three-dimensional data corresponding to the distal buccal cusp, pink data may be associated with the three-dimensional data corresponding to the distal lingual cusp, yellow-green data may be associated with the three-dimensional data corresponding to the mesial buccal cusp, and dark blue data may be associated with the three-dimensional data corresponding to the mesial lingual cusp. That is, in the learning data, different colors are associated with each site as correct answer data for the three-dimensional data corresponding to each cusp of each tooth, which is the input data.

[0080] In the training phase, the estimation model 50 finds features of the surface shape of an object represented by the three-dimensional data input to the neural network 51, estimates which part of which type of tooth the three-dimensional data corresponds to, and outputs the color associated with the estimated part as the identification result. The estimation model 50 determines whether the color of the output identification result matches the color of the correct data, and if they do not match, optimizes the parameters 52 so that they match. As a result, in the operation phase, the estimation model 50 can accurately estimate the color of the part of the tooth corresponding to the three-dimensional data, of the type of tooth corresponding to the three-dimensional data, based on the three-dimensional data input to the neural network 51, and output the estimated color as the identification result.

[0081] For example, as shown in Figure 7, when the data processing device 10 displays the surface shape of the upper and lower teeth, which are the subject of three-dimensional data acquisition, on the display 20 using multiple points (point cloud) having three-dimensional data, it uses an estimation model 50 to estimate the color of each part based on the three-dimensional data, assigns the estimated color to the points corresponding to each part, and further displays the surface shape of the upper and lower teeth indicated by the colored points on the display 20.

[0082] For example, as shown in Figure 8, when the data processing device 10 displays the surface shape of the upper and lower dental arches, which are the subject of three-dimensional data acquisition, on the display 20 using a polygon mesh containing three-dimensional data, it uses an estimation model 50 to estimate the color of each part based on the three-dimensional data, applies the estimated color to the polygon mesh corresponding to each part, and further displays the surface shape of the upper and lower dental arches shown by the colored polygon mesh on the display 20.

[0083] Furthermore, the data processing device 10 is not limited to identifying the location of each tooth based on three-dimensional data of multiple teeth such as a dentition, but may also identify the location of a single tooth based on three-dimensional data of that single tooth.

[0084] For example, as shown in Figures 9 and 10, when the data processing device 10 displays on the display 20 the surface shape of a tooth from which three-dimensional data is to be acquired, it uses the estimation model 50 to estimate the color of each part that makes up the surface of the tooth based on the three-dimensional data, and displays on the display 20 an image of the tooth with the estimated color assigned to each part.

[0085] In this way, the data processing device 10 can identify tooth regions based on the three-dimensional data and display the identification results in color using the estimation model 50. This allows the user to use the data processing device 10 to easily and accurately identify tooth regions.

[0086] [Data processing example] An example of data processing executed by the data processing device 10 will be described with reference to Fig. 11 to Fig. 13. Fig. 11 to Fig. 13 are diagrams showing an example of data processing using an estimation model 50 by the data processing device 10 according to the embodiment.

[0087] 11, the data processing device 10 may estimate the type and location of teeth based on three-dimensional data using one estimation model 50. In this case, a neural network 51 of the estimation model 50 is trained by machine learning (e.g., supervised learning) to identify both the type and location of teeth by directly inputting three-dimensional data of objects in the oral cavity.

[0088] As shown in FIG. 12, the data processing device 10 may include, as the estimation model 50, a first estimation model 50A including a first neural network 51A for identifying the type of tooth, and a second estimation model 50B including a second neural network 51B for identifying the part of the tooth.

[0089] Specifically, the data processing device 10 may identify the type of tooth using the first estimation model 50A based on three-dimensional data indicating the surface shape of an object in the oral cavity. In this case, the first neural network 51A of the first estimation model 50A is trained by machine learning (e.g., supervised learning) so as to identify the type of tooth by directly inputting the three-dimensional data of the object in the oral cavity.

[0090] For example, in the training phase, the first estimation model 50A finds features of the surface shape of an object represented by three-dimensional data input to the first neural network 51A, estimates the type of tooth corresponding to the three-dimensional data, and outputs the estimation result. The first estimation model 50A determines whether the output estimation result matches the tooth type represented by the correct answer data, and if they do not match, optimizes the parameters 52 so that they match. As a result, in the operation phase, the first estimation model 50A can accurately estimate the type of tooth corresponding to the three-dimensional data based on the three-dimensional data input to the first neural network 51A.

[0091] The data processing device 10 employs the method disclosed in Japanese Patent Laid-Open No. 2020-096691 (Japanese Patent No. 6650996) as a method for identifying the type of tooth using the first estimation model 50A based on three-dimensional data representing the surface shape of an object in the oral cavity. For this reason, the disclosure of Japanese Patent Laid-Open No. 2020-096691 (Japanese Patent No. 6650996) is incorporated herein by reference.

[0092] The data processing device 10 may identify tooth regions using the second estimation model 50B based on three-dimensional data indicating the surface shape of an intraoral object associated with the tooth type estimated by the first estimation model 50A. In this case, the second neural network 51B of the second estimation model 50B is trained by machine learning (e.g., supervised learning) so as to identify tooth regions by directly inputting the three-dimensional data of the intraoral object associated with the tooth type.

[0093] For example, in the training phase, the second estimation model 50B finds features of the surface shape of an object represented by three-dimensional data previously associated with tooth types input to the second neural network 51B, estimates the tooth region corresponding to the three-dimensional data, and outputs the estimation result. The second estimation model 50B determines whether the output estimation result matches the tooth region represented by the correct answer data, and if they do not match, optimizes the parameters 52 so that they match. As a result, in the operation phase, the second estimation model 50B can accurately estimate the tooth region corresponding to the three-dimensional data previously associated with tooth types based on the three-dimensional data input to the second neural network 51B. Unlike the estimation model 50 shown in FIG. 11, the second estimation model 50B does not need to identify the tooth type because the tooth type is previously associated with the input three-dimensional data. Therefore, the second estimation model 50B has improved machine learning efficiency compared to the estimation model 50 shown in FIG. 11, and can more accurately estimate the tooth region.

[0094] Note that the first estimation model 50A and the second estimation model 50B shown in FIG. 12 may be different estimation models, or as shown in FIG. 11, one estimation model 50 may have the functions of the first estimation model 50A and the second estimation model 50B.

[0095] As shown in Figure 13, the data processing device 10 may include, as estimation models 50, a first estimation model 50A including a first neural network 51A for identifying the type of tooth, a second estimation model 50B including a second neural network 51B for identifying parts of the tooth, a third estimation model 50C including a third neural network 51C for identifying processed parts of the tooth, a fourth estimation model 50D including a fourth neural network 51D for identifying diseased parts of the tooth, and a fifth estimation model 50E including a fifth neural network 51E for estimating the depth of periodontal pockets in the tooth.

[0096] Specifically, the data processing device 10 may identify the type of tooth using the first estimation model 50A based on three-dimensional data indicating the surface shape of an object in the oral cavity. In this case, as described in Fig. 12, the first neural network 51A of the first estimation model 50A is trained by machine learning (e.g., supervised learning) so as to identify the type of tooth by directly inputting the three-dimensional data of the object in the oral cavity.

[0097] The data processing device 10 may identify a region of the tooth using the second estimation model 50B based on three-dimensional data indicating the surface shape of an object in the oral cavity. In this case, as described with reference to Figures 7 to 10, the second neural network 51B of the second estimation model 50B is trained by machine learning (e.g., supervised learning) so as to identify a region of the tooth by directly inputting the three-dimensional data of the object in the oral cavity.

[0098] The data processing device 10 may identify the processed portion of the tooth using the third estimation model 50C based on the three-dimensional data representing the surface shape of the object in the oral cavity. In this case, the third neural network 51C of the third estimation model 50C is trained by machine learning (e.g., supervised learning) so as to identify the processed portion of the tooth by directly inputting the three-dimensional data of the object in the oral cavity.

[0099] For example, in the training phase, the third estimation model 50C finds features of the surface shape of the object represented by the three-dimensional data input to the third neural network 51C, estimates the machining area corresponding to the three-dimensional data, and outputs the estimation result. The third estimation model 50C determines whether the output estimation result matches the machining area of ​​the tooth, which is the correct data, and if they do not match, optimizes the parameters 52 so that they match. As a result, in the operation phase, the third estimation model 50C can accurately estimate the machining area of ​​the tooth corresponding to the three-dimensional data based on the three-dimensional data input to the third neural network 51C.

[0100] The data processing device 10 employs the method disclosed in Japanese Patent Laid-Open No. 2022-012198 (Patent No. 7267974) as a method for identifying the processed portion of the tooth using the third estimation model 50C based on three-dimensional data showing the surface shape of the object in the oral cavity. Therefore, the disclosure of Japanese Patent Laid-Open No. 2022-012198 (Patent No. 7267974) is incorporated herein by reference.

[0101] The data processing device 10 may identify a lesion site in a tooth using the fourth estimation model 50D based on three-dimensional data representing the surface shape of an object in the oral cavity. In this case, the fourth neural network 51D of the fourth estimation model 50D is trained by machine learning (e.g., supervised learning) so as to identify a lesion site in a tooth by directly inputting the three-dimensional data of the object in the oral cavity.

[0102] For example, in the training phase, the fourth estimation model 50D finds features of the surface shape of an object represented by the three-dimensional data input to the fourth neural network 51D, estimates the lesion location corresponding to the three-dimensional data, and outputs the estimation result. The fourth estimation model 50D determines whether the output estimation result matches the lesion in the tooth (the correct answer data), and if they do not match, optimizes the parameter 52 so that they match. As a result, in the operation phase, the fourth estimation model 50D can accurately estimate the lesion location in the tooth corresponding to the three-dimensional data based on the three-dimensional data input to the fourth neural network 51D.

[0103] The data processing device 10 employs the method disclosed in Japanese Patent Laid-Open No. 2022-012199 (Japanese Patent No. 7324735) as a method for identifying a lesion site in a tooth using the fourth estimation model 50D based on three-dimensional data representing the surface shape of an object in the oral cavity. For this reason, the disclosure of Japanese Patent Laid-Open No. 2022-012199 (Japanese Patent No. 7324735) is incorporated herein by reference.

[0104] The data processing device 10 may estimate the depth of a periodontal pocket in a tooth using a fifth estimation model 50E based on three-dimensional data representing the surface shape of an object in the oral cavity. In this case, a fifth neural network 51E of the fifth estimation model 50E is trained by machine learning (e.g., supervised learning) so as to estimate the depth of a periodontal pocket in a tooth by directly inputting the three-dimensional data of the object in the oral cavity.

[0105] For example, in the training phase, the fifth estimation model 50E identifies features of the surface shape of an object represented by three-dimensional data input to the fifth neural network 51E, estimates the depth of the periodontal pocket corresponding to the three-dimensional data, and outputs the estimation result. Specifically, the fifth neural network 51E of the fifth estimation model 50E receives input of composite image data obtained by combining IOS data and CT data acquired for the same subject, as well as a predetermined measurement direction and predetermined measurement points for measuring the periodontal pocket depth. The fifth estimation model 50E estimates the periodontal pocket depth based on this input data. The fifth estimation model 50E determines whether the output estimation result matches the periodontal pocket depth data that is the correct data, and if they do not match, optimizes the parameters 52 so that they match. As a result, in the operation phase, the fifth estimation model 50E can accurately estimate the periodontal pocket depth of the tooth corresponding to the three-dimensional data input to the fifth neural network 51E.

[0106] The data processing device 10 employs the method disclosed in Japanese Patent Application No. 2023-013834 as a method for estimating the depth of a periodontal pocket in a tooth using the fifth estimation model 50E based on three-dimensional data representing the surface shape of an object in the oral cavity. Therefore, the disclosure of Japanese Patent Application No. 2023-013834 is incorporated herein by reference.

[0107] The data processing device 10 may estimate the early contact position where early contact occurs when the upper and lower teeth occlude by performing an early contact position estimation process based on three-dimensional data indicating the surface shape of an object in the oral cavity.

[0108] For example, maxillary dentition data indicating the surface shape of the maxillary dentition of the subject acquired by the three-dimensional scanner 2, and mandibular dentition data indicating the surface shape of the mandibular dentition of the subject acquired by the three-dimensional scanner 2 are input to the data processing device 10. Using a rendering image indicating the maxillary dentition generated based on the maxillary dentition data and a rendering image indicating the mandibular dentition generated based on the mandibular dentition data, the data processing device 10 performs image processing so that the upper and lower dentitions are displaced from an occlusal state to an open state, thereby calculating the early contact position.

[0109] The data processing device 10 employs the method disclosed in Japanese Patent Application No. 2022-182611 as a method for estimating the early contact position based on three-dimensional data indicating the surface shape of an object in the oral cavity. Therefore, the disclosure of Japanese Patent Application No. 2022-182611 is incorporated herein by reference.

[0110] In this way, the data processing device 10 can estimate at least one of the tooth type and tooth location, as well as the processed tooth location, the lesioned tooth location, the periodontal pocket depth, and the early contact position on the tooth, based on three-dimensional data representing the surface shape of an object in the oral cavity. The first estimation model 50A for estimating the tooth type, the second estimation model 50B for estimating the tooth location, the third estimation model 50C for estimating the processed tooth location, the fourth estimation model 50D for estimating the lesioned tooth location, and the fifth estimation model 50E for estimating the periodontal pocket depth all have in common the fact that they calculate estimation results using three-dimensional data. Therefore, the data processing device 10 can collectively estimate the tooth type, the tooth location, the processed tooth location, the lesioned tooth location, the periodontal pocket depth, and the early contact position on the tooth simply by using the common three-dimensional data for each estimation model 50. The data processing device 10 can effectively utilize the estimation results obtained by each estimation model 50 by adding at least one of information on the processed area, the lesion area, the periodontal pocket depth, and the early contact position to a diagram or table showing the identification results of tooth type and tooth location. That is, the systems used in each of the above-mentioned identifications use the same coordinate system for three-dimensional data, and the position coordinates included in each identification result are relatively correct. This allows the data processing device 10 to easily determine which tooth and which part of the tooth has been processed or has a lesion. When using identification results obtained by systems using different coordinate systems, the data processing device 10 can simply perform a conversion process to relatively correct the position coordinates included in the identification results. This conversion process can be performed using a typical three-dimensional data alignment or registration process, such as ICP (Iterative Closest Point) or other techniques.

[0111] For example, the data processing device 10 may identify the type and location of a tooth where a processed portion is present, and display the results in a diagram or table, or output them as an electronic medical record. The data processing device 10 may identify the type and location of a tooth where a lesion portion is present, and display the results in a diagram or table, or output them as an electronic medical record. The data processing device 10 may estimate the periodontal pocket depth for each tooth type and location, and display the results in a diagram or table, or output them as an electronic medical record. The data processing device 10 may estimate the early contact position for each tooth type and location, and display the results in a diagram or table, or output them as an electronic medical record. Furthermore, the data processing device 10 may combine multiple results selected from the processed portion, lesion portion, periodontal pocket depth, and early contact position, associate them with the tooth type and location, and display the results in a diagram or table, or output them as an electronic medical record.

[0112] This eliminates the need for the user to perform various tests such as those for processed areas, diseased areas, periodontal pocket depth, or early contact locations, and also eliminates the need for the user to identify the types and areas of the teeth that are the subject of these tests. This allows the user to avoid time-consuming dental checkups and recording incorrect information in the patient chart.

[0113] 13 may be different estimation models, or one estimation model 50 may have the functions of the first estimation model 50A, the second estimation model 50B, the third estimation model 50C, the fourth estimation model 50D, and the fifth estimation model 50E. Furthermore, the data processing device 10 may be configured to combine multiple estimation models included in the first estimation model 50A, the second estimation model 50B, the third estimation model 50C, the fourth estimation model 50D, and the fifth estimation model 50E into a single model to obtain multiple types of classification results. For example, a single estimation model combining the first estimation model 50A and the second estimation model 50B may estimate the type of tooth and the location of the tooth.

[0114] [Modification of tooth region identification process] Fig. 14 is a diagram showing a modified example of the process of identifying tooth regions by the data processing device 10 according to the embodiment. As shown in Fig. 14, the data processing device 10 may estimate the type of tooth using a first estimation model 50A, and then estimate the tooth region according to tooth surface extraction logic without using a second estimation model 50B as exemplified in Fig. 12. An example of the identification process using tooth surface extraction logic will be described below with reference to Figs. 15 to 28.

[0115] 15 to 28 are diagrams showing an example of the identification process using tooth surface extraction logic by the data processing device 10 according to the embodiment. First, with reference to Figs. 15 to 21, an example will be described in which the data processing device 10 identifies tooth parts for the canine, lateral incisor, and central incisor of the upper or lower jaw.

[0116] As shown in FIG. 15, the data processing device 10 generates a rendering image showing the dental arch based on three-dimensional data showing the surface shape of the dental arch of the upper or lower jaw. FIG. 15 shows a rendering image showing the dental arch of the lower jaw. The data processing device 10 calculates the center of gravity of each tooth that makes up the dental arch. The data processing device 10 draws a curve (or a broken line) that passes through the center of gravity of each tooth in the rendering image. The data processing device 10 calculates the center of gravity of the curve (i.e., the center of gravity of the dental arch). The data processing device 10 may calculate the center of gravity of the teeth and dental arch based on the coordinates of each three-dimensional data. Furthermore, the data processing device 10 sets the tip of the dental arch through which the curve passes (between the left central incisor and the right central incisor) as the apex of the curve.

[0117] As shown in FIG. 16, the data processing device 10 extracts an image of one tooth from the teeth included in the tooth row shown in FIG. 15. In FIG. 16, an image of a lateral incisor included in the mandibular tooth row is extracted. The data processing device 10 calculates the coordinates of a bounding box surrounding the tooth, and generates a bounding box based on the calculated coordinates to surround the tooth. The data processing device 10 calculates the center of gravity of the tooth surrounded by the bounding box. The data processing device 10 generates axes of the tooth that pass through the center of gravity of the tooth and extend along the sides of the bounding box. For example, the data processing device 10 generates a first axis that passes through the center of gravity of the tooth and extends along the long sides of the bounding box, and a second axis that passes through the center of gravity of the tooth and extends along the short sides of the bounding box. The data processing device 10 sets the longest axis (e.g., the first axis) of the generated axes as the tooth axis.

[0118] As shown in FIG. 17, the data processing device 10 sets a plane along the direction of the curve of the tooth row (the curve shown in FIG. 15) as a plane P from among a plurality of planes passing through the centers of gravity of the teeth.

[0119] 18, the data processing device 10 generates a plane Q by rotating the plane P by a predetermined angle D1 in the first rotation direction around the tooth axis as the central axis. Note that the angle D1 can be set by the user.

[0120] 19, the data processing device 10 generates a plane R by rotating the plane P by a predetermined angle D2 in a second rotation direction opposite to the first rotation direction, with the tooth axis as the central axis. Note that the angle D2 can be set by the user, and may be the same angle as the angle D1 or a different angle.

[0121] As shown in FIG. 20, the data processing device 10 divides the tooth in the directions along each of the planes R and Q, thereby dividing the tooth surface into four parts.

[0122] As shown in FIG. 21 , the data processing device 10 assigns one of the distal surface, mesial surface, labial surface, and lingual surface to each of the four surface portions of the divided tooth (in this example, the lateral incisor). For example, the data processing device 10 sets the surface that has more contact with the adjacent teeth and is closer to the apex of the curve of the dental arch as the mesial surface. The data processing device 10 sets the surface that has more contact with the adjacent teeth and is farther from the apex of the curve of the dental arch as the distal surface. The data processing device 10 sets the surface that is farther from the center of gravity of the dental arch as the labial surface. The data processing device 10 sets the surface that is closer to the center of gravity of the dental arch as the lingual surface.

[0123] In the above-mentioned Figures 15 to 21, an example of identifying areas on the surface of a mandibular lateral incisor is given, but the data processing device 10 can identify areas on each tooth using the method shown in Figures 15 to 21 for both the mandibular canine and central incisor, and the maxillary canine, lateral incisor, and central incisor.

[0124] Next, with reference to FIGS. 22 to 28, an example will be described in which the data processing device 10 identifies tooth locations for the first premolar, second premolar, first molar, second molar, and third molar of the upper or lower jaw.

[0125] First, using the method shown in Fig. 15, the data processing device 10 draws a curve passing through the center of gravity of each tooth in a rendering image showing the row of teeth generated based on three-dimensional data showing the surface shape of the row of teeth of the upper or lower jaw, and calculates the center of gravity of the curve. The data processing device 10 also determines the tip of the row of teeth through which the curve passes (between the left central incisor and the right central incisor) as the apex of the curve.

[0126] As shown in FIG. 22, the data processing device 10 extracts an image of one tooth from the teeth included in the tooth row shown in FIG. 15. In FIG. 22, an image of a first molar included in the mandibular tooth row is extracted. The data processing device 10 calculates the coordinates of a bounding box surrounding the tooth, and generates a bounding box based on the calculated coordinates to surround the tooth. The data processing device 10 calculates the center of gravity of the tooth surrounded by the bounding box. The data processing device 10 generates axes of the tooth that pass through the center of gravity of the tooth and extend along the sides of the bounding box. For example, the data processing device 10 generates a first axis that passes through the center of gravity of the tooth and extends along the long sides of the bounding box, and a second axis that passes through the center of gravity of the tooth and extends along the short sides of the bounding box. The data processing device 10 sets the shortest axis (e.g., the second axis) of the generated axes as the tooth axis.

[0127] As shown in Fig. 23, the data processing device 10 applies a cylinder of a predetermined radius r to the tooth, with the tooth axis as its central axis. The data processing device 10 sets the tooth surface contained within the cylinder as the occlusal surface. The radius r can be set by the user. The data processing device 10 is not limited to applying a cylinder, and may also apply an elliptical cylinder or a rectangular prism to the tooth.

[0128] As shown in FIG. 24, the data processing device 10 sets a plane that is along the direction of the curve of the tooth row (the curve shown in FIG. 15) among a plurality of planes that pass through the centers of gravity of the teeth as a plane P'.

[0129] 25, the data processing device 10 generates a plane Q' by rotating the plane P' in the first rotation direction by a predetermined angle D1' around the tooth axis as the central axis. Note that the angle D1' can be set by the user.

[0130] 26, the data processing device 10 generates a plane R' by rotating the plane P' by a predetermined angle D2' in a second rotation direction opposite to the first rotation direction, with the tooth axis as the central axis. Note that the angle D2' can be set by the user, and may be the same as or different from the angle D1'.

[0131] As shown in FIG. 27, the data processing device 10 can divide the tooth surface into four parts by dividing the tooth in the directions along each of the planes R' and Q'.

[0132] As shown in FIG. 28, the data processing device 10 assigns one of the occlusal surface, distal surface, mesial surface, lingual surface, and buccal surface to each of the five surface portions of the divided tooth (in this example, the first molar). For example, the data processing device 10 sets the surface determined by the cylinder shown in FIG. 23 as the occlusal surface. The data processing device 10 sets the surface that has more contact with adjacent teeth and is closer to the apex of the curve of the dental arch as the mesial surface. The data processing device 10 sets the surface that has more contact with adjacent teeth and is farther from the apex of the curve of the dental arch as the distal surface. The data processing device 10 sets the surface that is farther from the center of gravity of the dental arch as the buccal surface. The data processing device 10 sets the surface that is closer to the center of gravity of the dental arch as the lingual surface.

[0133] 22 to 28 described above show an example of identifying a location on the surface of a mandibular first molar, but the data processing device 10 can identify a location on each tooth using the method shown in FIGS. 22 to 28 for any of the mandibular first molar, second molar, first premolar, and second premolar, and the maxillary first molar, second molar, third molar, first premolar, and second premolar.

[0134] In this way, the data processing device 10 can identify each of the multiple regions based on the multiple regions divided by multiple planes along the tooth axis passing through the center of gravity of the tooth and the center of gravity of the dental arch including the tooth. Furthermore, the data processing device 10 can identify the occlusal surface of the tooth using at least one of a circular cylinder, an elliptical cylinder, and a rectangular cylinder with the tooth axis as its central axis. This allows the data processing device 10 to estimate the region of the tooth according to tooth surface extraction logic without using the estimation model 50, eliminating the need to train the estimation model 50 by machine learning.

[0135] [Data Processing] The data processing executed by the data processing device 10 according to the embodiment will be described with reference to Fig. 29. Fig. 29 is a flowchart defining the data processing executed by the data processing device 10 according to the embodiment. Each step (hereinafter, indicated by "S") shown in Fig. 29 is realized by the arithmetic device 11 of the data processing device 10 executing the data processing program 100.

[0136] 29, the data processing device 10 determines whether or not three-dimensional data including three-dimensional position information corresponding to each of a plurality of points indicating the surface shape of an object in the oral cavity has been acquired (S1). If three-dimensional data has not been acquired (NO in S1), the data processing device 10 ends this process.

[0137] On the other hand, when the data processing device 10 acquires three-dimensional data (YES in S1), it identifies the type of tooth based on the three-dimensional data (S2). For example, the data processing device 10 directly inputs the three-dimensional data acquired in S1 into the neural network 51 (or the first neural network 51A) of the estimation model 50 (or the first estimation model 50A), thereby identifying the type of tooth from which the three-dimensional data is acquired.

[0138] The data processing device 10 identifies each of the multiple regions constituting the tooth surface based on the three-dimensional data (S3). For example, the data processing device 10 directly inputs the three-dimensional data acquired in S1 into the neural network 51 (or the second neural network 51B) of the estimation model 50 (or the second estimation model 50B) to identify each of the multiple regions constituting the tooth surface from which three-dimensional data is to be acquired. Alternatively, the data processing device 10 directly inputs the three-dimensional data to which the tooth type identification result acquired in S2 has been added into the neural network 51 (or the second neural network 51B) of the estimation model 50 (or the second estimation model 50B) to identify each of the multiple regions constituting the tooth surface from which three-dimensional data is to be acquired. Alternatively, the data processing device 10 identifies each of the multiple regions constituting the tooth surface from which three-dimensional data is to be acquired according to the tooth surface extraction logic described with reference to FIGS. 15 to 28.

[0139] The data processing device 10 identifies the processed portion of the tooth based on the three-dimensional data (S4). For example, the data processing device 10 directly inputs the three-dimensional data acquired in S1 into the neural network 51 (or the third neural network 51C) of the estimation model 50 (or the third estimation model 50C), thereby identifying the processed portion of the tooth from which the three-dimensional data is acquired.

[0140] The data processing device 10 identifies the lesion site in the tooth based on the three-dimensional data (S5). For example, the data processing device 10 directly inputs the three-dimensional data acquired in S1 into the neural network 51 (or the fourth neural network 51D) of the estimation model 50 (or the fourth estimation model 50D), thereby identifying the lesion site in the tooth from which the three-dimensional data is acquired.

[0141] The data processing device 10 estimates the depth of the periodontal pocket in the tooth based on the three-dimensional data (S6). For example, the data processing device 10 directly inputs the three-dimensional data acquired in S1 into the neural network 51 (or the fifth neural network 51E) of the estimation model 50 (or the fifth estimation model 50E), thereby estimating the depth of the periodontal pocket in the tooth from which the three-dimensional data is acquired.

[0142] The data processing device 10 estimates the position of premature contact of the teeth based on the three-dimensional data (S7). For example, the data processing device 10 separates the three-dimensional data acquired in S1 into maxillary dentition data showing the surface shape of the maxillary dentition and mandibular dentition data showing the surface shape of the mandibular dentition, and estimates the position of premature contact by performing image processing using the maxillary dentition data and the mandibular dentition data so that the upper and lower dentitions are displaced from an occlusal state to an open state.

[0143] The data processing device 10 may execute only the process of S3, or may execute at least one of the processes of S4 to S7 in addition to the process of S3. The data processing device 10 may execute the processes of S2 and S3, or may execute at least one of the processes of S4 to S7 in addition to the processes of S2 and S3.

[0144] The data processing device 10 outputs the results of the data processing obtained by the processes of S2 to S7 to the display 20 and reflects them in the electronic medical record (S8). Thereafter, the data processing device 10 ends this process. Note that the data processing device 10 may execute the processes of S2 to S8 in real time every time three-dimensional data obtained during scanning by the three-dimensional scanner 2 is acquired in S1, or may execute the processes of S2 to S8 on the condition that a predetermined amount of three-dimensional data has been acquired in S1.

[0145] [Example of an electronic medical record] An example of an electronic medical record generated by the data processing device 10 will be described with reference to FIGS. 30 to 35. FIGS. 30 to 35 are diagrams showing an example of an electronic medical record generated by the data processing device 10 according to the embodiment. The data processing device 10 reflects the processing results of the tooth type, tooth location, processed location, lesion location, periodontal pocket depth, or early contact position obtained by data processing in the electronic medical record, and displays a diagram or table showing the contents of the electronic medical record on the display 20. Here, reflecting each processing result in the electronic medical record means that the data processing device 10 grasps and stores the correspondence between the items corresponding to each location of each tooth type in the electronic medical record format and each recognized location of each tooth type, and further grasps and stores the correspondence between each recognized location of each tooth type and at least one of the presence or type of processing, the presence or degree or type of lesion, the depth of periodontal pocket, and the presence or degree of early contact, based on the coordinate position of each location. As a result, the data processing device 10 can display on the display 20, based on this stored information, a diagram or table showing various conditions at each site for each tooth type.

[0146] 30, the data processing device 10 aligns multiple teeth included in the dentition and displays them on the display 20, and reflects the processing results of the processed area, diseased area, periodontal pocket depth, or early contact position for each tooth. In particular, the data processing device 10 reflects in the electronic medical record which parts of the multiple parts that make up the tooth surface have processed areas or diseased areas, and displays an image showing the reflection results on the display 20.

[0147] As shown in FIG. 31 , the data processing device 10 may create a chart summarizing the processing results of the periodontal pocket depth for each tooth and display the created chart on the display 20. In particular, the data processing device 10 summarizes the processing results of the periodontal pocket depth for each of the multiple sites constituting the tooth surface in a chart and displays the chart on the display 20. The data processing device 10 also summarizes the presence or absence of plaque (dental plaque) adhering to the tooth surface for each of the multiple sites constituting the tooth surface in a chart and displays the chart on the display 20. Furthermore, the data processing device 10 summarizes the degree of tooth mobility, which indicates tooth mobility, for each tooth in a chart and displays the chart on the display 20. Note that the degree of tooth mobility may be measured by a dentist or the like based on the feel of the tooth when touched, and the result may be input to the data processing device 10.

[0148] 32, the data processing device 10 may display at least one of the maxillary dentition and the mandibular dentition in three dimensions on the display 20. Furthermore, for the teeth included in the dentition shown in three dimensions, the data processing device 10 may annotate tooth numbers indicating the tooth types, annotate processed areas such as inlays (metal) and abutment teeth, annotate diseased areas such as defects, and annotate contact positions with opposing teeth, such as areas that will be early contact positions or areas that will be final contact positions.

[0149] 33, the data processing device 10 may enlarge a portion of the dentition and display some of the teeth in three dimensions on the display 20. Furthermore, the data processing device 10 may annotate lesions such as caries in the teeth displayed in three dimensions.

[0150] As shown in FIG. 34, the data processing device 10 may annotate the positions of the identified cusps for the first molar, second molar, third molar, first premolar, or second premolar in the three-dimensional representation of the teeth.

[0151] 35, the data processing device 10 may display a three-dimensional image showing the rows of teeth of the upper and lower jaws and a two-dimensional image showing the dental formula side by side on the display 20. Furthermore, when a user adds an annotation to a predetermined position on the teeth displayed in three dimensions, the data processing device 10 may add the annotation to the predetermined position on the teeth displayed in two dimensions. Furthermore, when a user places a cursor on a predetermined position on the teeth displayed in three dimensions, the data processing device 10 may highlight the predetermined position on the teeth displayed in two dimensions.

[0152] 35, for example, caries has occurred on the buccal surface of the mandibular second premolar, and when the data processing device 10 receives an input from the input device 30 to add an annotation of "caries" to the buccal surface of the second premolar shown in the three-dimensional image, the data processing device 10 also adds the annotation of "caries" to the buccal surface of the second premolar shown in the two-dimensional image. Furthermore, when the data processing device 10 receives an input from the input device 30 to place a cursor on the buccal surface of the second premolar shown in the three-dimensional image, the data processing device 10 also places the cursor on the buccal surface of the second premolar shown in the two-dimensional image and displays the annotation "caries," thereby emphasizing the buccal surface of the second premolar where caries has occurred.

[0153] Conversely, when a user adds an annotation to a predetermined position of a tooth displayed in two dimensions, the data processing device 10 may add the annotation to the predetermined position of the tooth displayed in three dimensions. Furthermore, when a user places a cursor on a predetermined position of a tooth displayed in two dimensions, the data processing device 10 may highlight the predetermined position of the tooth displayed in three dimensions.

[0154] For example, when the data processing device 10 receives an input from the input device 30 to add an annotation "caries" to the buccal surface of a second premolar shown in the two-dimensional image, the data processing device 10 also adds the annotation "caries" to the buccal surface of the second premolar shown in the three-dimensional image. Furthermore, when the data processing device 10 receives an input from the input device 30 to place a cursor on the buccal surface of the second premolar shown in the two-dimensional image, the data processing device 10 also places the cursor on the buccal surface of the second premolar shown in the three-dimensional image and enlarges the buccal surface of the second premolar to display the annotation "caries," thereby emphasizing the buccal surface of the second premolar where caries has occurred. In the example of FIG. 35, the data processing device 10 displays on the display 20 an enlarged image of the buccal surface of the second premolar as viewed from the direction of the buccal surface of the second premolar.

[0155] 35 is an example, and the data processing device 10 may highlight the area where the cursor is placed using other methods. For example, when the data processing device 10 receives an input from the input device 30 to place the cursor on the buccal surface of a second premolar shown in the two-dimensional image, the data processing device 10 may display an image of the buccal surface of the second premolar shown in the three-dimensional image, cropped out from the buccal surface of the second premolar shown in the three-dimensional image, in a predetermined display area. Furthermore, when the cursor is placed on a specific tooth area in one of the two-dimensional image and the three-dimensional image, the data processing device 10 may highlight the specific tooth area in the other image using various methods, such as changing the color of the specific tooth area, blinking the specific tooth area, enlarging the specific tooth area, or reducing the specific tooth area.

[0156] The user may also use the QLF (Quantitative Light-induced Fluorescence) method, which utilizes the autofluorescence of teeth, to scan the teeth with the three-dimensional scanner 2 while irradiating the teeth with visible light. The user may also use the three-dimensional scanner 2 to scan teeth discolored by a stain. Furthermore, the user may also use the three-dimensional scanner 2 to scan teeth discolored by a sheet of articulating paper with a red or blue paint transferred onto its surface.

[0157] As described above, the data processing device 10 may determine at least one of the presence or absence of an abnormality (e.g., plaque) and the degree of abnormality in the colored tooth portion based on three-dimensional data of the tooth colored with visible light such as blue or a dye. For example, the estimation model 50 of the data processing device 10 is trained by machine learning so as to be able to determine at least one of the presence or absence of an abnormality and the degree of abnormality in the colored tooth portion based on the three-dimensional data of the colored tooth. The data processing device 10 determines at least one of the presence or absence of an abnormality and the degree of abnormality in the colored tooth portion by directly inputting the three-dimensional data of the colored tooth into the neural network 51 of the estimation model 50. Furthermore, the data processing device 10 may display on the display 20 an image showing the colored tooth portion (e.g., the enlarged image shown in FIG. 35) and an image showing at least one of the determination results of the presence or absence of an abnormality and the degree of abnormality (e.g., the "caries" image shown in FIG. 35) in association with each other. Furthermore, when the data processing device 10 determines the presence or absence of plaque as an abnormality in a tooth region, it may display an image indicating the presence or absence of plaque, such as that shown in FIG. 31, on the display 20. Note that the determination of the presence or absence of an abnormality includes, for example, the determination of the presence or absence of a lesion such as caries, the determination of the presence or absence of tartar, and the determination of the presence or absence of plaque, which can be performed based on three-dimensional data of the tooth colored with visible light such as blue or a dye. Note that the determination of the degree of abnormality also includes, for example, the determination of the degree of lesion such as CO or C1, which indicates the degree of caries, and the determination of the degree of tartar, which can be performed based on three-dimensional data of the tooth colored with visible light such as blue or a dye. Note that the presence or absence of an abnormality in a tooth region and the degree of abnormality can also be determined by the fourth estimation model 50D that identifies the lesion region. For example, when the data processing device 10 determines the presence or absence of plaque, as with dental caries, the fourth neural network 51D included in the fourth estimation model 50D can determine the presence or absence of plaque primarily based on the shape characteristics of the area of ​​the tooth where plaque has developed.In this case, the fourth neural network 51D can determine the presence or absence of plaque with higher accuracy by including features such as the actual color or light intensity of the tooth region in the input data. By using the fourth estimation model 50D including the fourth neural network 51D, the data processing device 10 can determine the presence or absence of plaque without coloring the tooth with visible light such as blue or a dye. However, the presence or absence of plaque can be determined with higher accuracy based on three-dimensional data of the tooth colored with visible light such as blue or a dye. Thus, by using three-dimensional data of the tooth colored with visible light such as blue or a dye as input data, the data processing device 10 can determine the presence or absence of an abnormality in the tooth region or the degree of abnormality with higher accuracy. The estimation model 50 determines the presence or absence of an abnormality or the degree of abnormality by taking into account not only the color of the colored tooth but also the shape, actual color, and light intensity of the abnormal region, thereby increasing the accuracy of the determination.

[0158] As described above, the three-dimensional data includes color information indicating the actual color of a portion (the surface portion of the object) corresponding to each point in the point cloud indicating the surface shape of an object in the oral cavity. Therefore, the data processing device 10 may determine the actual color of each of multiple portions based on the three-dimensional data. For example, the estimation model 50 of the data processing device 10 is trained by machine learning to be able to determine the actual color of a tooth based on three-dimensional tooth data including color information. The data processing device 10 determines the actual color of a tooth by directly inputting the three-dimensional tooth data including color information into the neural network 51 of the estimation model 50. Conventionally, when a dentist or other practitioner replaces a patient's missing tooth with a ceramic prosthesis, they visually determine the actual color of the tooth using a predetermined color sample, etc. However, the data processing device 10 can perform such a determination using the estimation model 50. Furthermore, the data processing device 10 may display on the display 20 an image showing the portion of each tooth and an image showing the determination result of the actual color of each tooth in association with each other. Furthermore, the data processing device 10 may estimate which color in the color sample corresponds to the actual color of each of the multiple regions based on the three-dimensional data. This allows, for example, a dentist to appropriately select a ceramic prosthesis that matches the color of the patient's tooth when replacing a missing tooth of the patient with a ceramic prosthesis.

[0159] As described above, the data processing device 10 can identify the type of each tooth included in the dentition and the surface location of each tooth by using the estimation model 50. Therefore, the data processing device 10 may generate a triangle by connecting the distobuccal cusp of the left second molar, the distobuccal cusp of the right second molar, and the midpoints (incisal points) of the approximal surfaces of the left and right central incisors in the mandibular dentition, and may regard the triangle as an occlusal plane and output image data for displaying the occlusal plane to the display 20. In this way, the data processing device 10 can display on the display 20 a virtual occlusal plane generated by the positions of the cusps of the left first molar, the cusps of the right first molar, and the central incisors identified using the estimation model 50.

[0160] As described above, the data processing device 10 can automatically recognize the type of tooth, tooth location, processed location, diseased location, periodontal pocket depth, and early contact location, etc., simply by acquiring three-dimensional data of the teeth obtained by the three-dimensional scanner 2 or a CT imaging device, and reflect this information in the electronic medical record, thereby improving convenience for users such as surgeons and also increasing the accuracy of dental examinations.

[0161] The embodiments disclosed herein should be considered to be illustrative in all respects and not restrictive. The scope of the present disclosure is defined by the claims, not by the above description, and is intended to include all modifications within the meaning and scope of the claims. Note that the configurations exemplified in the present embodiment and the configurations exemplified in the modified examples can be combined as appropriate. [Explanation of symbols]

[0162] 1 data processing system, 2 three-dimensional scanner, 10 data processing device, 11 arithmetic unit, 12 memory, 13 storage device, 14 scanner interface, 15 display interface, 16 input device interface, 17 storage medium interface, 18 communication device, 20 display, 30 input device, 31 keyboard, 32 mouse, 40 removable disk, 50 estimation model, 50A first estimation model, 50B second estimation model, 50C third estimation model, 50D fourth estimation model, 50E fifth estimation model, 51 neural network, 51A first neural network, 51B second neural network, 51C third neural network, 51D fourth neural network, 51E fifth neural network, 52 parameters, 100 data processing program, 101 input unit, 102 calculation unit, 103 output unit.

Claims

1. A data processing device for processing three-dimensional data, an input unit to which the three-dimensional data including three-dimensional position information corresponding to each of a plurality of points indicating the surface shape of an object in the oral cavity including teeth is input; a calculation unit that executes a process for identifying each of a plurality of regions of the tooth based on the three-dimensional data input from the input unit; an output unit that outputs a processing result by the calculation unit.

2. When the tooth is a first molar, a second molar, a third molar, a first premolar, or a second premolar of the maxilla, the plurality of sites include at least one of an occlusal surface, a distal surface, a mesial surface, a palatal surface, and a buccal surface of the tooth; When the tooth is a canine, a lateral incisor, or a central incisor of the maxilla, the plurality of sites include at least one of a distal surface, a mesial surface, a palatal surface, and a labial surface of the tooth; When the tooth is a first molar, a second molar, a third molar, a first premolar, or a second premolar of a mandible, the plurality of sites include at least one of an occlusal surface, a distal surface, a mesial surface, a lingual surface, and a buccal surface of the tooth; The data processing device according to claim 1 , wherein when the tooth is a canine, a lateral incisor, or a central incisor of the mandible, the plurality of sites include at least one of a distal surface, a mesial surface, a lingual surface, and a labial surface of the tooth.

3. 2. The data processing device according to claim 1, wherein when the tooth is a first molar, a second molar, a third molar, a first premolar, or a second premolar, the plurality of regions includes each of a plurality of cusps constituting the tooth.

4. The data processing device according to claim 1 , wherein when the tooth is a canine, a lateral incisor, or a central incisor, the plurality of regions includes a plurality of surfaces that form a labial surface of the tooth.

5. The data processing device according to claim 1 , wherein the calculation unit identifies whether each of the plurality of regions is a region of the upper jaw teeth or a region of the lower jaw teeth.

6. The data processing device according to any one of claims 1 to 5, wherein the calculation unit identifies the type of tooth based on the three-dimensional data input from the input unit and a first estimation model on which machine learning has been performed.

7. The data processing device according to claim 6 , wherein the first estimation model includes a first neural network that has been trained by machine learning to identify the type of tooth by inputting the three-dimensional data.

8. The data processing device according to any one of claims 1 to 5, wherein the calculation unit identifies each of the plurality of parts based on the three-dimensional data input from the input unit and a second estimation model on which machine learning has been performed.

9. The data processing device according to claim 8 , wherein the second estimation model includes a second neural network that has been trained by machine learning to identify each of the plurality of body parts by inputting the three-dimensional data.

10. The data processing device according to any one of claims 1 to 5, wherein the calculation unit identifies the processed area of ​​the tooth based on the three-dimensional data input from the input unit and a third estimation model that has undergone machine learning.

11. The data processing device according to claim 10 , wherein the third estimation model includes a third neural network that has been trained by machine learning to identify the machined portion by inputting the three-dimensional data.

12. The data processing device according to any one of claims 1 to 5, wherein the calculation unit identifies a lesion site in the tooth or gum based on the three-dimensional data input from the input unit and a fourth estimation model that has undergone machine learning.

13. The data processing device according to claim 12 , wherein the fourth estimation model includes a fourth neural network that has been trained by machine learning to identify the lesion site by inputting the three-dimensional data.

14. A data processing device described in any one of claims 1 to 5, wherein the calculation unit estimates the depth of the periodontal pocket in the tooth based on the three-dimensional data input from the input unit and a fifth estimation model that has undergone machine learning.

15. The data processing device according to claim 14 , wherein the fifth estimation model includes a fifth neural network that has been machine-trained to estimate the depth of the periodontal pocket by inputting the three-dimensional data.

16. 6. The data processing device according to claim 1, wherein the calculation unit estimates an early contact position where early contact occurs when the upper and lower teeth occlude, based on the three-dimensional data input from the input unit.

17. 6. A data processing device according to claim 1, wherein the calculation unit identifies each of the plurality of regions based on the plurality of regions divided by a plurality of planes along the tooth axis passing through the center of gravity of the tooth and the center of gravity of the dental arch including the tooth.

18. The data processing device according to claim 17 , wherein the calculation unit identifies the occlusal surface of the tooth using at least one of a circular cylinder, an elliptical cylinder, and a rectangular cylinder having the tooth axis as a central axis.

19. 6. The data processing device according to claim 1, wherein the output unit outputs image data for displaying a diagram or a table showing the processing results.

20. The data processing device according to claim 19, wherein the processing results are used in an electronic medical record.

21. 20. The data processing device of claim 19, wherein the image data includes data for displaying the tooth together with the processing results in at least one of two and three dimensions.

22. the image data includes data for displaying the tooth in two and three dimensions; 22. The data processing device of claim 21, wherein when an annotation is added by a user to a predetermined position of the tooth displayed in three dimensions, the annotation is added to the predetermined position of the tooth displayed in two dimensions.

23. the image data includes data for displaying the tooth in two and three dimensions; 22. The data processing device of claim 21, wherein when an annotation is added by a user to a predetermined position of the tooth displayed in two dimensions, the annotation is added to the predetermined position of the tooth displayed in three dimensions.

24. the image data includes data for displaying the tooth in two and three dimensions; 22. The data processing device according to claim 21, wherein when a user places a cursor on a predetermined position of the tooth displayed in three dimensions, the predetermined position of the tooth displayed in two dimensions is highlighted.

25. the image data includes data for displaying the tooth in two and three dimensions; 22. The data processing device according to claim 21, wherein when a user places a cursor on a predetermined position of the tooth displayed in two dimensions, the predetermined position of the tooth displayed in three dimensions is highlighted.

26. 6. The data processing device according to claim 1, wherein the processing result includes color information associated with each of the plurality of body parts.

27. The calculation unit determining at least one of the presence or absence of an abnormality and the degree of abnormality of the colored tooth portion based on the three-dimensional data input from the input unit; 6. The data processing device according to claim 1, wherein the colored tooth regions are associated with at least one of the determination results of the presence or absence of abnormality and the degree of abnormality based on the processing results.

28. The calculation unit determining the actual color or the color of a color sample for each of the plurality of regions based on the three-dimensional data input from the input unit; 6. The data processing device according to claim 1, wherein each of the plurality of parts is associated with the actual color or the color judgment result in the color sample based on the processing result.

29. the processing results include positions of the distal buccal cusp of the mandibular left second molar, the distal buccal cusp of the mandibular right second molar, and the proximal surfaces of the mandibular left and right central incisors; 6. A data processing device according to claim 1, wherein the output unit outputs image data for displaying an occlusal plane generated based on the distal buccal cusp of the second molar on the left side of the mandible, the distal buccal cusp of the second molar on the right side of the mandible, and the midpoint of the proximal surfaces of the central incisors on the left and right sides of the mandible.

30. the three-dimensional data is three-dimensional scanner data obtained by scanning the teeth using a three-dimensional scanner, 6. The data processing device according to claim 1, wherein the output unit outputs the processing results obtained during scanning by the three-dimensional scanner in real time.

31. 6. The data processing device according to claim 1, wherein the three-dimensional data includes at least one of three-dimensional scanner data obtained by scanning the teeth using a three-dimensional scanner, CBCT data obtained by imaging the teeth using a cone beam CT, and OCT data obtained by imaging the teeth using an optical coherence tomography.

32. A data processing method for processing three-dimensional data by a computer, comprising: The process executed by the computer is acquiring the three-dimensional data including three-dimensional position information corresponding to each of a plurality of points indicating a surface shape of an object in the oral cavity including teeth; a step of executing a process for identifying each of a plurality of regions of the tooth based on the three-dimensional data acquired by the acquiring step; and outputting a processing result from the step of executing the processing.

33. A data processing program for processing three-dimensional data by a computer, The computer, acquiring the three-dimensional data including three-dimensional position information corresponding to each of a plurality of points indicating a surface shape of an object in the oral cavity including teeth; a step of executing a process for identifying each of a plurality of regions of the tooth based on the three-dimensional data acquired by the acquiring step; and a step of outputting a processing result obtained by the step of executing the processing.

Images