Data generation program, data generation method, data generation device, and x-ray imaging device

Abstract

This data generation program causes a calculation device to execute: a display step for displaying, on a display, an oral cavity image showing, in three dimensions, an oral cavity including a tooth; an image superimposition step for superimposing a virtual surface on the oral cavity image on the basis of user input or machine learning; and an image change step for changing the oral cavity image according to the distance between the virtual surface and the occlusal surface of the tooth indicated in the oral cavity image.

Description

Data generation program, data generation method, data generation device, and X-ray imaging device

[0001] The present disclosure relates to a data generation program, a data generation method, a data generation device, and an X-ray imaging device for generating image data for checking the state of teeth.

[0002] It is important for patients to achieve and maintain an ideal dental occlusion. The occlusion is typically confirmed using articulating paper or a dental model, or by using digital data acquired by an intraoral scanner (IOS). Patent Document 1 discloses a dental occlusion pressure measuring device that includes a sensor sheet to which a marking material is applied and which is occluded by the subject, and a computer that processes data indicating pressure detection points of the sensor sheet during occlusion. The dental occlusion pressure measuring device can detect early contact positions by detecting tooth contact positions and occlusion pressure using the sensor sheet.

[0003] Japanese Patent Application Laid-Open No. 2005-279094

[0004] According to the dental occlusion pressure measuring device disclosed in Patent Document 1, the early contact position can be detected using a sensor sheet, but the tooth contact position and occlusion pressure may vary depending on the degree of occlusion of the sensor sheet by the subject, and there is a risk that the occlusion state cannot be confirmed with high accuracy. Furthermore, the above-mentioned conventional confirmation method is limited to showing the patient's current occlusion state, and the surgeon cannot confirm the early contact position of each tooth with high accuracy.

[0005] The present disclosure has been made to solve such problems, and aims to provide a technique that can check the occlusion state of teeth with high accuracy.

[0006] According to one example of the present disclosure, there is provided a data generation program for generating image data for checking the condition of teeth. The data generation program causes a computer to execute a display step of displaying an intraoral cavity image showing a three-dimensional image of an intraoral cavity including teeth on a display, an image superimposing step of superimposing a virtual surface on the intraoral cavity image based on a user input or machine learning, and an image modifying step of modifying the intraoral cavity image according to the distance between the virtual surface and the occlusal surfaces of the teeth shown in the intraoral cavity image.

[0007] According to one example of the present disclosure, there is provided a data generation method for generating image data for checking the condition of teeth, the data generation method including, as processing executed by a computer, a display step of displaying on a display an intraoral image showing a three-dimensional image of an intraoral cavity including teeth, an image superimposing step of superimposing a virtual surface on the intraoral image based on user input or machine learning, and an image modifying step of modifying the intraoral image according to the distance between the virtual surface and the occlusal surfaces of the teeth shown in the intraoral image.

[0008] According to one example of the present disclosure, there is provided a data generating device that generates image data for checking the condition of teeth. The data generating device includes a display that displays an image, an input device that accepts user input, and a control device that controls the display based on the user input. The control device displays an intraoral image that shows a three-dimensional image of an intraoral cavity including teeth on the display, and based on the user input or machine learning, superimposes a virtual surface on the intraoral image and changes the intraoral image according to the distance between the virtual surface and the occlusal surfaces of the teeth shown in the intraoral image.

[0009] According to one example of the present disclosure, there is provided an X-ray imaging device that generates image data for checking the condition of teeth. The X-ray imaging device includes an imaging unit that images an oral cavity including teeth, a display that displays the image, an input device that accepts user input, and a control device that controls the display based on the user input. The control device displays an oral cavity image that shows a three-dimensional image of the oral cavity captured by the imaging unit on the display, and superimposes a virtual surface on the oral cavity image based on the user input or machine learning, and changes the oral cavity image according to the distance between the virtual surface and the occlusal surfaces of the teeth shown in the oral cavity image.

[0010] According to the present disclosure, a user can superimpose a virtual surface on an oral cavity image displayed on a display and change the oral cavity image according to the distance between the virtual surface and the occlusal surfaces of the teeth shown in the oral cavity image, thereby enabling the user to check the occlusion state of the teeth with high accuracy.

[0011] 1 is a diagram illustrating an application example of a data generating device according to embodiment 1. FIG. 2 is a block diagram illustrating the configuration of a data generating device according to embodiment 1. FIG. 3 is a diagram illustrating an example of a virtual plane set in an oral cavity image. FIG. 4 is a diagram illustrating an example of a virtual plane set in an oral cavity image. FIG. 5 is a diagram illustrating an example of calculation of Bulk-Will angle. FIG. 6 is a diagram illustrating an example of a change in a heat map in an oral cavity image with movement of a virtual plane. FIG. 7 is a diagram illustrating an example of a change in a heat map in an oral cavity image with movement of a virtual plane. FIG. 8 is a diagram illustrating an example of an upper jaw dentition and a lower jaw dentition with heat maps attached. FIG. 9 is a flowchart illustrating an example of a data generation process performed by the data generating device according to embodiment 1. FIG. 10 is a flowchart illustrating an example of a Bulk-Will angle determination process performed by the data generating device according to embodiment 1. FIG. 11 is a diagram illustrating an example of a Monson sphere set in an oral cavity image. FIG. 12 is a diagram illustrating an example of a change in a heat map in an oral cavity image with movement of a virtual curved surface. FIG. 13 is a diagram illustrating an example of a change in a heat map in an oral cavity image with movement of a virtual curved surface.

[0012] First Embodiment A first embodiment of the present disclosure will be described in detail with reference to the drawings. In the drawings, the same or corresponding parts are designated by the same reference numerals and description thereof will not be repeated.

[0013] [Application Example] An application example of the data generating device 1 according to the first embodiment will be described with reference to FIG. 1 . FIG. 1 is a diagram illustrating an application example of the data generating device 1 according to the first embodiment. In a preparation stage, a user of the data generating device 1 scans the oral cavity of a subject using a three-dimensional scanner (optical scanner) (not shown) to acquire three-dimensional data (optical scanner data) including position information of each point of a point cloud (multiple points) indicating the surface of biological tissue, including the teeth and gums, in the oral cavity. The three-dimensional data includes, as position information, coordinates (X, Y, Z) of each point indicating the surface of the biological tissue in predetermined X-axis, Y-axis, and Z-axis directions. Note that the directions in the above-mentioned position information may be, for example, the X-axis direction being the horizontal direction (e.g., the left-right direction of the head), the Y-axis direction being the vertical direction (e.g., the front-back direction of the head), and the Z-axis direction being the height direction (e.g., the extension direction of the body axis of the head of a standing human body). For example, if the center of the head is the reference point of the coordinate system (e.g., the point where the numerical value is zero), the right of the reference point may be set as the +X direction (positive direction) of the X axis, the left of the reference point as the -X direction (negative direction) of the X axis, the front of the reference point as the +Y direction (positive direction) of the Y axis, the rear of the reference point as the -Y direction (negative direction) of the Y axis, the top of the reference point as the +Z direction (positive direction) of the Z axis, and the bottom of the reference point as the -Z direction (negative direction) of the Z axis. On each of the X, Y, and Z axes, the direction from the smaller numerical value to the larger numerical value is the + direction, and the direction from the larger numerical value to the smaller numerical value is the - direction. For example, on the X axis, the direction from the -X side to the +X side is the +X direction, and the direction from the +X side to the -X side is the -X direction. Furthermore, the three-dimensional data may include color information indicating the actual color of the portion (surface portion of the biological tissue) corresponding to each point of a point cloud (multiple points) representing the surface of the biological tissue, including the teeth and gums, in the oral cavity. Furthermore, with regard to the "tooth axis" described below, the teeth on the upper jaw are considered to be the "opposite teeth" of the teeth on the lower jaw, and the teeth on the lower jaw are considered to be the "opposite teeth" of the teeth on the upper jaw. The side of the tooth axis that is on the "opposite tooth" side is also referred to as the "upper tooth axis," and the side of the tooth axis that is farther from the "opposite tooth" is also referred to as the "lower tooth axis."

[0014] "User" includes practitioners (such as doctors) or assistants (such as dental assistants, dental technicians, nurses) in various fields, such as dentistry, oral surgery, orthopedic surgery, plastic surgery, and cosmetic surgery. "Subject" includes patients of dentistry, oral surgery, orthopedic surgery, plastic surgery, and cosmetic surgery. In the following, the expression "a user does ..." includes a user issuing an execution command and causing a device or program to execute or operate a process.

[0015] A three-dimensional scanner is an intraoral scanner that can optically capture images of a subject's oral cavity using a confocal method, triangulation, or the like, and can acquire positional information for each point in a point cloud constituting the surface of the biological tissue (e.g., the teeth and gums in the oral cavity) that is the subject of the scan, placed in a coordinate space. Optically capturing and scanning the oral cavity is also referred to as an intraoral scan (IOS). A user can use the three-dimensional data acquired by the three-dimensional scanner to generate a rendered image (appearance image) showing the three-dimensional shape of the biological tissue. A "rendered image" is an image generated by processing or editing certain data. For example, a user can process or edit the three-dimensional data of the biological tissue acquired by the three-dimensional scanner to generate a rendered image showing the biological tissue (a portion of the biological tissue that can be shown by the IOS data) from a specific viewpoint. Furthermore, a user can generate multiple rendered images showing the biological tissue from multiple directions by changing the specific viewpoint in multiple directions. A rendered image may be a two-dimensional image. For example, the two-dimensional image may be an image obtained by projecting image data onto a projection plane on a coordinate system that intersects with the line of sight. For example, the rendering image may be generated as a three-dimensional image that can be viewed stereoscopically with a viewer such as general-purpose VR goggles.

[0016] A user can also acquire three-dimensional volume (voxel) data of hard tissues (bones, teeth, etc.) around the subject's upper and lower jaws by using a computed tomography (CT) imaging device (not shown). The CT imaging device is an "X-ray imaging device" that performs CT imaging of the subject's upper and lower jaws by rotating an X-ray transmitter and receiver, a type of radiation, around the subject's face. Using the volume data of the biological tissue being imaged acquired by the CT imaging device, a user can generate a rendering image (a tomographic image or an external image) showing the three-dimensional shape of the biological tissue. For example, by processing or editing the volume data of the biological tissue acquired by the CT imaging device, a user can generate a rendering image showing the biological tissue (a portion of the biological tissue that can be shown by CT data) from a specific viewpoint. Furthermore, a user can generate multiple rendering images showing the biological tissue from multiple directions by changing the specific viewpoint in multiple directions. As with IOS data, the rendered image may be a two-dimensional or three-dimensional image.

[0017] Hereinafter, three-dimensional data including positional information of each point in a point cloud representing the surface of biological tissue acquired by a three-dimensional scanner will be referred to as "IOS data," and a rendering image generated based on the IOS data will be referred to as "IOS image." Furthermore, three-dimensional volume data acquired by a CT imaging device will be referred to as "CT data," and a rendering image generated based on the CT data will be referred to as "CT image." IOS images can show the surface shape of the biological tissue being scanned in great detail, but cannot show internal structures (such as alveolar bone and root apex) that do not appear on the surface of the biological tissue. CT images can show hard tissue portions (such as bones and teeth) of the subject in relatively high detail, but cannot show soft tissue portions (such as skin and gums) in as much detail as hard tissue portions.

[0018] A user can generate oral image data by combining IOS data and CT data acquired for the same subject. Here, the IOS data and CT data have different data formats. For example, the user may convert the data format of the IOS data to that of CT data and then use the converted data to pattern match the three-dimensional shape of the surface of the biological tissue to generate oral image data by combining the IOS data and CT data. Alternatively, the user may convert the data format of the CT data to that of IOS data and then use the converted data to pattern match the three-dimensional shape of the surface of the biological tissue to generate oral image data. Alternatively, the user may convert the data formats of the CT data and the IOS data into a common data format and then use the converted data to pattern match the three-dimensional shape of the surface of the biological tissue to generate oral image data. The data format to be processed may be determined by the program.

[0019] By processing or editing the oral cavity image data, a user can generate a rendering image (e.g., the oral cavity image shown in FIG. 1 ) showing two-dimensional biological tissue (portions of biological tissue that can be represented by both IOS data and CT data) from a predetermined viewpoint. As shown in FIG. 1 , the oral cavity image can show, in three dimensions, the surface shape of the biological tissue represented by the IOS data and the cross-sectional structure or appearance of hard tissue portions (bones, teeth, etc.) represented by the CT data. When generating the oral cavity image, the user may adjust the brightness, contrast, transparency, etc. of each of the IOS data and CT data as needed.

[0020] The data generating device 1 may acquire IOS data from a 3D scanner and CT data from a CT scanner, and generate an oral cavity image using the acquired IOS data and CT data based on a user input. Alternatively, the data generating device 1 may acquire oral cavity image data generated by a user using another device from the other device without acquiring the IOS data and CT data.

[0021] In oral cavity images, the CT data shows the three-dimensional shapes of hard tissue parts such as the alveolar bone and root apex, and the IOS data can show the three-dimensional shapes of soft tissue parts such as the gums that cannot be shown by CT data. As a result, oral cavity images can show in detail the soft tissue parts such as the gums that cannot be shown by CT data alone, along with the hard tissue parts such as the alveolar bone and root apex, by supplementing them with the IOS data.

[0022] As shown in FIG. 1 , the data generating device 1 displays an image of an oral cavity, including teeth, on a display 20. The data generating device 1 processes the image of an oral cavity displayed on the display 20 based on user input, thereby displaying images of the oral cavity viewed from various viewpoints on the display 20. Furthermore, the data generating device 1 can add a heat map to the image of the oral cavity displayed on the display 20, indicating the position of early contact for at least one occlusal surface of the maxillary dentition and the mandibular dentition. This allows the user to check the occlusal state of each tooth by checking the occlusal surface displayed on the heat map. Here, a "heat map" refers to an image that indicates the difference between values ​​at one location and values ​​at another location by the degree of color change. For example, the heat map may be processed to gradually change the image from large values ​​to small values ​​and from warm colors to cool colors depending on the distance between the virtual plane and the occlusal surface. The heat map may also be processed to gradually change the image from large values ​​to small values ​​and from cool colors to warm colors depending on the distance between the virtual plane and the occlusal surface.

[0023] In a narrow sense, the term "occlusal surface" refers to the surface where opposing molar teeth contact each other when the upper and lower jaw teeth are occluded, and where food is crushed and ground when eaten. In the embodiment, this occlusal surface is referred to as the "occlusal surface OS." The region to which a heat map is applied may be the occlusal surface OS, but as shown in FIG. 5 (described later), it may also include not only the occlusal surface OS but also the region EO adjacent to the occlusal surface OS. Such a region of the tooth surface that is the target region for the heat map is also referred to as the "occlusal surface region OA" (see FIG. 5 (described later)). Of course, because occlusal areas in areas of teeth other than molars, such as the anterior teeth, also exist, these regions may be included in the region covered by the occlusal surface region OA. When the term "occlusal surface" in the embodiment refers to the region to which a heat map is applied, the term "occlusal surface" may refer to the occlusal surface region OA. Hereinafter, the "occlusal surface" meaning the occlusal surface region OA will also be referred to as the "occlusal surface OA."

[0024] Note that the data generating device 1 is not limited to generating oral cavity images based on data obtained by combining IOS data and CT data. For example, the data generating device 1 may generate oral cavity images based only on IOS data. An oral cavity image generated based only on IOS data can at least show the surface shapes of the teeth and gums in the oral cavity. The data generating device 1 may also generate oral cavity images based only on CT data. An oral cavity image generated based only on CT data can at least show the shapes of hard tissues such as teeth and bones. Even when generating oral cavity images based only on CT data, it is possible to form images of the tooth surfaces, and this surface shape data may be used instead of oral cavity images based on IOS data.

[0025] Whether generating an oral cavity image using IOS data or CT data, the data generating device 1 may perform segmentation for each anatomical element shown in the oral cavity image. For example, the segmentation may be various, such as segmentation of hard tissues and soft tissues, segmentation of maxillary hard tissues and mandibular hard tissues, segmentation of dentition and alveolar bone, segmentation of maxillary and mandibular dentition, or segmentation of multiple teeth. For example, based on user input, the data generating device 1 may recognize the regions of the maxillary dentition, mandibular dentition, each tooth included in the maxillary dentition, and each tooth included in the mandibular dentition separated by the user, and segment these regions to display them in the oral cavity image.

[0026] Alternatively, the data generating device 1 may automatically segment each part using AI (artificial intelligence) technology or the like, without following user input. For example, the data generating device 1 may include an estimation model including a neural network for identifying the upper jaw dentition, the lower jaw dentition, and each of the multiple teeth shown in the oral cavity image based on the oral cavity image data. The estimation model is trained by machine learning using training data that includes a set of multiple oral cavity images and ground truth data indicating the regions of each segmented part shown in each of the multiple oral cavity images, so as to segment the upper jaw dentition, the lower jaw dentition, and each of the multiple teeth shown in the oral cavity image based on the input oral cavity image data. Using such an estimation model, a user can input oral cavity image data into the data generating device 1 to obtain oral cavity images showing the upper jaw dentition, the lower jaw dentition, and each of the multiple teeth segmented by the estimation model of the control device 10.

[0027] [Configuration of Data Generating Device] The configuration of the data generating device 1 according to the first embodiment will be described with reference to Fig. 2. Fig. 2 is a block diagram showing the configuration of the data generating device 1 according to the first embodiment. The data generating device 1 may be realized by, for example, a general-purpose computer or a dedicated computer.

[0028] As shown in FIG. 2, the data generating device 1 includes a control device 10, a display 20, and an input device 30.

[0029] The control device 10 includes a calculation device 11 , a memory 12 , a storage device 13 , a display interface 14 , an input device interface 15 , a storage medium interface 16 , and a communication device 17 .

[0030] The arithmetic device 11 is a computing entity (computer) that executes various processes by executing various programs. The arithmetic device 11 is configured with a processor such as a CPU (Central Processing Unit), an MPU (Micro Processing Unit), a TPU (Tensor Processing Unit), or a GPU (Graphics Processing Unit). A processor, which is an example of the arithmetic device 11, has the function of executing various processes by executing programs. However, some or all of these functions may be implemented using dedicated hardware circuits such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array). The term "processor" is not limited to processors in the narrow sense that execute processes using stored programs, such as a CPU, MPU, TPU, or GPU, but may also include hardwired circuits such as an ASIC or FPGA. Furthermore, the arithmetic device 11 is not limited to von Neumann computers such as a CPU or GPU, but may also be configured with non-von Neumann computers such as a quantum computer or an optical computer. The arithmetic device 11 described above may also be interpreted as a processing circuit that executes a predetermined process. The computing device 11 may be configured as a single chip or multiple chips. Furthermore, the processor and related processing circuits may be configured as multiple computers interconnected by wire or wirelessly via a local area network, a wireless network, or the like. The processor and related processing circuits may be configured as a cloud computer that performs remote calculations based on input data and outputs the calculation results to another device located at a distance. The execution of various programs by the computing device 11 is equivalent to the various programs causing the computing device 11 to perform various processes.

[0031] The memory 12 includes a volatile storage area (for example, a working area) that temporarily stores program code or work memory when the arithmetic unit 11 executes various programs. Examples of the memory 12 include volatile memory such as DRAM (Dynamic Random Access Memory) or SRAM (Static Random Access Memory), or non-volatile memory such as ROM (Read Only Memory) or flash memory.

[0032] The storage device 13 stores various programs and various data executed by the arithmetic device 11. The storage device 13 may be one or more non-transitory computer readable media, or may be one or more computer readable storage media. Examples of the storage device 13 include a hard disk drive (HDD) and a solid state drive (SSD).

[0033] The storage device 13 stores a data generation program 100. The data generation program 100 describes the contents of a data generation process for the arithmetic device 11 to generate oral cavity image data for checking the occlusion state of teeth.

[0034] The display interface 14 is an interface for connecting to the display 20. The display interface 14 realizes input and output of data between the control device 10 and the display 20. For example, the control device 10 causes the display 20 to display an oral cavity image based on oral cavity image data via the display interface 14.

[0035] The input device interface 15 is an interface for connecting an input device 30 such as a keyboard 31 and a mouse 32. The input device interface 15 realizes input and output of data between the control device 10 and the input device 30. For example, a user can input desired commands via the input device interface 15 using the input device 30 such as the keyboard 31 and the mouse 32, and can cause the control device 10 to generate and edit oral cavity image data based on the commands. Note that input by the user to the control device 10 using the input device 30, or data (commands) input by the user to the control device 10, is also referred to as "user input."

[0036] The storage medium interface 16 reads various data stored in a storage medium 40 such as a removable disk and writes various data to the storage medium 40. For example, the data generation device 1 may obtain the data generation program 100 from the storage medium 40 via the storage medium interface 16, or may write oral cavity image data for checking the occlusion state of teeth to the storage medium 40 via the storage medium interface 16. The storage medium 40 may be one or more non-transitory computer-readable media, or one or more computer-readable storage media. The control device 10 may obtain IOS data, CT data, or data combining IOS data and CT data from the storage medium 40 via the storage medium interface 16.

[0037] The communication device 17 transmits and receives data to and from an external device via wired or wireless communication. For example, the control device 10 may receive the data generation program 100 from the external device via the communication device 17, or may transmit oral cavity image data for checking the occlusion state of teeth to the external device via the communication device 17. The control device 10 may also obtain IOS data, CT data, or data combining IOS data and CT data from the external device via the communication device 17.

[0038] [Example of Generation of Intraoral Image with Heat Map] An example of generation of an intraoral image with a heat map by the data generating device 1 according to embodiment 1 will be described with reference to Figures 3 to 7. Note that the processing of the data generating device 1 described below is realized by the control device 10 (arithmetic unit 11) of the data generating device 1 executing a data generating program 100. Figures 3 and 4 are diagrams for explaining an example of a virtual plane set on an intraoral image.

[0039] As shown in FIG. 3 , a user displays an oral cavity image on the display 20. The oral cavity image displayed on the display 20 shows hard tissue portions such as teeth and bones. The oral cavity image may be an image showing hard tissue portions consisting of at least teeth. The oral cavity image may be an image showing hard tissue portions consisting of at least a dental arch. The user uses the input device 30 to superimpose a virtual plane on the oral cavity image. The virtual plane is an example of a "virtual surface." For example, by moving or rotating the oral cavity image displayed on the display 20, the user can display an oral cavity image of the subject viewed from a diagonal front perspective as shown in FIG. 3 or an oral cavity image of the subject viewed from a side perspective as shown in FIG. 4 on the display 20.

[0040] While viewing the oral cavity image, the user determines the position of the virtual plane and places it at the determined position, thereby superimposing the virtual plane on the oral cavity image. That is, the user can superimpose the virtual plane on the oral cavity image viewed from a specific viewpoint selected from multiple viewpoints. In the examples of FIGS. 3 and 4 , the user superimposes the virtual plane on the oral cavity image so that it aligns with the occlusal plane OS, where the upper and lower jaw dentitions intersect. That is, the user ideally determines the position of the virtual plane so that it coincides with the occlusal plane OS. At this time, the user can move or rotate the oral cavity image to view the subject from various viewpoints and set the virtual plane at a desired position. Furthermore, the user can change at least one of the color and transparency of the virtual plane.

[0041] In the embodiment, the virtual surface set at the position of the occlusal surface OS is also referred to as a "virtual surface VS." The virtual surface VS may also be referred to as an occlusal position virtual surface OV. When the virtual surface VS is a flat surface, the occlusal position virtual surface OV is an occlusal position virtual plane. When the virtual surface VS is a curved surface, the occlusal position virtual surface OV is an occlusal position virtual curved surface. The occlusal position virtual surface OV may be set by a user's operation or automatically set by image recognition, machine learning, or the like. The occlusal position virtual surface OV is set to be located at each of the occlusal surfaces OS of multiple teeth in the dental arch. Whether the virtual surface VS is a flat surface or a curved surface, it is preferable that the virtual surface VS be a continuous surface without fine irregularities so that overall uniformity is maintained for heat map observation. Hereinafter, the virtual surface representing the occlusal position virtual surface OV may be referred to as "OV." For example, the virtual plane representing the occlusal position virtual surface OV may be referred to as the "virtual plane OV."

[0042] Here, the user can determine the position of the median plane while viewing the oral cavity image and place the median plane at the determined position, thereby superimposing the median plane on the oral cavity image. The median plane is a vertical plane (vertical when upright) that passes through the midline of the human body, which is the subject of the image shown in the oral cavity image, and divides the human body in half when viewed from the front. Furthermore, the user can determine the position of a Camper's plane while viewing the oral cavity image and place the Camper's plane at the determined position, thereby superimposing the Camper's plane on the oral cavity image. The Camper's plane is a plane formed by a line connecting the subnasal point and the tragus point. The Camper's plane may be defined as a plane formed by a line connecting the left and right subala points and the upper edge of the external auditory canal. The user can translate the Camper's plane downward (toward the jaw, for example, in the −Z direction) along the median plane and determine the position that coincides with the occlusal plane OS as the position of a virtual plane (occlusal position virtual plane OV).

[0043] FIG. 5 is a diagram for explaining an example of calculation of the Balk-Will angle. As shown in FIG. 5, the Balk-Will angle is the angle formed by the occlusal plane OS and the Bonwill triangle. Typically, the Balk-Will angle is within the range of 21 degrees to 31 degrees, with the average Balk-Will angle being 26 degrees. The Bonwill triangle is a triangle formed by lines connecting both temporomandibular condyles and the incisors. The Bonwill triangle may also be defined as a triangle formed by lines connecting the vertices of the central upper surfaces of both mandibular condyles and the incisal points.

[0044] After setting the virtual plane OV along the occlusal surface OS, the user further sets the Bonwill triangle described above and determines whether the Bark-Will angle formed by the set occlusal surface and the Bonwill triangle is within a reference range (e.g., 21 to 31 degrees) or whether the Bark-Will angle deviates from a standard angle (e.g., 26 degrees) by a predetermined value or more, thereby determining whether an abnormality has occurred in the subject's jaw or occlusion. The user may also set the virtual plane (occlusal position virtual plane) OV so that the Bark-Will angle is at a standard angle (e.g., 26 degrees). As shown in FIG. 5 , the virtual plane OV is depicted as a triangle with a small interior angle to facilitate understanding of the use of the Bonwill triangle. However, in reality, as shown in FIG. 3 , the virtual plane OV has an area large enough to cover the entire dental arch.

[0045] The virtual surface VS at a displaced position relative to the occlusal position virtual surface OV is also referred to as the displaced virtual surface TS. In the embodiment, the virtual surface representing the displaced virtual surface TS may be abbreviated as "TS." For example, the virtual plane representing the displaced virtual surface TS may be referred to as the "virtual plane TS." The virtual surface at the position of the occlusal position virtual surface OV may be considered to be a virtual plane TS with zero displacement. The direction in which the displaced virtual surface TS displaces relative to the occlusal position virtual surface OV may be a direction perpendicular to the virtual surface, as described above, or in the -Z or +Z direction, as long as the virtual surface is planar. The displacement of the displaced virtual surface TS is a displacement that includes a component along the tooth axis. This displacement is also referred to as the "tooth axial component displacement." The tooth axial component displacement away from the occlusal surface OS is also referred to as the "occlusal separation displacement." The displacement approaching the occlusal surface OS from a position away from the occlusal surface OS is also referred to as the "occlusal approach displacement." The direction including a component along the tooth axis is also referred to as the "tooth axial component direction." It is preferable that the amount of displacement of the tooth axial component is uniform with respect to the occlusal position virtual plane OV at any three points on the virtual displacement plane TS that do not overlap on a straight line. The distance between each point on the surface of the occlusal surface forming area OA and the virtual displacement plane TS is also referred to as the "virtual plane separation distance VD." In the tooth axial component direction, the direction from the occlusal surface OS toward the tooth root may be considered the downward direction, and the direction from the tooth root toward the occlusal surface may be considered the upward direction. If point A is located further downhill than point B, point A may be considered downstream of point B. If point A is located further uphill than point B, point A may be considered upstream of point B. Since the virtual occlusal position plane OV is set at the position of the occlusal surface OS, the occlusal separation displacement and the occlusal approach displacement may be considered as the approach / separation displacement with respect to the virtual occlusal position plane OV.

[0046] 6 and 7 are diagrams illustrating an example of a change in a heat map in an oral cavity image accompanying movement of a virtual plane. A user can move the position of the virtual plane TS in a predetermined direction on the oral cavity image by dragging the virtual plane TS using the input device 30. For example, a user can move the virtual plane TS in a direction perpendicular to the virtual plane TS. As illustrated in FIGS. 6 and 7 , an oral cavity image may display one of the mandibular hard tissue and the maxillary hard tissue, but not the other. When a user views the heat map of the side of interest from above the tooth axis or obliquely above the tooth axis during observation, visibility is improved if the other side is not displayed.

[0047] When the user moves the virtual plane TS, the distance VD between the virtual plane TS and each point on the occlusal surface (occlusal surface forming area) OA changes in accordance with the movement of the virtual plane TS, so that the virtual plane TS approaches each point on the occlusal surface OA of each tooth or moves away from the occlusal surface OS of each tooth. For example, when the virtual plane TS set on the occlusal surface OS moves in the -Z direction (toward the mandible), the virtual plane TS approaches a point on the occlusal surface OA of each mandibular tooth that the virtual plane TS has not passed through, while the virtual plane TS moves away from the occlusal surface OA of each maxillary tooth. Also, when the virtual plane TS set on the occlusal surface OA moves in the +Z direction (toward the maxillary tooth), the virtual plane TS approaches a point on the occlusal surface OA of each maxillary tooth that the virtual plane TS has not passed through, while the virtual plane TS moves away from the occlusal surface OA of each mandibular tooth.

[0048] The data generating device 1 calculates a distance VD between the virtual plane TS, which has been moved based on a user input, and the occlusal surface OA of each tooth of the upper or lower jaw shown in the intraoral image, and changes the intraoral image according to the distance that changes depending on the position of the virtual plane. Specifically, the data generating device 1 calculates the distance VD between each point constituting the occlusal surface OA of each tooth shown in the intraoral image and the virtual plane TS, and adds a heat map corresponding to the distance VD to each point on the occlusal surface OA.

[0049] For example, if the distance between a point constituting the occlusal surface OA of a tooth and the virtual plane TS in a predetermined direction (a direction perpendicular to the virtual plane) is 0 mm, i.e., if the point constituting the occlusal surface OA is on the virtual plane, the data generating device 1 adds a blue color to the point on the occlusal surface that is the subject of distance calculation. If the tooth protrudes from the virtual plane TS in a predetermined direction (a direction perpendicular to the virtual plane, in this case, upward along the tooth axis) and the distance between a point constituting the occlusal surface OA of the tooth and the virtual plane is more than 0 mm and less than 1.0 mm, the data generating device 1 adds a green color to the point on the occlusal surface OA that is the subject of distance calculation. When a tooth protrudes from the imaginary plane TS in a predetermined direction (a direction perpendicular to the imaginary plane, in this case, above the tooth axis), if the distance between a point (protruding point) constituting the occlusal surface OA of the tooth and the imaginary plane TS is 1.0 mm or more and less than 2.0 mm, the data generating device 1 adds yellow to the point (protruding point) on the occlusal surface OA that is the subject of distance calculation.When a tooth protrudes from the imaginary plane TS in a predetermined direction (a direction perpendicular to the imaginary plane TS, in this case, above the tooth axis), if the distance between a point (protruding point) constituting the occlusal surface OA and the imaginary plane TS is 2.0 mm or more and less than 3.0 mm, the data generating device 1 adds red to the point (protruding point) on the occlusal surface that is the subject of distance calculation. When a tooth protrudes from the virtual plane TS in a predetermined direction (a direction perpendicular to the virtual plane TS, in this case, upward on the tooth axis), if the distance between a point (protruding point) constituting the occlusal plane OA and the virtual plane TS is 3.0 mm or more, the data generating device 1 does not add color to the point (protruding point) on the occlusal plane OA that is the subject of distance calculation. Note that the data generating device 1 may display the oral cavity image as a heat map using a color specified by the user based on user input.

[0050] There are various possible distributions of heat map colors, for example, alternative distributions such as the following:

[0051] (Separate Allocation 1) When a tooth protrudes from the imaginary plane TS in a predetermined direction (a direction perpendicular to the imaginary plane TS, in this case, upward along the tooth axis), no color is added. When the distance between a point constituting the occlusal surface OA of the tooth and the imaginary plane TS is 0 mm, red is added to the point on the occlusal surface OA that is the subject of distance calculation. When a tooth is spaced apart from the imaginary plane TS in a predetermined direction (a direction perpendicular to the imaginary plane TS, in this case, downward along the tooth axis), when the distance between a point constituting the occlusal surface OA of the tooth and the imaginary plane TS is more than 0 mm and less than 1.0 mm, yellow is added to the point on the occlusal surface OA that is the subject of distance calculation. When the tooth is spaced apart in a predetermined direction from the imaginary plane TS (a direction perpendicular to the imaginary plane TS, in this case, below the tooth axis), if the distance between the imaginary plane TS and a point constituting the occlusal surface OA of the tooth is 1.0 mm or more and less than 2.0 mm, a green color is added to the point on the occlusal surface OA that is the subject of distance calculation.When the tooth is spaced apart in a predetermined direction from the imaginary plane TS (a direction perpendicular to the imaginary plane TS, in this case, below the tooth axis), if the distance between the imaginary plane TS and a point constituting the occlusal surface OA of the tooth is 2.0 mm or more and less than 3.0 mm, a blue color is added to the point on the occlusal surface OA that is the subject of distance calculation.

[0052] (Separate Allocation 2) When a tooth protrudes from the imaginary plane TS in a predetermined direction (perpendicular to the imaginary plane TS, in this case, above the tooth axis), if the distance between a point (protruding point) constituting the occlusal surface OA of the tooth and the imaginary plane TS is 1.0 mm or more, red is added to the point on the occlusal surface OA that is the subject of distance calculation. When a tooth protrudes from the imaginary plane TS in a predetermined direction (perpendicular to the imaginary plane TS, in this case, above the tooth axis), if the distance between a point (protruding point) constituting the occlusal surface OA of the tooth and the imaginary plane TS is 0 mm or more but less than 1.0 mm, orange is added to the point on the occlusal surface OA that is the subject of distance calculation. When the distance between a point constituting the occlusal surface OA of the tooth and the imaginary plane TS is 0 mm, yellow is added to the point on the occlusal surface OA that is the subject of distance calculation. When the tooth is spaced apart in a predetermined direction (perpendicular to the imaginary plane TS, in this case, below the tooth axis) from the imaginary plane TS, if the distance between the imaginary plane TS and a point constituting the occlusal surface OA of the tooth is greater than 0 mm and less than 1.0 mm, a green color is added to the point on the occlusal surface OA that is the subject of distance calculation.When the tooth is spaced apart in a predetermined direction (perpendicular to the imaginary plane TS, in this case, below the tooth axis) from the imaginary plane TS, if the distance between the imaginary plane TS and a point constituting the occlusal surface OA of the tooth is 1.0 mm or greater, a blue color is added to the point on the occlusal surface OA that is the subject of distance calculation.

[0053] As described above, the data generating device 1 can display the distance between each point constituting the occlusal surface OA of each tooth in the oral cavity image and the virtual plane TS in a heat map, so that the user can easily confirm the early contact position of each tooth.

[0054] For example, when the user moves the virtual plane TS from the position of the virtual plane shown in Fig. 6 toward the mandibular side, the heat map added to each point on the occlusal plane OA changes according to the distance between each point constituting the occlusal plane OA of each mandibular tooth and the virtual plane TS, as shown in Fig. 7. For example, when focusing on the yellow areas in the heat map shown in Fig. 7 and the heat map shown in Fig. 6, the state in Fig. 7 has a larger amount of occlusal separation displacement than the state in Fig. 6, and the area that is equal to or greater than 1.0 mm and less than 2.0 mm has increased, so the colored area is larger.

[0055] In this way, the user can move the virtual plane TS and check which positions on the occlusal surface OA of each tooth come into contact with the virtual plane TS and in what order, thereby enabling the user to check the occlusion state of the teeth with high accuracy.

[0056] Furthermore, based on user input, the data generating device 1 can set a point designated by the user on the occlusal surface OA of the tooth shown in the intraoral image, and add a numerical value corresponding to the distance between the point constituting the occlusal surface OA of the tooth and the virtual plane TS to the set point. For example, as shown in Fig. 7 , when the user moves the cursor using the input device 30 to designate a desired point on the occlusal surface OA of the desired tooth, the data generating device 1 displays a pop-up near the designated point indicating the distance between the point corresponding to the designated point and the virtual plane TS ("1.5 mm" in the example of Fig. 7 ). This allows the user to easily check the occlusion condition at the designated point.

[0057] 8 is a diagram illustrating an example of the maxillary dentition and the mandibular dentition to which heat maps are added. As shown in FIG. 8, the data generating device 1 can display the occlusal surfaces OA of the teeth to which heat maps are added separately for the maxillary dentition and the mandibular dentition on the display 20.

[0058] For example, the data generating device 1 can add a heat map to each of the upper and lower jaw dentitions while they are open, and capture the occlusal surfaces OA of each of the upper and lower jaw dentitions to which the heat map has been added, thereby saving the state of the occlusal surfaces OA of each of the upper and lower jaw dentitions to which the heat map has been added as a still image in the storage device 13.

[0059] [Processing of Data Generating Device] Processing executed by the data generating device 1 according to the first embodiment will be described with reference to Figures 9 and 10. Figure 9 is a flowchart for explaining an example of data generation processing executed by the data generating device 1 according to the first embodiment. Note that each step (hereinafter, indicated by "S") shown in Figures 9 and 10 is realized by the control device 10 (arithmetic unit 11) of the data generating device 1 executing the data generating program 100.

[0060] As shown in FIG. 9 , the data generating device 1 displays an intraoral cavity image showing a three-dimensional image of an intraoral cavity, including teeth, on the display 20 (S1). For example, as illustrated in FIG. 3 , the data generating device 1 displays an intraoral cavity image showing two-dimensional hard tissue portions (bones, teeth, etc.) viewed from a predetermined viewpoint on the display 20. At this time, the data generating device 1 may segment the upper jaw dentition, the lower jaw dentition, and each of the multiple teeth shown in the intraoral cavity image based on user input. Alternatively, the data generating device 1 may use an estimation model trained by machine learning or the like to identify the upper jaw dentition, the lower jaw dentition, and each of the multiple teeth shown in the intraoral cavity image, segment these portions, and display them in the intraoral cavity image. The processing of S1 is an example of a "display step."

[0061] The data generating device 1 superimposes a virtual plane on the oral cavity image displayed on the display 20 based on user input entered using the input device 30 (S2). For example, as illustrated in FIG. 3 , the data generating device 1 superimposes a virtual plane on the oral cavity image based on the user input so that the virtual plane is aligned with the occlusal plane where the maxillary and mandibular dentitions occlude. Based on the user input, the data generating device 1 displays oral cavity images viewed from multiple viewpoints on the display 20 and superimposes a virtual plane (occlusion position virtual plane) OV on the oral cavity image viewed from a specific viewpoint selected from the multiple viewpoints. Furthermore, the data generating device 1 may change at least one of the color and transparency of the virtual plane based on the user input. The process of S2 is an example of an "image superimposition step."

[0062] The data generating device 1 calculates the distance between the virtual plane and the occlusal surface of each tooth (S3). For example, the data generating device 1 calculates the distance between the virtual plane fixed at a predetermined position and each point constituting the occlusal surface of each tooth of the upper or lower jaw shown in the oral cavity image.

[0063] The data generating device 1 adds a heat map to the oral cavity image displayed on the display 20 according to the calculated distance between the virtual surface and the occlusal surface, specifically, the distance between the virtual surface and each point constituting the occlusal surface of each tooth (S4). For example, as illustrated in FIG. 6, the data generating device 1 adds a predetermined color to each point constituting the occlusal surface of each tooth according to the distance between the virtual surface and the point. The data generating device 1 then terminates this process. Note that, as illustrated in FIG. 7, if the virtual plane is moved based on user input, the data generating device 1 can recalculate the distance between the moved virtual plane and each point constituting the occlusal surface of each tooth and add a heat map to the oral cavity image displayed on the display 20 according to the calculated distance, i.e., change the oral cavity image. The processes of S3 and S4 are an example of an "image changing step."

[0064] Preferably, the data generating device 1 first accepts a position setting operation for the occlusion position virtual surface OV as an initial operation in S2, and sets the position of the occlusion position virtual surface OV. As described above, the data generating device 1 may automatically set the position of the occlusion position virtual surface OV. Furthermore, the data generating device 1 accepts a position setting operation for the displacement virtual surface TS in S2, and sets the position of the displacement virtual surface TS. The data generating device 1 calculates the virtual surface separation distance VD in S3. The data generating device 1 adds a heat map in S4. Here, the data generating device 1 returns to the process after S1 and determines whether a position setting operation for a different position of the displacement virtual surface TS has been accepted as a process before S2. If no position setting operation has been accepted, the data generating device 1 ends this process. However, if a position setting operation has been accepted, the data generating device 1 sets a new displacement virtual surface TS in S2. After setting the new displacement virtual surface TS, the data generating device 1 proceeds to S3, calculates the virtual surface separation distance VD, and then proceeds to S4, and adds a heat map. The above-described loop process is repeated while the moving operation continues. Note that the data generating device 1 may also proceed to S3 and S4 at the stage of setting the occlusion position virtual plane OV and apply a heat map with zero displacement.

[0065] More preferably, the data generating device 1 may execute a step of determining whether the displacement virtual plane TS set in S2 is downstream of the occlusal position virtual plane OV. If the displacement virtual plane TS is downstream of the occlusal position virtual plane OV, the data generating device 1 calculates the virtual surface separation distance VD in S3 and adds a heat map in S4. However, if the displacement virtual plane TS is upstream of the occlusal position virtual plane OV, the data generating device 1 may not proceed to S3 or may proceed to step S3 but not to S4. More preferably, the heat map may be added to the occlusal surface (occlusal surface forming area) OA upstream of the displacement virtual plane TS, but not to the occlusal surface (occlusal surface forming area) OA downstream of the displacement virtual plane TS.

[0066] That is, the data generating device 1 may be configured as follows: The image changing step moves the position of the virtual surface in a predetermined direction based on a user input, changes the oral cavity image according to the distance that changes depending on the position of the virtual surface, and adds a heat map according to the distance to the occlusal surfaces of the teeth shown in the oral cavity image.

[0067] When the area consisting of the occlusal surface OS or the area consisting of the occlusal surface OS and the area connected to the occlusal surface OS is defined as the occlusal surface forming area OA, the virtual surface at the occlusal position of the teeth is defined as the occlusal position virtual surface OV, and the virtual surface obtained by displacing the occlusal position virtual surface OV is defined as the displaced virtual surface TS, the direction including a component along the tooth axis is defined as the tooth axial component direction, and in the tooth axial component direction, the direction from the occlusal surface toward the tooth root is defined as the down direction, the direction from the tooth root toward the occlusal surface is defined as the up direction, the position proceeding in the down direction is defined as the downstream direction, and the position proceeding in the up direction is defined as the upstream direction, and the distance between each point on the surface of the occlusal surface forming area OA and the displaced virtual surface TS is defined as the virtual surface separation distance VD, the virtual surface to be moved is the displaced virtual surface TS obtained by moving the virtual surface at the position of the occlusal position virtual surface OV in the tooth axial component direction. When the displacement virtual surface TS is downstream of the occlusal position virtual surface OV, the data generating device 1 calculates the virtual surface separation distance VD and adds a heat map to the occlusal surface forming area OA, and when the displacement virtual surface TS is upstream of the occlusal position virtual surface OV, the data generating device 1 does not add a heat map to at least the occlusal surface forming area OA. Furthermore, the data generating device 1 adds a heat map to the occlusal surface forming area OA upstream of the displacement virtual surface TS, but does not add a heat map to the occlusal surface forming area OA downstream of the displacement virtual surface TS.

[0068] As shown in FIG. 8 , the data generating device 1 may display a heat map of the upper jaw image UJ and the lower jaw image LJ side by side on the screen of the display 20. In the example of FIG. 8 , the upper jaw image UJ is displayed at the top of the screen, and the lower jaw image LJ is displayed at the bottom of the screen. Furthermore, individual displacement virtual planes TS may be set for each of the upper jaw image UJ and the lower jaw image LJ. An axial component displacement operation may be individually performed on each individual displacement virtual plane TS, or the amount of displacement of the axial component displacement of the displacement virtual plane TS of the upper jaw image UJ may be the same as the amount of displacement of the axial component displacement of the displacement virtual plane TS of the lower jaw image LJ. For example, when the data generating device 1 receives an operation of downward occlusal separation displacement on the upper jaw image UJ, it may automatically perform the same amount of occlusal separation displacement on the lower jaw image LJ.

[0069] As illustrated in FIG. 8 , the data generating device 1 may capture the occlusal surfaces of the maxillary and mandibular dentition in an intraoral image to which a heat map has been added based on user input, and store the states of the occlusal surfaces of the maxillary and mandibular dentition in the intraoral image to which the heat map has been added as still images in the storage device 13. Furthermore, as illustrated in FIG. 7 , the data generating device 1 may set a point designated by the user on the occlusal surface of a tooth shown in the intraoral image based on user input, and display a pop-up display indicating a numerical value corresponding to the distance between a point constituting the occlusal surface of the tooth and a virtual plane for the set point. The data generating device 1 may output the numerical value corresponding to the distance between a point constituting the occlusal surface of the tooth and the virtual plane as a comma separated values ​​(CSV) file or a text file.

[0070] As described above, the data generating device 1 according to the first embodiment can superimpose a virtual plane on the intraoral image displayed on the display 20 based on user input, and change the heat map in the intraoral image according to the distance between the virtual plane and the occlusal surfaces of the teeth shown in the intraoral image. This allows the user to accurately check the occlusal state of the teeth using the intraoral image with the heat map displayed on the display 20, and therefore easily understand which parts of the upper and lower jaw dentition should be treated and which parts should have their occlusion adjusted.

[0071] 10 is a flowchart illustrating an example of a Bulk-Will angle determination process executed by the data generating device according to embodiment 1. By executing the process shown in FIG. 10 , the data generating device 1 can determine whether the Bulk-Will angle of the subject is appropriate.

[0072] As shown in Fig. 10 , the data generating device 1 sets three points for the Bonwill triangle in the oral cavity image displayed on the display 20 (S11). For example, as shown in Fig. 5 , the data generating device 1 sets three points for the Bonwill triangle on both temporomandibular condyles and incisors based on user input. The data generating device 1 connects the three points for the Bonwill triangle with lines to display the Bonwill triangle superimposed on the oral cavity image on the display 20 (S12).

[0073] The data generating device 1 calculates the Bulk-Will angle, which is formed by the occlusal surface or a virtual plane set along the occlusal surface and the Bonwill triangle (S13). The data generating device 1 determines whether the calculated Bulk-Will angle is within a reference range (S14). For example, the data generating device 1 determines whether the Bulk-Will angle is within a range of 21 degrees to 31 degrees. Alternatively, the data generating device 1 determines whether the Bulk-Will angle deviates from a standard angle (e.g., 26 degrees) by a predetermined value or more. The data generating device 1 can arbitrarily set the reference range (e.g., 21 degrees to 31 degrees) or the standard angle (e.g., 26 degrees) based on user input.

[0074] If the Bulk-Will angle is within the reference range (YES in S14), the data generating device 1 ends this process. On the other hand, if the Bulk-Will angle is not within the reference range (NO in S14), the data generating device 1 indicates that there may be an abnormality in the subject's jaw or occlusion, and displays a message on the display 20 to prompt a jaw and occlusion examination (S15). Thereafter, the data generating device 1 ends this process.

[0075] As described above, the data generating device 1 can determine whether the Bulk-Will angle of the subject is normal, and if the Bulk-Will angle is not normal, can prompt the subject to undergo a jaw and occlusion examination.

[0076] 11 to 13, a data generating device 1 according to a second embodiment will be described. Note that the processing of the data generating device 1 according to the second embodiment described below is realized by the control device 10 (arithmetic unit 11) of the data generating device 1 executing a data generating program 100. In addition, in the data generating device 1 according to the second embodiment, only the parts that are different from the data generating device 1 according to the first embodiment will be described, and the parts that are the same as those in the data generating device 1 according to the first embodiment will be assigned the same reference numerals and will not be described repeatedly.

[0077] The data generating device 1 according to the first embodiment described above superimposes a virtual plane on the oral cavity image as an example of a "virtual surface," but the data generating device 1 according to the second embodiment may superimpose a virtual curved surface using a Monson sphere on the oral cavity image as an example of a "virtual surface." Fig. 11 is a diagram for explaining an example of a Monson sphere set on an oral cavity image.

[0078] As shown in Figure 11, Monson's sphere is a sphere with a radius of approximately 10 cm, centered near the crest of the ethmoid bone. Theoretically, the surface of this Monson's sphere passes through the mandibular condyle, incisal edge, and cusp tips, and mandibular movement is believed to occur along the surface of Monson's sphere. It is also believed that extending the tooth axis perpendicular to the occlusal surface of each tooth passes through the center of Monson's sphere. The curve of Spee, which extends upward from the top to the bottom of the head when the mandibular dentition is viewed from the side and the cusps (cusps of the mandibular canines, premolars, buccal cusps of the molars, the entire surface of the mandibular ramus, and the anteriormost surface of the mandibular condyle) are aligned, is also known as the curve of Spee. When occlusion is normal, the curve of Spee is nearly flat or slightly arched, whereas when occlusion is abnormal, the curve of Spee is larger. It is also believed that this curve of Spee passes along the surface of Monson's sphere.

[0079] A user can use the input device 30 to superimpose the virtual curved surface described above on an oral cavity image. For example, while viewing the oral cavity image from any viewpoint, the user can set the center position of the Monson sphere and set the diameter or radius of the Monson sphere based on the set center position. For example, as shown in FIG. 11 , the user can superimpose the spherical surface (curvature of Spee) of the Monson sphere on the oral cavity image so that it aligns with the occlusal plane where the maxillary and mandibular dentitions occlude. That is, the user ideally determines the position of the virtual curved surface so that it coincides with the occlusal plane. At this time, the user can move or rotate the oral cavity image to view the subject from various viewpoints and set the virtual curved surface at a desired position. Furthermore, the user can change at least one of the color and transparency of the virtual curved surface.

[0080] 12 and 13 are diagrams illustrating an example of how a heat map changes in an oral cavity image as a virtual surface moves. The virtual surface is a surface formed by a partial surface of a sphere, in this case, a partial surface of a Monson sphere. A user can move the position of the virtual surface in a predetermined direction on the oral cavity image by dragging the virtual surface using the input device 30. Specifically, based on user input, the data generating device 1 moves the center position of the Monson sphere toward the occlusal surface along a predetermined plane, specifically the median plane, set in the oral cavity image. As a result, the data generating device 1 changes the heat map, i.e., changes the oral cavity image, in accordance with the distance between the virtual surface and the occlusal surface, which changes depending on the center position of the sphere (Monson sphere).

[0081] When the user moves the virtual curved surface, the virtual curved surface moves closer to or farther away from the occlusal surfaces of the teeth depending on the movement of the virtual curved surface. For example, when the virtual curved surface set on the occlusal surface moves downward (toward the mandible), the virtual curved surface and the mandibular teeth move closer together, while the virtual curved surface and the occlusal surfaces of the maxillary teeth move farther apart. Also, when the virtual curved surface set on the occlusal surface moves upward (toward the maxillary), the virtual curved surface and the maxillary teeth move closer together, while the virtual curved surface and the occlusal surfaces of the mandibular teeth move farther apart.

[0082] The data generating device 1 calculates the distance between the virtual curved surface moved based on user input and the occlusal surface of each tooth of the upper or lower jaw shown in the oral cavity image, and changes the oral cavity image according to the distance that changes depending on the position of the virtual curved surface. Specifically, the data generating device 1 calculates the distance between each point constituting the occlusal surface of each tooth shown in the oral cavity image and the virtual curved surface, and adds a heat map corresponding to the distance to each point on the occlusal surface.

[0083] For example, as shown in Figures 12 and 13, when the user moves the virtual curved surface from the upper jaw side to the lower jaw side, the heat map added to each point on the occlusal surface changes depending on the distance between each point constituting the occlusal surface of each tooth on the lower jaw and the virtual curved surface. In the example shown in Figures 12 and 13, the color area of ​​the heat map added to the oral cavity image gradually increases as the virtual curved surface is moved.

[0084] In this way, the user can move the virtual curved surface and check which positions on the occlusal surface of each tooth and in what order they come into contact with the virtual curved surface, thereby enabling the user to check the occlusion state of the teeth with high accuracy.

[0085] The data generating device 1 according to the second embodiment may set a point designated by a user on the occlusal surface of the tooth shown in the intraoral image, and add a numerical value corresponding to the distance between the point constituting the occlusal surface of the tooth and a virtual plane to the set point, similar to the data generating device 1 according to the first embodiment illustrated in Fig. 7. Furthermore, the data generating device 1 according to the second embodiment may display the occlusal surfaces of the tooth, to which the heat maps have been added, separately for the upper jaw dentition and the lower jaw dentition on the display 20, similar to the data generating device 1 according to the first embodiment illustrated in Fig. 8.

[0086] As described above, the data generating device 1 according to the second embodiment can superimpose a virtual surface on the intraoral image displayed on the display 20 based on user input, and change the heat map in the intraoral image according to the distance between the virtual surface and the occlusal surfaces of the teeth shown in the intraoral image. This allows the user to accurately check the occlusal state of the teeth using the intraoral image with the heat map displayed on the display 20, and therefore easily understand which parts of the upper and lower dentition need to be treated and the occlusion adjusted.

[0087] <Modifications> The present disclosure is not limited to the above-described embodiments, and various modifications and applications are possible. Modifications that can be applied to the present disclosure will be described below.

[0088] Although the data generating device 1 is configured to change the heat map attached to the intraoral image according to the distance between the virtual surface and the occlusal surfaces of the teeth shown in the intraoral image, other methods of changing the intraoral image may also be used. For example, the data generating device 1 may display on the display 20 a numerical value indicating the distance between the virtual surface and the occlusal surfaces of the teeth shown in the intraoral image, and change the numerical value displayed on the display 20 according to changes in the distance. The data generating device 1 may also change the pattern attached to the occlusal surface of each tooth shown in the intraoral image or the transparency of each tooth according to the distance between the virtual surface and the occlusal surfaces of the teeth shown in the intraoral image. The data generating device 1 may also display the heat map using shades of achromatic color.

[0089] The data generating device 1 is configured to move the virtual plane in a direction perpendicular to the virtual plane or to move the center position of the Monson sphere along the median plane set in the oral cavity image based on user input, but may also be configured to move the virtual plane in any direction desired by the user based on user input.

[0090] The data generating device 1 is configured to set a virtual surface manually specified by a user, but may also automatically set the virtual surface using AI technology without following user input. For example, the data generating device 1 (control device 10) may include an estimation model for estimating the position of a virtual surface to be superimposed on an oral cavity image based on the oral cavity image. The estimation model is trained by machine learning using training data that includes a set of multiple oral cavity images and ground truth data indicating the position of a virtual surface to be superimposed on each of the multiple oral cavity images to estimate an appropriate position of a virtual surface to be superimposed on an oral cavity image based on input oral cavity image data. Using such an estimation model, a user can input oral cavity image data into the data generating device 1 to automatically superimpose a virtual surface at the position estimated by the estimation model of the control device 10. Furthermore, the data generating device 1 may be configured to allow a user to move and adjust the automatically set virtual surface to a desired position.

[0091] The configuration of the data generating device 1 described above may be included in an X-ray imaging device (CT imaging device). For example, the X-ray imaging device may include an imaging unit that images the upper and lower jaws of a subject. The X-ray imaging device may acquire CT data indicating hard tissues (bones, teeth, etc.) around the upper and lower jaws of the subject through imaging by the imaging unit, generate an oral cavity image based on the acquired CT data, and display the image on the display 20. Furthermore, the X-ray imaging device may superimpose a virtual surface on the oral cavity image displayed on the display 20 based on user input, and change the heat map added to the oral cavity image depending on the distance between the virtual surface and the occlusal surfaces of the teeth shown in the oral cavity image.

[0092] The embodiments disclosed herein should be considered to be illustrative in all respects and not restrictive. The scope of the present disclosure is defined by the claims, not by the above description, and is intended to include all modifications within the meaning and scope of the claims. Note that the configurations exemplified in the present embodiment and the configurations exemplified in the modified examples can be combined as appropriate.

[0093] 1 Data generation device, 10 Control device, 11 Arithmetic device, 12 Memory, 13 Storage device, 14 Display interface, 15 Input device interface, 16 Storage medium interface, 17 Communication device, 20 Display, 30 Input device, 31 Keyboard, 32 Mouse, 40 Storage medium, 100 Data generation program.

Claims

1. A data generation program for generating image data for checking the condition of teeth, the data generation program causing a computer to execute the following steps: a display step for displaying, on a display, an oral cavity image showing an oral cavity including the teeth in three dimensions; an image superimposing step for superimposing a virtual surface on the oral cavity image based on user input or machine learning; and an image changing step for changing the oral cavity image according to the distance between the virtual surface and the occlusal surfaces of the teeth shown in the oral cavity image.

2. The data generation program according to claim 1, wherein the image changing step includes a step of causing the computer to move the position of the virtual surface in a predetermined direction based on the user input, and changing the oral cavity image according to the distance that changes depending on the position of the virtual surface.

3. The data generation program according to claim 2, wherein the predetermined direction is a direction perpendicular to the virtual surface.

4. A data generation program as described in claim 1 or claim 2, wherein the image superimposition step includes a step of causing the computer to display the oral cavity images viewed from a plurality of viewpoints on the display based on the user input, and superimposing the virtual surface on the oral cavity image viewed from a specific viewpoint selected from among the plurality of viewpoints.

5. The data generation program according to claim 1 or 2, further comprising the step of causing the computer to change at least one of a color and a transparency of the virtual surface based on the user input.

6. A data generation program as described in claim 1 or claim 2, wherein the image changing step includes a step of causing the computer to add a heat map according to the distance to the occlusal surfaces of the teeth shown in the intraoral image.

7. The data generating program according to claim 6, further comprising a step of causing the computer to display on the display the occlusal surfaces of the teeth to which the heat maps have been added, separated into the upper jaw dentition and the lower jaw dentition.

8. A data generation program as described in claim 1 or claim 2, further comprising a step of causing the computer to set a point designated by a user on the occlusal surface of the tooth shown in the intraoral image based on the user input, and to add a numerical value corresponding to the distance to the point.

9. The data generation program according to claim 1 or 2, wherein the virtual surface is a plane or a curved surface.

10. The data generation program according to claim 9, wherein the curved surface is composed of a surface of a portion of a sphere set in the intraoral image, and the data generation program further includes a step of causing the computer to set a center position and a diameter or radius of the sphere based on the user input.

11. The data generation program of claim 10, wherein the image changing step includes a step of causing the computer to move the center position of the sphere along a predetermined plane set in the oral cavity image based on the user input, and changing the oral cavity image according to the distance that changes depending on the center position of the sphere.

12. A data generation program as described in claim 1 or claim 2, wherein the oral cavity image is generated based on at least one of optical scanner data captured by an optical scanner including positional information of each point of a point cloud representing the surface of the teeth, and CT data obtained by performing CT (Computed Tomography) on the teeth.

13. The data generation program according to claim 1 or 2, further comprising a step of causing the computer to perform segmentation for each anatomical element shown in the intraoral image.

14. A data generation method for generating image data for checking the condition of teeth, comprising, as processing executed by a computer, a display step for displaying on a display an oral cavity image showing the oral cavity including the teeth in three dimensions; an image superimposition step for superimposing a virtual surface on the oral cavity image based on user input or machine learning; and an image modification step for changing the oral cavity image according to the distance between the virtual surface and the occlusal surfaces of the teeth shown in the oral cavity image.

15. A data generating device that generates image data for checking the condition of teeth, comprising: a display that displays an image; an input device that accepts user input; and a control device that controls the display based on the user input, wherein the control device displays an oral cavity image showing the oral cavity including the teeth in three dimensions on the display; superimposes a virtual surface on the oral cavity image based on the user input or machine learning; and changes the oral cavity image depending on the distance between the virtual surface and the occlusal surfaces of the teeth shown in the oral cavity image.

16. An X-ray imaging device that generates image data for checking the condition of teeth, comprising: an imaging unit that images an oral cavity including the teeth; a display that displays an image; an input device that accepts user input; and a control device that controls the display based on the user input, wherein the control device displays on the display an oral cavity image showing the oral cavity in three dimensions imaged by the imaging unit; superimposes a virtual surface on the oral cavity image based on the user input or machine learning; and changes the oral cavity image depending on the distance between the virtual surface and the occlusal surfaces of the teeth shown in the oral cavity image.

Images