Data generation program, data generation method, data generation device, and x-ray imaging device

Abstract

The data generation program causes the arithmetic device to execute: a display step for displaying, on a display, an oral cavity image representing an oral cavity including teeth in three dimensions; an image superimposing step of superimposing the virtual plane on the oral cavity image on the basis of user input or machine learning; and an image changing step for changing the oral cavity image on the basis of the distance between the imaginary plane and the occlusal surface of the teeth indicated by the oral cavity image.

Description

Technical Field

[0001] The present disclosure relates to a data generation program, a data generation method, a data generation device, and an X-ray imaging device for generating image data for confirming the state of teeth. Background Art

[0002] It is important for a patient to form and maintain an ideal occlusal state of teeth. The occlusal state is usually confirmed using articulating paper or dental models, or using digital data obtained by an intraoral scanner (IOS: Intra Oral Scanner) or the like. Patent Document 1 discloses a dental occlusal pressure measurement device including: a sensor sheet coated with an imprinting material and bitten by a subject; and a computer that processes data representing pressure detection points of the sensor sheet during biting. The dental occlusal pressure measurement device can detect the contact position and occlusal pressure of teeth by using the sensor sheet, thereby detecting early contact positions.

[0003] Prior Art Documents Patent Documents Patent Document 1: Japanese Patent Application Laid-Open No. 2005-279094 Summary of the Invention

[0004] Problems to be Solved by the Invention According to the dental occlusal pressure measurement device disclosed in Patent Document 1, an early contact position can be detected by using a sensor sheet, but the contact position and occlusal pressure of teeth may vary depending on the degree of biting of the sensor sheet by the subject, and it may not be possible to accurately confirm the occlusal state. In addition, in the above-described conventional confirmation method, only the current occlusal state of the patient is shown, and the operator cannot accurately confirm the early contact positions of individual teeth.

[0005] The present disclosure has been made to solve such problems, and an object thereof is to provide a technique capable of accurately confirming the occlusal state of teeth.

[0006] Means for Solving the Problems According to an example of the present disclosure, there is provided a data generation program for generating image data for confirming the state of teeth. The data generation program causes a computer to execute: a display step of displaying an oral cavity image representing an oral cavity including teeth in three dimensions on a display; an image overlapping step of overlapping a virtual plane on the oral cavity image based on user input or machine learning; and an image change step of changing the oral cavity image according to the distance between the virtual plane and the occlusal surface of the teeth shown in the oral cavity image.

[0007] According to an example of the present disclosure, there is provided a data generation method for generating image data for confirming the state of teeth. The data generation method includes the following steps as processing executed by a computer: a display step of displaying an oral cavity image representing an oral cavity including teeth in three dimensions on a display; an image overlapping step of overlapping a virtual plane on the oral cavity image based on user input or machine learning; and an image changing step of changing the oral cavity image according to the distance between the virtual plane and the occlusal surface of the teeth shown in the oral cavity image.

[0008] According to an example of the present disclosure, there is provided a data generation device for generating image data for confirming the state of teeth. The data generation device includes: a display for displaying an image; an input device for receiving user input; and a control device for controlling the display based on the user input. The control device displays an oral cavity image representing an oral cavity including teeth in three dimensions on the display, overlaps a virtual plane on the oral cavity image based on user input or machine learning, and changes the oral cavity image according to the distance between the virtual plane and the occlusal surface of the teeth shown in the oral cavity image.

[0009] According to an example of the present disclosure, there is provided an X-ray imaging device for generating image data for confirming the state of teeth. The X-ray imaging device includes: an imaging unit for imaging an oral cavity including teeth; a display for displaying an image; an input device for receiving user input; and a control device for controlling the display based on the user input. The control device displays an oral cavity image representing the oral cavity imaged by the imaging unit in three dimensions on the display, overlaps a virtual plane on the oral cavity image based on user input or machine learning, and changes the oral cavity image according to the distance between the virtual plane and the occlusal surface of the teeth shown in the oral cavity image.

[0010] Advantageous Effects of the Invention According to the present disclosure, the user can overlap the virtual plane with the oral cavity image displayed on the display and change the oral cavity image according to the distance between the virtual plane and the occlusal surface of the teeth shown in the oral cavity image, so that the occlusal state of the teeth can be confirmed with high accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 FIG. is a diagram showing an application example of the data generation device according to Embodiment 1.

[0012] Figure 2 FIG. is a block diagram showing the structure of the data generation device according to Embodiment 1.

[0013] Figure 3 FIG. is a diagram for explaining an example of a virtual plane set in an oral cavity image.

[0014] Figure 4 FIG. is a diagram for explaining an example of a virtual plane set in an oral cavity image.

[0015] Figure 5 This is a diagram for explaining an example of calculating the Balkwill angle.

[0016] Figure 6 This is a diagram for explaining an example of the change in the heat map in the oral cavity image accompanying the movement of the imaginary plane.

[0017] Figure 7 This is a diagram for explaining an example of the change in the heat map in the oral cavity image accompanying the movement of the imaginary plane.

[0018] Figure 8 This is a diagram for explaining an example of the maxillary dentition and the mandibular dentition with a heat map attached.

[0019] Figure 9 This is a flowchart for explaining an example of the data generation process executed by the data generation device according to Embodiment 1.

[0020] Figure 10 This is a flowchart for explaining an example of the Balkwill angle determination process executed by the data generation device according to Embodiment 1.

[0021] Figure 11 This is a diagram for explaining an example of the Monson sphere set in the oral cavity image.

[0022] Figure 12 This is a diagram for explaining an example of the change in the heat map in the oral cavity image accompanying the movement of the imaginary curved surface.

[0023] Figure 13 This is a diagram for explaining an example of the change in the heat map in the oral cavity image accompanying the movement of the imaginary curved surface. Detailed implementation mode

[0024] <Embodiment 1> Embodiment 1 of the present disclosure will be described in detail with reference to the accompanying drawings. It should be noted that for the same or corresponding parts in the drawings, the same reference numerals are used and their descriptions are not repeated.

[0025] [Application example] Refer to Figure 1 to explain an application example of the data generation device 1 according to Embodiment 1. Figure 1FIG. 0 is a diagram showing an application example of the data generation device 1 according to Embodiment 1. In the preparation stage, a user of the data generation device 1 scans the oral cavity of a subject using a three-dimensional scanner (optical scanner) (not shown), thereby obtaining three-dimensional data (optical scanner data) that includes position information of each point of a point group (a plurality of points) representing the surface of a biological tissue including teeth and gums in the oral cavity. The three-dimensional data includes coordinates (X, Y, Z) of each point representing the surface of the biological tissue in the X-axis direction, Y-axis direction, and Z-axis direction determined in advance as position information. It should be noted that regarding the directions in the above position information, for example, the X-axis direction may be the horizontal direction (e.g., the left-right direction of the head), the Y-axis direction may be the longitudinal direction (e.g., the front-back direction of the head), and the Z-axis direction may be the height direction (e.g., the extending direction of the body axis of the head of a standing person). For example, when the center of the head is set as the reference point of the coordinates (e.g., a point with a value of zero), the right side of the reference point may be set as the +X direction (positive direction) of the X-axis, the left side of the reference point may be set as the -X direction (negative direction) of the X-axis, the front side of the reference point may be set as the +Y direction (positive direction) of the Y-axis, the rear side of the reference point may be set as the -Y direction (negative direction) of the Y-axis, the upper side of the reference point may be set as the +Z direction (positive direction) of the Z-axis, and the lower side of the reference point may be set as the -Z direction (negative direction) of the Z-axis. In each of the X-axis, Y-axis, and Z-axis, the direction from the side with a smaller value to the side with a larger value is the + direction, and the direction from the side with a larger value to the side with a smaller value is the - direction. For example, in the case of the X-axis, the direction from the -X side to the +X side is the +X direction, and the direction from the +X side to the -X side is the -X direction. Further, the three-dimensional data may also include color information that represents the actual color of the portion (the surface portion of the biological tissue) corresponding to each point of the point group (a plurality of points) representing the surface of the biological tissue including teeth and gums in the oral cavity. In addition, regarding the "tooth axis" described later, it is considered that the upper teeth are "opposite teeth" relative to the lower teeth, and the lower teeth are "opposite teeth" relative to the upper teeth. Also, the side of the "opposite tooth" on the tooth axis of a certain tooth is referred to as "above the tooth axis", and the side of a certain tooth on the tooth axis that is far from the "opposite tooth" is referred to as "below the tooth axis".

[0026] The "user" includes operators (such as doctors) or assistants (such as dental assistants, dental technicians, nurses, etc.) in various fields such as dentistry, oral surgery, orthopedics, plastic surgery, and cosmetic surgery. The "subject" includes patients in dentistry, oral surgery, orthopedics, plastic surgery, and cosmetic surgery, etc. In addition, hereinafter, the expression "the user performs..." includes: the user issues an execution instruction and the device or program executes or runs the process.

[0027] A three-dimensional scanner is an intraoral scanner that can optically photograph the inside of a subject's oral cavity by methods such as confocal method or triangulation method, and can obtain the position information of each point of a point cloud on the surface of a biological tissue (e.g., teeth and gums in the oral cavity) placed in a certain coordinate space. The operation of optically photographing and scanning the inside of the oral cavity is also called IOS (intraoral scanning). The user can generate a rendered image (appearance image) representing the three-dimensional shape of the biological tissue by using the three-dimensional data obtained by the three-dimensional scanner. A "rendered image" is an image generated by processing or editing a certain data. For example, the user can process or edit the three-dimensional data of the biological tissue obtained by the three-dimensional scanner to generate a rendered image representing the biological tissue (the part of the biological tissue that can be represented by IOS data) observed from a given viewpoint. And the user can generate multiple rendered images representing the biological tissue observed from multiple directions by changing the given viewpoint over multiple directions. The rendered image can also be a two-dimensional image. For example, a two-dimensional image can also be an image obtained by projecting image data on a projection plane in a coordinate operation that intersects the line-of-sight direction. For example, the rendered image can also be generated by a three-dimensional image that can be stereoscopically observed by a viewer such as a general VR headset.

[0028] In addition, the user can also photograph the maxilla and mandible of the subject by using a CT (Computed Tomography) imaging device (not shown) to obtain three-dimensional volume (voxel) data of the hard tissue parts (bones, teeth, etc.) around the maxilla and mandible of the subject. The CT imaging device is an "X-ray imaging device" that performs CT imaging of the maxilla and mandible of the subject by rotating an X-ray transmitter and receiver, which are a type of radiation, around the subject's face. The user can generate a rendered image (tomographic image or appearance image) representing the three-dimensional shape of the biological tissue by using the volume data of the biological tissue to be imaged obtained by the CT imaging device. For example, the user can process or edit the volume data of the biological tissue obtained by the CT imaging device to generate a rendered image representing the biological tissue (the part of the biological tissue that can be represented by CT data) observed from a given viewpoint. And the user can generate multiple rendered images representing the biological tissue observed from multiple directions by changing the given viewpoint in multiple directions. Similar to the case of IOS data, the rendered image can be a two-dimensional or three-dimensional image.

[0029] Hereinafter, the three-dimensional data including the position information of each point of the point group representing the surface of the biological tissue acquired by the three-dimensional scanner is also referred to as "IOS data", and the rendered image generated based on the IOS data is also referred to as "IOS image". In addition, the three-dimensional volume data acquired by the CT imaging device is also referred to as "CT data", and the rendered image generated based on the CT data is also referred to as "CT image". The IOS image can represent the surface shape of the biological tissue to be scanned in great detail, but cannot represent the internal structures (such as alveolar bone and root apex) that do not appear on the surface of the biological tissue. The CT image can represent the hard tissue parts (bones, teeth, etc.) in the object to be imaged in relatively detail, but for the soft tissue parts (skin, gums, etc.), it cannot represent them in more detail than the hard tissue parts.

[0030] The user can generate oral image data by synthesizing the IOS data and CT data acquired for the same subject. Here, the data formats of the IOS data and CT data are different from each other. Therefore, for example, the user converts the data format of the IOS data into the data format of the CT data and uses the two converted data to perform pattern matching on the three-dimensional shape of the surface of the biological tissue, thereby generating oral image data that synthesizes the IOS data and CT data. It should be noted that the user can also convert the data format of the CT data into the data format of the IOS data and use the two converted data to perform pattern matching on the three-dimensional shape of the surface of the biological tissue, thereby generating oral image data. Or, the user can also convert the data formats of the CT data and IOS data into a common data format and use the two converted data to perform pattern matching on the three-dimensional shape of the surface of the biological tissue, thereby generating oral image data. Regarding how to handle the data format, it can also be determined on the program side.

[0031] The user can generate a rendered image (for example, Figure 1 the oral image shown) representing the two-dimensional biological tissue (the part of the biological tissue that can be represented by both the IOS data and CT data) observed from a given viewpoint by performing processing or editing on the oral image data. As Figure 1 shown, the oral image can represent the surface shape of the biological tissue represented by the IOS data and the tomographic structure or appearance of the hard tissue parts (bones, teeth, etc.) represented by the CT data in three dimensions. It should be noted that when generating the oral image, the user can also adjust the brightness, contrast, transparency, etc. as needed in each of the IOS data and CT data.

[0032] It should be noted that the data generation device 1 can also obtain IOS data from a three-dimensional scanner, obtain CT data from a CT imaging device, and generate an oral cavity image based on user input using the obtained IOS data and CT data. Alternatively, the data generation device 1 may not obtain IOS data and CT data, but obtain oral cavity image data generated by the user using other devices from other devices.

[0033] In the oral cavity image, the three-dimensional shapes of hard tissue parts such as the alveolar bone and the root apex are represented by CT data, and for the three-dimensional shapes of soft tissue parts such as the gingiva that cannot be represented by CT data, they can be represented by IOS data. Thus, in the oral cavity image, soft tissue parts such as the gingiva that cannot be represented only by CT data are also supplemented by IOS data, and thus can be represented in detail together with hard tissue parts such as the alveolar bone and the root apex.

[0034] As Figure 1 shown, the data generation device 1 displays an oral cavity image representing the oral cavity including teeth on the display 20. The data generation device 1 can display an oral cavity image representing the oral cavity observed from various viewpoints on the display 20 by processing the oral cavity image displayed on the display 20 based on user input. Further, the data generation device 1 can attach a heat map indicating an early contact position or the like to at least one occlusal surface of the upper dental arch and the lower dental arch in the oral cavity image displayed on the display 20. Thus, the user can confirm the occlusal state of each tooth by confirming the occlusal surface on which the heat map is displayed. Here, the "heat map" refers to an image representing the difference between the values of a certain part and other parts by the degree of color change. For example, in the heat map, the following processing can be adopted: corresponding to the distance between the imaginary plane and the occlusal surface, the image is gradually changed from a larger value to a smaller value and from warm colors to cool colors. It should be noted that in the heat map, the following processing can also be adopted: according to the distance between the imaginary plane and the occlusal surface, the image is gradually changed from a large value to a small value and from cool colors to warm colors.

[0035] Narrowly speaking, the "occlusal surface" is, in the molar teeth, the surface that comes into contact with the opposing teeth when the upper teeth and the lower teeth are meshed, and is the surface for crushing and grinding food when eating. In the embodiment, the occlusal surface in this sense is represented as "occlusal surface OS". In addition, the area to which the heat map is attached can be the occlusal surface OS, but as shown in Figure 5 below, it may include not only the occlusal surface OS but also the area EO connected to the occlusal surface OS. The area that becomes the heat map target area of such a tooth surface is also referred to as the "occlusal surface forming area OA" (refer to Figure 5). Of course, there are occlusal parts in areas of teeth other than molars, such as the anterior tooth area, so these areas can also be included in the area covered by the occlusal surface forming area OA. When the "occlusal surface" in the embodiment refers to the area that is the object of the additional heat map, the "occlusal surface" can also refer to the occlusal surface forming area OA. Hereinafter, the "occlusal surface" representing the occlusal surface forming area OA will also be referred to as "occlusal surface OA".

[0036] It should be noted that the data generation device 1 is not limited to generating an oral cavity image based on the data obtained by synthesizing IOS data and CT data. For example, the data generation device 1 can also generate an oral cavity image based only on IOS data. The oral cavity image generated based only on IOS data can at least represent the surface shapes of the teeth and gums in the oral cavity. In addition, the data generation device 1 can also generate an oral cavity image based only on CT data. The oral cavity image generated based only on CT data can at least represent the shapes of hard tissue parts such as teeth and bones. Even when generating an oral cavity image based only on CT data, an image of the surface of the teeth can be formed, so the surface shape data can also be used instead of the oral cavity image based on IOS data.

[0037] Even when generating an oral cavity image using either IOS data or CT data, the data generation device 1 can perform segmentation for each anatomical element shown in the oral cavity image. For example, the segmentation can also be various segmentations such as the respective segmentations of hard tissue and soft tissue, the respective segmentations of maxillary hard tissue and mandibular hard tissue, the respective segmentations of dentition and alveolar bone, the respective segmentations of the maxillary dentition and the mandibular dentition, and the respective segmentations of multiple teeth. For example, the data generation device 1 can also identify, based on user input, the areas of the maxillary dentition, the mandibular dentition, the individual teeth included in the maxillary dentition, and the individual teeth included in the mandibular dentition divided by the user, and segment these respective parts and represent them in the oral cavity image.

[0038] Alternatively, the data generation device 1 may also automatically segment each part without following the user input, but using AI (Artificial Intelligence) technology or the like. For example, the data generation device 1 may also include a estimation model including a neural network, and the neural network is used to identify the dental arches of the upper jaw, the dental arches of the lower jaw, and each of the plurality of teeth shown in the oral cavity image based on the oral cavity image data. The estimation model is trained by machine learning in the following manner: using the learning data obtained by grouping a plurality of oral cavity images with the correct answer data, and based on the input oral cavity image data, segmenting the dental arches of the upper jaw, the dental arches of the lower jaw, and each of the plurality of teeth shown in the oral cavity image, where the correct answer data represents the regions of the segmented parts shown in each of the plurality of oral cavity images. If such an estimation model is used, the user can input oral cavity image data to the data generation device 1, and thereby obtain an oral cavity image representing the dental arches of the upper jaw, the dental arches of the lower jaw, and each of the plurality of teeth segmented by the estimation model of the control device 10.

[0039] [Structure of Data Generation Device] Refer to Figure 2 The structure of the data generation device 1 according to Embodiment 1 will be described. Figure 2 It is a block diagram showing the structure of the data generation device 1 according to Embodiment 1. The data generation device 1 can be implemented by a general-purpose computer, for example, or by a dedicated computer.

[0040] As Figure 2 shown, the data generation device 1 includes a control device 10, a display 20, and an input device 30.

[0041] The control device 10 includes an arithmetic device 11, a memory 12, a storage device 13, a display interface 14, an input device interface 15, a storage medium interface 16, and a communication device 17.

[0042] The arithmetic device 11 is an arithmetic entity (computer) that executes various processes by executing various programs. The arithmetic device 11 is constituted by a processor such as a CPU (Central Processing Unit), an MPU (Micro Processing Unit), a TPU (Tensor Processing Unit), or a GPU (Graphics Processing Unit), for example. It should be noted that the processor, which is an example of the arithmetic device 11, has a function of executing various processes by executing a program, but a part or all of these functions may also be installed using a dedicated hardware circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array). The "processor" is not limited to a narrow sense of a processor that executes processes in a stored-program manner such as a CPU, an MPU, a TPU, or a GPU, and can include hardwired circuits such as an ASIC or an FPGA. In addition, the arithmetic device 11 is not limited to a von Neumann type computer such as a CPU or a GPU, and may be constituted by a non-von Neumann type computer such as a quantum computer or an optical computer. The arithmetic device 11 as described above can also be renamed as a processing circuitry that executes a given process. It should be noted that the arithmetic device 11 may be constituted by one chip or multiple chips. Also, the processor and the associated processing circuitry may be constituted by multiple computers, and the multiple computers are connected to each other by wire or wirelessly via a local area network or a wireless network. The processor and the associated processing circuitry may also be constituted by the following cloud computing mechanism, and the cloud computer performs arithmetic operations remotely based on input data and outputs the arithmetic operation results to other devices located at a remote location. The arithmetic device 11 executing various programs is also equivalent to various programs causing the arithmetic device 11 to execute various processes.

[0043] The memory 12 includes a volatile storage area (such as a working area) that temporarily stores program codes or a working memory, etc. when the arithmetic device 11 executes various programs. As an example of the memory 12, a volatile memory such as a DRAM (Dynamic Random Access Memory) or an SRAM (Static Random Access Memory), or a non-volatile memory such as a ROM (Read Only Memory) or a flash memory can be cited.

[0044] The storage device 13 stores various programs, various data, etc. executed by the arithmetic device 11. The storage device 13 may be one or more non-transitory computer readable media, or may be one or more computer readable storage media. As an example of the storage device 13, an HDD (Hard Disk Drive) and an SSD (Solid State Drive) etc. can be cited.

[0045] The storage device 13 stores the data generation program 100. The data generation program 100 describes the content of data generation processing for causing the arithmetic device 11 to generate oral cavity image data for confirming the occlusal state of teeth.

[0046] The display interface 14 is an interface for connecting the display 20. The display interface 14 realizes the input / output of data between the control device 10 and the display 20. For example, the control device 10 displays an oral cavity image based on the oral cavity image data on the display 20 via the display interface 14.

[0047] The input device interface 15 is an interface for connecting input devices 30 such as the keyboard 31 and the mouse 32. The input device interface 15 realizes the input / output of data between the control device 10 and the input device 30. For example, the user can input a desired instruction via the input device interface 15 by using input devices 30 such as the keyboard 31 and the mouse 32, and can cause the control device 10 to generate and edit oral cavity image data based on the instruction. It should be noted that the operation input by the user to the control device 10 using the input device 30, or the data (instruction) input by the user to the control device 10 is also referred to as "user input".

[0048] The storage medium interface 16 reads out various data stored in a storage medium 40 such as a removable disk, or writes various data into the storage medium 40. For example, the data generation device 1 can obtain a data generation program 100 from the storage medium 40 via the storage medium interface 16, or can write oral image data for confirming the occlusal state of teeth into the storage medium 40 via the storage medium interface 16. The storage medium 40 can be one or more non-transitory computer readable media, or can be one or more computer readable storage media. It should be noted that the control device 10 can also obtain IOS data, CT data, or data obtained by synthesizing IOS data and CT data from the storage medium 40 via the storage medium interface 16.

[0049] The communication device 17 transmits and receives data to and from an external device via wired communication or wireless communication. For example, the control device 10 can receive a data generation program 100 from an external device via the communication device 17, or can send oral image data for confirming the occlusal state of teeth to an external device via the communication device 17. It should be noted that the control device 10 can also obtain IOS data, CT data, or data obtained by synthesizing IOS data and CT data from an external device via the communication device 17.

[0050] [An example of generation of an oral image with a heat map attached] Refer to Figures 3 to 7 , and an example of generating an oral image with a heat map attached by the data generation device 1 according to Embodiment 1 will be described. It should be noted that the processing of the data generation device 1 described below is implemented by the control device 10 (arithmetic device 11) of the data generation device 1 executing the data generation program 100. Figure 3 and Figure 4 are diagrams for explaining an example of a virtual plane set in an oral image.

[0051] As Figure 3 shown, the user causes the display 20 to display an oral image. The oral image displayed on the display 20 shows hard tissue parts such as teeth and bone. The oral image can be an image representing at least the hard tissue part composed of teeth. The oral image can also be an image representing at least the hard tissue part composed of dental arches. The user uses the input device 30 to superimpose a virtual plane on the oral image. The virtual plane is an example of a "virtual surface". For example, the user can move or rotate the oral image displayed on the display 20, and thereby, as Figure 3 shown, cause an oral image obtained by observing the object from an oblique front view point to be displayed on the display 20, or as Figure 4As shown, an oral cavity image obtained by observing a subject from a side view point is displayed on the display 20.

[0052] While observing the oral cavity image, the user determines the position of the imaginary plane and places the imaginary plane at the determined position, thereby overlapping the imaginary plane on the oral cavity image. That is, the user can overlap the imaginary plane on the oral cavity image obtained by observing from a specific view point selected from multiple view points. In Figure 3 and Figure 4 example, the user overlaps the imaginary plane on the oral cavity image in such a way that it follows the occlusal surface OS where the dental arches of the upper jaw and the lower jaw occlude. That is, the user ideally determines the position of the imaginary plane so that it coincides with the occlusal surface OS. At this time, the user can move or rotate the oral cavity image to observe the subject from various view points and set the imaginary plane at the desired position. Also, the user can change at least one of the color and transparency of the imaginary plane.

[0053] In the embodiment, the imaginary plane set at the position of the occlusal surface OS is also referred to as the "imaginary plane VS". Additionally, the imaginary plane VS can also be called the occlusal position imaginary plane OV. When the imaginary plane VS is a plane, the occlusal position imaginary plane OV is the occlusal position imaginary plane. When the imaginary plane VS is a curved surface, the occlusal position imaginary plane OV is the occlusal position imaginary curved surface. The occlusal position imaginary plane OV can be set by the operation of the user or automatically by image recognition or machine learning, etc. The occlusal position imaginary plane OV is set to be located at the occlusal surface OS of each of the multiple teeth in the dental arch. Whether the imaginary plane VS is a plane or a curved surface, it is preferably a continuous surface without fine irregularities in order to maintain the uniformity of the entire surface for observing the heat map. Hereinafter, there are cases where the imaginary plane representing the occlusal position imaginary plane OV is labeled "OV". For example, there are cases where the imaginary plane representing the occlusal position imaginary plane OV is represented as the "imaginary plane OV".

[0054] Here, the user can also determine the position of the median plane while observing the oral cavity image, and place the median plane at the determined position, thereby overlapping the median plane on the oral cavity image. The median plane refers to a vertical (vertical when standing upright) plane passing through the midline of the human body shown in the oral cavity image, and is a plane that divides the human body into two equal parts when viewed from the front. Also, the user can determine the position of the Camper plane while observing the oral cavity image, and place the Camper plane at the determined position, thereby overlapping the Camper plane on the oral cavity image. The Camper plane is a plane formed by the line connecting the subnasale point and the tragion point. The Camper plane can also be defined as a plane formed by the line connecting the subalar points on the left and right and the upper edge of the external auditory meatus. The user can also move the Camper plane parallel downward (toward the jaw, for example, in the -Z direction) along the median plane, and determine the position of the imaginary plane (occlusal position imaginary plane OV) that coincides with the occlusal surface OS.

[0055] Figure 5 This is a diagram for explaining an example of calculating the Balkwill angle. As Figure 5 shown, the Balkwill angle is the angle formed by the occlusal surface OS and the Bonwill triangle. Generally, the Balkwill angle is in the range of 21 degrees to 31 degrees, and the average value of the Balkwill angle is 26 degrees. The Bonwill triangle is a triangle formed by the line connecting the two condylar heads and the incisors. The Bonwill triangle can also be defined as a triangle formed by the line connecting the vertex at the center of the upper surface of the two mandibular heads and the incisor point.

[0056] After setting the imaginary plane OV in a manner along the occlusal surface OS, the user can further set the above-mentioned Bonwill triangle, and determine whether the Balkwill angle formed by the set occlusal surface and the Bonwill triangle is within the reference range (for example, in the range of 21 degrees to 31 degrees), and also determine whether the Balkwill angle deviates from the standard angle (for example, 26 degrees) by more than a given value, thereby determining whether there is an abnormality in the jaw and occlusion of the subject. In addition, the user can also set the imaginary plane (occlusal position imaginary plane) OV in such a way that the Balkwill angle becomes the standard angle (for example, 26 degrees). As Figure 5 shown, the imaginary plane OV is represented by a triangle with a small interior angle to make it easy to understand the use of the Bonwill triangle. However, in reality, as Figure 3 shown, it has the following area: The area has an extent that can cover the entire area of the dental arch.

[0057] The imaginary plane VS located at the displaced position relative to the occlusal position imaginary plane OV is also referred to as the displaced imaginary plane TS. In the embodiment, there is a case where the imaginary plane representing the displaced imaginary plane TS is labeled "TS". For example, there is a case where the imaginary plane representing the displaced imaginary plane TS is represented as "imaginary plane TS". It can also be considered that the imaginary plane located at the position of the occlusal position imaginary plane OV is the imaginary plane TS with a displacement amount of zero. Regarding the direction in which the displaced imaginary plane TS is displaced relative to the occlusal position imaginary plane OV, if the imaginary plane is a plane, it can be the direction perpendicular to the imaginary plane as described above, or the -Z direction or the +Z direction. The displacement of the displaced imaginary plane TS is a displacement including a component in the direction along the tooth axis. Such a displacement is also referred to as "tooth axis direction component displacement". The tooth axis direction component displacement away from the occlusal surface OS is also referred to as "occlusal away displacement". The displacement from a position away from the occlusal surface OS approaching the occlusal surface OS is also referred to as "occlusal approaching displacement". In addition, the direction including a component in the direction along the tooth axis is also referred to as "tooth axis component direction". The displacement amount of the tooth axis direction component displacement is preferably uniform relative to the occlusal position imaginary plane OV at any three points that do not overlap linearly on the displaced imaginary plane TS. The distance between each point on the surface of the occlusal surface constituting region OA and the displaced imaginary plane TS is also referred to as "imaginary plane away distance VD". In the tooth axis component direction, the direction from the occlusal surface OS toward the tooth root can be considered the downward direction, and the direction from the tooth root toward the occlusal surface can be considered the upward direction. When location A is in a position advancing in the downward direction compared to location B, it can be considered that location A is in a position downstream of location B. When location A is in a position advancing in the upward direction compared to location B, it can be considered that location A is in a position upstream of location B. Since the occlusal position imaginary plane OV is set at the position of the occlusal surface OS, the occlusal away displacement and the occlusal approaching displacement can also be set as approaching and away from the occlusal position imaginary plane OV.

[0058] Figure 6 and Figure 7 is a diagram for explaining an example of the change in the heat map in the oral cavity image accompanying the movement of the imaginary plane. The user can drag the imaginary plane TS by using the input device 30, thereby moving the position of the imaginary plane TS in a given direction on the oral cavity image. For example, the user can move the imaginary plane TS in a direction perpendicular to the imaginary plane TS. As Figure 6 and Figure 7 illustrated, the oral cavity image can also be set such that in the image showing one of the mandibular hard tissue and the maxillary hard tissue, the other of the mandibular hard tissue and the maxillary hard tissue is not visible. When observing, when the user observes the heat map of the more concerned side from above the tooth axis or obliquely above the tooth axis, the visual recognition is improved when the other side is not shown.

[0059] When the user moves the imaginary plane TS, the distance VD between the imaginary plane TS and each location on the occlusal surface (occlusal surface formation region) OA changes according to the movement of the imaginary plane TS. The imaginary plane TS approaches each location on the occlusal surface OA of each tooth, or the imaginary plane TS moves away from the occlusal surface OS of each tooth. For example, when the imaginary plane TS set on the occlusal surface OS moves in the -Z direction (mandibular side), the imaginary plane TS approaches the locations on the occlusal surface OA of the teeth in the mandible where the imaginary plane TS does not pass. On the other hand, the imaginary plane TS moves away from the occlusal surface OA of the teeth in the maxilla. In addition, when the imaginary plane TS set on the occlusal surface OA moves in the +Z direction (maxillary side), the imaginary plane TS approaches the locations on the occlusal surface OA of the teeth in the maxilla where the imaginary plane TS does not pass. On the other hand, the imaginary plane TS moves away from the occlusal surface OA of the teeth in the mandible.

[0060] The data generation device 1 calculates the distance VD between the imaginary plane TS that moves based on the user input and the occlusal surface OA of each tooth in the maxilla or mandible shown in the oral cavity image, and changes the oral cavity image according to the distance that changes due to the position of the imaginary plane. Specifically, the data generation device 1 calculates the distance VD between each point constituting the occlusal surface OA of each tooth shown in the oral cavity image and the imaginary plane TS, and attaches a heat map corresponding to the distance VD to each point of the occlusal surface OA.

[0061] For example, when the distance between the points forming the occlusal surface OA of the tooth in a given direction (a direction perpendicular to the imaginary plane) and the imaginary plane TS is 0 mm, that is, when the points forming the occlusal surface OA are located on the imaginary plane, the data generation device 1 attaches blue to the points of the occlusal surface that are the calculation objects of the distance. When the tooth protrudes from the imaginary plane TS in a given direction (a direction perpendicular to the imaginary plane, which is above the tooth axis here), and the distance between the points forming the occlusal surface OA of the tooth and the imaginary plane exceeds 0 mm and is less than 1.0 mm, the data generation device 1 attaches green to the points of the occlusal surface OA that are the calculation objects of the distance. When the tooth protrudes from the imaginary plane TS in a given direction (a direction perpendicular to the imaginary plane, which is above the tooth axis here), and the distance between the points (protruding points) forming the occlusal surface OA of the tooth and the imaginary plane TS is 1.0 mm or more and less than 2.0 mm, the data generation device 1 attaches yellow to the points (protruding points) of the occlusal surface OA that are the calculation objects of the distance. When the tooth protrudes from the imaginary plane TS in a given direction (a direction perpendicular to the imaginary plane TS, which is above the tooth axis here), and the distance between the points (protruding points) forming the occlusal surface OA and the imaginary plane TS is 2.0 mm or more and less than 3.0 mm, the data generation device 1 attaches red to the points (protruding points) of the occlusal surface that are the calculation objects of the distance. When the tooth protrudes from the imaginary plane TS in a given direction (a direction perpendicular to the imaginary plane TS, which is above the tooth axis here), and the distance between the points (protruding points) forming the occlusal surface OA and the imaginary plane TS is 3.0 mm or more, the data generation device 1 does not attach a color to the points (protruding points) of the occlusal surface OA that are the calculation objects of the distance. It should be noted that the data generation device 1 can also perform a heat map display of the oral cavity image using the colors specified by the user based on the user input.

[0062] Regarding the colors of the heat map, various assignments can be considered. For example, other assignments such as the following can also be considered.

[0063] (1 of other assignments) In the case where the tooth protrudes from the imaginary plane TS in a given direction (a direction perpendicular to the imaginary plane TS, which is above the tooth axis here), no color is added. When the distance between the point on the occlusal surface OA that constitutes the tooth and the imaginary plane TS is 0 mm, red is added to the point on the occlusal surface OA that is the object of distance calculation. In the state where the tooth is separated from the imaginary plane TS by a distance in a given direction (a direction perpendicular to the imaginary plane TS, which is below the tooth axis here), when the distance between the point on the occlusal surface OA that constitutes the tooth and the imaginary plane TS exceeds 0 mm and is less than 1.0 mm, yellow is added to the point on the occlusal surface OA that is the object of distance calculation. In the state where the tooth is separated from the imaginary plane TS by a distance in a given direction (a direction perpendicular to the imaginary plane TS, which is below the tooth axis here), when the distance between the point on the occlusal surface OA that constitutes the tooth and the imaginary plane TS is 1.0 mm or more and less than 2.0 mm, green is added to the point on the occlusal surface OA that is the object of distance calculation. In the state where the tooth is separated from the imaginary plane TS by a distance in a given direction (a direction perpendicular to the imaginary plane TS, which is below the tooth axis here), when the distance between the point on the occlusal surface OA that constitutes the tooth and the imaginary plane TS is 2.0 mm or more and less than 3.0 mm, blue is added to the point on the occlusal surface OA that is the object of distance calculation.

[0064] (2 of other allocations) In a state where a tooth protrudes from an imaginary plane TS in a given direction (a direction perpendicular to the imaginary plane TS, which is above the tooth axis here), when the distance between a point (the protruding point) on the occlusal surface OA of the tooth and the imaginary plane TS is 1.0 mm or more, red is added to the point on the occlusal surface OA that is the object of distance calculation. In a state where a tooth protrudes from an imaginary plane TS in a given direction (a direction perpendicular to the imaginary plane TS, which is above the tooth axis here), when the distance between a point (the protruding point) on the occlusal surface OA of the tooth and the imaginary plane TS is 0 mm or more and less than 1.0 mm, orange is added to the point on the occlusal surface OA that is the object of distance calculation. When the distance between a point on the occlusal surface OA of the tooth and the imaginary plane TS is 0 mm, yellow is added to the point on the occlusal surface OA that is the object of distance calculation. In a state where a tooth is separated from the imaginary plane TS in a given direction (a direction perpendicular to the imaginary plane TS, which is below the tooth axis here), when the distance between a point on the occlusal surface OA of the tooth and the imaginary plane TS exceeds 0 mm and is less than 1.0 mm, green is added to the point on the occlusal surface OA that is the object of distance calculation. In a state where a tooth is separated from the imaginary plane TS in a given direction (a direction perpendicular to the imaginary plane TS, which is below the tooth axis here), when the distance between a point on the occlusal surface OA of the tooth and the imaginary plane TS is 1.0 mm or more, blue is added to the point on the occlusal surface OA that is the object of distance calculation.

[0065] As described above, the data generation device 1 can represent the distance between each point on the occlusal surface OA of each tooth and the imaginary plane TS in the oral cavity image by a heat map, so that the user can easily confirm the early contact positions of each tooth.

[0066] For example, when the user moves the imaginary plane TS from Figure 6 the position of the imaginary plane shown downward toward the mandibular side, as Figure 7 shown, according to the distance between each point on the occlusal surface OA of each tooth forming the mandible and the imaginary plane TS, the heat map added to each point on the occlusal surface OA changes. For example, when focusing on Figure 7 the heat map shown in Figure 6 and the yellow area in the heat map shown in Figure 7 the state of Figure 6 compared to the state of

[0067] the amount of displacement of the occlusion away is larger, and the area of 1.0 mm or more and less than 2.0 mm increases, so the area to which the color is added becomes larger.

[0068] Further, the data generation device 1 can, based on a user input, set a specified point on the occlusal surface OA of the teeth shown in the oral cavity image, and attach a value corresponding to the distance between the point constituting the occlusal surface OA of the teeth and the imaginary plane TS to the set point. For example, as Figure 7 shown, when the user moves the cursor using the input device 30 and specifies a desired point on the occlusal surface OA of the desired tooth, the data generation device 1 pop - up - displays a value (in the Figure 7 example, "1.5 mm") representing the distance between the point corresponding to this point and the imaginary plane TS near the specified point. Thus, the user can easily confirm the occlusal state of the specified point.

[0069] Figure 8 is a diagram for explaining an example of the maxillary dentition and the mandibular dentition to which a heat map is attached. As Figure 8 shown, the data generation device 1 can divide the occlusal surface OA of the teeth to which the heat map is attached into the maxillary dentition and the mandibular dentition and display them on the display 20.

[0070] For example, the data generation device 1 can attach a heat map to each dentition in a state where the maxillary dentition and the mandibular dentition are open, capture the occlusal surface OA of each of the maxillary dentition and the mandibular dentition to which the heat map is attached, and thus save the states of the occlusal surface OA of each of the maxillary dentition and the mandibular dentition to which the heat map is attached as still images in the storage device 13.

[0071] [Processing of the data generation device] With reference to Figure 9 and Figure 10 , the processing performed by the data generation device 1 according to Embodiment 1 will be described. Figure 9 is a flowchart for explaining an example of the data generation processing performed by the data generation device 1 according to Embodiment 1. It should be noted that Figure 9 and Figure 10 each step (hereinafter represented by "S") shown is implemented by the control device 10 (arithmetic device 11) of the data generation device 1 executing the data generation program 100.

[0072] As Figure 9 shown, the data generation device 1 displays an oral cavity image representing the oral cavity including teeth in three - dimensions on the display 20 (S1). For example, as Figure 3As illustrated, the data generation device 1 displays an oral cavity image representing a two-dimensional hard tissue part (bone, tooth, etc.) observed from a given viewpoint on the display 20. At this time, the data generation device 1 can also segment the dental arch of the upper jaw, the dental arch of the lower jaw, and each of the multiple teeth shown in the oral cavity image based on a user input. Alternatively, the data generation device 1 can also use a estimation model trained by machine learning or the like to identify the dental arch of the upper jaw, the dental arch of the lower jaw, and each of the multiple teeth shown in the oral cavity image, and segment and display these respective parts on the oral cavity image. The process of S1 is an example of a "display step".

[0073] Based on a user input entered using the input device 30, the data generation device 1 overlays a virtual plane on the oral cavity image displayed on the display 20 (S2). For example, as Figure 3 illustrated, based on a user input, the data generation device 1 overlays the virtual plane on the oral cavity image in a manner along the occlusal plane where the dental arch of the upper jaw and the dental arch of the lower jaw occlude. At this time, based on a user input, the data generation device 1 displays oral cavity images observed from multiple viewpoints on the display 20, and overlays the virtual plane (occlusal position virtual plane) OV on the oral cavity image observed from a specific viewpoint selected from the multiple viewpoints. And the data generation device 1 can also change at least one of the color and transparency of the virtual plane based on a user input. The process of S2 is an example of an "image overlay step".

[0074] The data generation device 1 calculates the distance between the virtual plane and the occlusal plane of each tooth (S3). For example, the data generation device 1 calculates the distance between the virtual plane fixed at a given position and each point on the occlusal plane of each tooth that constitutes the upper jaw or the lower jaw shown in the oral cavity image.

[0075] Based on the calculated distance between the virtual plane and the occlusal plane, specifically based on the distance between the virtual plane and each point on the occlusal plane of each tooth, the data generation device 1 attaches a heat map to the oral cavity image displayed on the display 20 (S4). For example, as Figure 6 illustrated, the data generation device 1 attaches a color determined in advance according to the distance between the virtual plane and each point to each point on the occlusal plane of each tooth. Then, the data generation device 1 ends this process. It should be noted that, as Figure 7 illustrated, when the virtual plane has moved based on a user input, the data generation device 1 can also recalculate the distance between the moved virtual plane and each point on the occlusal plane of each tooth, and attach a heat map to the oral cavity image displayed on the display 20 according to the calculated distance, that is, change the oral cavity image. The processes of S3 and S4 are an example of an "image change step".

[0076] Preferably, as an initial operation, in step S2, the data generation device 1 first accepts the position setting operation of the occlusal position imaginary plane OV and sets the position of the occlusal position imaginary plane OV. As described above, the data generation device 1 can also automatically set the position of the occlusal position imaginary plane OV. And, as step S2, the data generation device 1 accepts the position setting operation of the displacement imaginary plane TS and sets the position of the displacement imaginary plane TS. The data generation device 1 calculates the imaginary plane separation distance VD in step S3. The data generation device 1 attaches a heat map in step S4. Here, the data generation device 1 returns to after step S1, and as the process before step S2, determines whether there is an acceptance of the position setting operation of the displacement imaginary plane TS at other positions. When there is no position setting operation, the data generation device 1 ends this process, but when there is a position setting operation, a new displacement imaginary plane TS is set in step S2. After setting the new displacement imaginary plane TS, the data generation device 1 enters step S3, calculates the imaginary plane separation distance VD, enters step S4, and attaches a heat map. During the period when the movement operation continues, the above-described loop process is repeated. It should be noted that the data generation device 1 can also enter steps S3 and S4 and perform the assignment of a heat map with a displacement amount of zero at the stage of setting the occlusal position imaginary plane OV.

[0077] More preferably, the data generation device 1 can also execute the step of determining whether the displacement imaginary plane TS set in step S2 is located downstream of the occlusal position imaginary plane OV. When the displacement imaginary plane TS is located downstream of the occlusal position imaginary plane OV, the data generation device 1 calculates the imaginary plane separation distance VD in step S3 and attaches a heat map in step S4, but when the displacement imaginary plane TS is located upstream of the occlusal position imaginary plane OV, it may not enter step S3 and later, or even if it enters step S3, it does not enter step S4. More preferably, the heat map can be assigned to the occlusal surface (occlusal surface formation region) OA upstream of the displacement imaginary plane TS and not assigned to the occlusal surface (occlusal surface formation region) OA downstream of the displacement imaginary plane TS.

[0078] That is, the data generation device 1 can also be configured as follows. In the image change step, based on the user input, the position of the imaginary plane is moved in a given direction, the oral cavity image is changed according to the distance changed due to the position of the imaginary plane, and a heat map corresponding to the distance is attached to the occlusal surface of the teeth shown in the oral cavity image.

[0079] When a region formed by the occlusal surface OS or a region formed by the occlusal surface OS and a region connected to the occlusal surface OS is set as the occlusal surface forming region OA, and a virtual surface at the occlusal position of the tooth is set as the occlusal position virtual surface OV, and a virtual surface obtained by displacing the occlusal position virtual surface OV is set as the displacement virtual surface TS. Additionally, when the direction including the component along the tooth axis direction is set as the tooth axis component direction, and in the tooth axis component direction, the direction from the occlusal surface toward the root is set as the downward direction, the direction from the root toward the occlusal surface is set as the upward direction, the position advanced along the downward direction is set as the downstream, and the position advanced along the upward direction is set as the upstream. Additionally, when the distance between each point on the surface of the occlusal surface forming region OA and the displacement virtual surface TS is set as the virtual surface separation distance VD, the moving virtual surface is the displacement virtual surface TS obtained by moving the virtual surface at the position of the occlusal position virtual surface OV along the tooth axis component direction. When the displacement virtual surface TS is located at a position downstream of the occlusal position virtual surface OV, the data generation device 1 calculates the virtual surface separation distance VD and attaches a heat map to the occlusal surface forming region OA. When the displacement virtual surface TS is located at a position upstream of the occlusal position virtual surface OV, the data generation device 1 does not attach a heat map to the occlusal surface forming region OA at least. And the data generation device 1 assigns a heat map to the occlusal surface forming region OA upstream of the displacement virtual surface TS and does not assign a heat map to the occlusal surface forming region OA downstream of the displacement virtual surface TS.

[0080] As Figure 8 shown, the data generation device 1 can perform heat map display by juxtaposing the image UJ of the upper jaw and the image LJ of the lower jaw on the screen of the display 20. In Figure 8 this example, the image UJ of the upper jaw is displayed at the upper part of the screen, and the image LJ of the lower jaw is displayed at the lower part of the screen. And separate displacement virtual surfaces TS can be set in each of the image UJ of the upper jaw and the image LJ of the lower jaw. An operation of axial component displacement can be applied to the separate displacement virtual surfaces TS individually, but the displacement amount of the axial component displacement of the displacement virtual surface TS of the image UJ of the upper jaw can also be made the same as the displacement amount of the axial component displacement of the displacement virtual surface TS of the image LJ of the lower jaw. For example, when the data generation device 1 receives an operation of occlusal separation displacement in the downward direction in the image UJ of the upper jaw, the data generation device 1 can also automatically perform occlusal separation displacement of the same displacement amount in the image LJ of the lower jaw.

[0081] It should be noted that, as Figure 8 illustrated, the data generation device 1 can also capture the occlusal surfaces of the dental arches of the upper jaw and the lower jaw in the oral image with a heat map attached based on user input, and store the states of the occlusal surfaces of the dental arches of the upper jaw and the lower jaw with a heat map attached as still images in the storage device 13. And, asFigure 7 As exemplified, the data generation device 1 can also, based on a user input, set a point designated by the user on the occlusal surface of the teeth shown in the oral cavity image, and pop up and display a value corresponding to the distance between the point constituting the occlusal surface of the teeth and the imaginary plane. It should be noted that the data generation device 1 can also output, as a CSV (Comma Separated Values) file or a text file, a value corresponding to the distance between the point constituting the occlusal surface of the teeth and the imaginary plane.

[0082] As described above, the data generation device 1 according to Embodiment 1 can, based on a user input, overlap an imaginary plane on the oral cavity image displayed on the display 20, and change the heat map in the oral cavity image according to the distance between the imaginary plane and the occlusal surface of the teeth shown in the oral cavity image. Thus, the user can, using the oral cavity image with the heat map displayed on the display 20, accurately confirm the occlusal state of the teeth, and thus can easily grasp which parts of the maxillary dentition and the mandibular dentition should be treated to adjust the occlusion.

[0083] Figure 10 is a flowchart for explaining an example of the Balkwill angle determination process executed by the data generation device according to Embodiment 1. The data generation device 1 can, by executing Figure 10 the processing shown, thereby determine whether the Balkwill angle of the subject is appropriate.

[0084] As Figure 10 shown, the data generation device 1 sets three points for the Bonwill triangle (S11) in the oral cavity image displayed on the display 20. For example, as Figure 5 exemplified, the data generation device 1, based on a user input, sets three points for the Bonwill triangle in the condylar heads of both jaws and the incisors. The data generation device 1 overlaps the Bonwill triangle on the oral cavity image and displays it on the display 20 by connecting the three points for the Bonwill triangle with lines (S12).

[0085] The data generation device 1 calculates the Balkwill angle formed by the occlusal surface or an imaginary plane set to be along the occlusal surface, and the Bonwill triangle (S13). The data generation device 1 determines whether the calculated Balkwill angle is within a reference range (S14). For example, the data generation device 1 determines whether the Balkwill angle is within the range of 21 degrees to 31 degrees. Alternatively, the data generation device 1 determines whether the Balkwill angle deviates from a standard angle (for example, 26 degrees) by a given value or more. It should be noted that the data generation device 1 can arbitrarily set the reference range (for example, 21 degrees to 31 degrees) or the standard angle (for example, 26 degrees) based on a user input.

[0086] When the Balkwill angle is within the reference range (Yes in S14), the data generation device 1 ends this process. On the other hand, when the Balkwill angle is not within the reference range (No in S14), since there may be abnormalities in the subject's jaw and occlusion, the data generation device 1 displays a message for prompting jaw and occlusion examination on the display 20 (S15). Then, the data generation device 1 ends this process.

[0087] As described above, the data generation device 1 determines whether the Balkwill angle of the subject is normal, and when the Balkwill angle is abnormal, it can prompt the subject for jaw and occlusion examination.

[0088] <Embodiment 2> Refer to Figures 11 to 13 , the data generation device 1 according to Embodiment 2 will be described. It should be noted that the processing of the data generation device 1 according to Embodiment 2 described below is realized by the control device 10 (arithmetic device 11) of the data generation device 1 executing the data generation program 100. In addition, in the data generation device 1 according to Embodiment 2, only the parts different from the data generation device 1 according to Embodiment 1 will be described, and the same parts as those of the data generation device 1 according to Embodiment 1 are denoted by the same reference numerals and will not be described repeatedly.

[0089] In the data generation device 1 according to the above-described Embodiment 1, as an example of the "imaginary plane", an imaginary plane is overlapped with the oral cavity image. However, in the data generation device 1 according to Embodiment 2, as an example of the "imaginary plane", an imaginary curved surface using the Monson sphere may also be overlapped with the oral cavity image. Figure 11 is a diagram for explaining an example of the Monson sphere set in the oral cavity image.

[0090] As Figure 11 shown, the Monson sphere is a sphere with a radius of about 10 cm centered around the crista galli contained in the ethmoid bone. In theory, it is considered that the spherical surface of such a Monson sphere passes through the condyle of the mandible, the incisal edge, and the cusp tip, and the mandibular movement proceeds along the spherical surface of the Monson sphere. In addition, it is considered that if the tooth axis orthogonal to the occlusal surface of each tooth is extended, it passes through the center of the Monson sphere. It should be noted that the curvature that extends upward in the vertical direction of the head when connecting the cusps (the tip of the mandibular canine, premolars, buccal cusps of molars, the entire surface of the mandibular ramus, and the frontmost part of the condyle of the mandible) when observing the mandibular dentition from the side is also called the Spee curve. In the case of normal occlusion, the Spee curve becomes approximately flat or slightly curved in an arc shape, but in the case of abnormal occlusion, the Spee curve becomes larger. Such a Spee curve is also considered to pass through the spherical surface of the Monson sphere.

[0091] The user can use the input device 30 to superimpose the above-mentioned imaginary surface on the oral cavity image. For example, while observing the oral cavity image from an arbitrary viewpoint, the user sets the center position of the Monson sphere and sets the diameter or radius of the Monson sphere based on the set center position. For example, as Figure 11 shown, the user superimposes the spherical surface (Spee curve) of the Monson sphere on the oral cavity image in a manner along the occlusal surface where the dental arches of the upper jaw and the lower jaw occlude. That is, the user ideally determines the position of the imaginary surface in such a way that the imaginary surface coincides with the occlusal surface. At this time, the user can move or rotate the oral cavity image to observe the subject from various viewpoints and set the imaginary surface at the desired position. Also, the user can change at least one of the color and transparency of the imaginary surface.

[0092] Figure 12 And Figure 13 are diagrams for explaining an example of the change in the heat map in the oral cavity image accompanying the movement of the imaginary surface. As the imaginary surface, a surface composed of a part of the surface of a sphere, here a part of the surface of the Monson sphere, is used. The user can drag the imaginary surface by using the input device 30, thereby moving the position of the imaginary surface on the oral cavity image in a given direction. Specifically, the data generation device 1 moves the center position of the Monson sphere toward the occlusal surface along a given plane set in the oral cavity image, as a specific example, the median plane, based on the user input. Thereby, the data generation device 1 changes the heat map, that is, changes the oral cavity image, corresponding to the distance between the imaginary surface and the occlusal surface that changes according to the center position of the sphere (Monson sphere).

[0093] When the user moves the imaginary surface, according to the movement of the imaginary surface, the imaginary surface approaches the occlusal surface of each tooth, or the imaginary surface moves away from the occlusal surface of each tooth. For example, when the imaginary surface set on the occlusal surface moves downward (toward the mandible), the imaginary surface approaches each tooth of the mandible, and on the other hand, the imaginary surface moves away from the occlusal surface of each tooth of the maxilla. Also, when the imaginary surface set on the occlusal surface moves upward (toward the maxilla), the imaginary surface approaches each tooth of the maxilla, and on the other hand, the imaginary surface moves away from the occlusal surface of each tooth of the mandible.

[0094] The data generation device 1 calculates the distance between the imaginary surface moved based on the user input and the occlusal surface of each tooth of the maxilla or mandible shown in the oral cavity image, and changes the oral cavity image according to the distance that changes due to the position of the imaginary surface. Specifically, the data generation device 1 calculates the distance between each point constituting the occlusal surface of each tooth shown in the oral cavity image and the imaginary surface, and attaches a heat map corresponding to the distance to each point of the occlusal surface.

[0095] For example, asFigure 12 and Figure 13 As shown, when the user moves the imaginary surface from the maxillary side toward the mandibular side, the heat map attached to each point on the occlusal surface changes according to the distance between each point on the occlusal surface of each tooth constituting the mandible and the imaginary surface. In Figure 12 and Figure 13 the example shown, as the imaginary surface moves, the area of the color of the heat map attached to the oral cavity image gradually becomes larger.

[0096] In this way, the user can move the imaginary surface while confirming which position on the occlusal surface of each tooth contacts the imaginary surface in what order, and thus can confirm the occlusal state of the teeth with high precision.

[0097] It should be noted that the data generation device 1 according to the second embodiment can also, in the same manner as the data generation device 1 according to the first embodiment exemplified in Figure 7 , set the point positions specified by the user on the occlusal surface of the teeth shown in the oral cavity image, and attach a value corresponding to the distance between the points constituting the occlusal surface of the teeth and the imaginary plane to the set point positions. And the data generation device 1 according to the second embodiment can also, in the same manner as the data generation device 1 according to the first embodiment exemplified in Figure 8 , divide the occlusal surface of the teeth with the heat map attached into the maxillary dentition and the mandibular dentition and display them on the display 20.

[0098] As described above, the data generation device 1 according to the second embodiment can overlap the imaginary surface on the oral cavity image displayed on the display 20 based on the user input, and change the heat map in the oral cavity image according to the distance between the imaginary surface and the occlusal surface of the teeth shown in the oral cavity image. Thus, the user can use the oral cavity image with the heat map attached displayed on the display 20 to confirm the occlusal state of the teeth with high precision, and thus can easily grasp which part of the maxillary dentition and the mandibular dentition should be treated to adjust the meshing.

[0099] <Variation> The present disclosure is not limited to the above-described embodiments, and various modifications and applications can also be made. Hereinafter, variations applicable to the present disclosure will be described.

[0100] The data generating device 1 is configured to change the heat map attached to the oral image according to the distance between the imaginary plane and the occlusal surface of the teeth shown in the oral image, but the oral image may be changed by other methods. For example, the data generating device 1 may also display a numerical value representing the distance between the imaginary plane and the occlusal surface of the teeth shown in the oral image on the display 20, and change the numerical value displayed on the display 20 according to the change of the distance. In addition, the data generating device 1 may also change the pattern attached to the occlusal surface of each tooth shown in the oral image or the transparency of each tooth according to the distance between the imaginary plane and the occlusal surface of the teeth shown in the oral image. In addition, the data generating device 1 may also use achromatic shades to represent the heat map.

[0101] The data generating device 1 is configured to move the imaginary plane in a direction perpendicular to the imaginary plane or to move the center position of the Monsen sphere along the median plane set in the oral image based on user input, but can also be configured to move the imaginary plane in any direction desired by the user based on user input.

[0102] The data generation device 1 is configured to set a virtual surface manually specified by the user, but the virtual surface can also be automatically set using AI technology, etc., instead of according to the user input. For example, the data generation device 1 (control device 10) can also have an estimation model, which is used to estimate the position of the virtual surface superimposed on the oral image based on the oral image. The estimation model is trained by machine learning in the following manner: using learning data after grouping multiple oral images with correct answer data, based on the input oral image data, the appropriate position of the virtual surface superimposed on the oral image is estimated, and the correct answer data represents the position of the virtual surface superimposed on each of the multiple oral images. If such an estimation model is used, the user can automatically overlap the virtual surface at the position estimated by the estimation model of the control device 10 by inputting the oral image data to the data generation device 1. In addition, the data generation device 1 can also be configured to be able to move and adjust the virtual surface automatically set in this way to the position desired by the user.

[0103] The structure of the above-mentioned data generating device 1 can also be provided by an X-ray imaging device (CT imaging device). For example, the X-ray imaging device has an imaging unit for imaging the upper jaw and the lower jaw of the subject. The X-ray imaging device can also obtain CT data representing the hard tissue parts (bone, teeth, etc.) around the upper jaw and the lower jaw of the subject by imaging by the imaging unit, generate an oral cavity image based on the acquired CT data, and display it on the display 20. In addition, the X-ray imaging device can also overlap a virtual plane on the oral cavity image displayed on the display 20 based on user input, and change the heat map attached to the oral cavity image according to the distance between the virtual plane and the occlusal surface of the teeth shown in the oral cavity image.

[0104] The embodiments disclosed herein should be considered illustrative in all respects and not restrictive. The scope of the present disclosure is shown not by the above description but by the claims, and is intended to include meanings equivalent to the claims and all modifications within the scope. It should be noted that the structures exemplified in this embodiment and the structures exemplified in the modification examples can be appropriately combined.

[0105] Reference Numeral Explanation 1: Data generation device; 10: Control device; 11: Arithmetic device; 12: Memory; 13: Storage device; 14: Display interface; 15: Input device interface; 16: Storage medium interface; 17: Communication device; 20: Display; 30: Input device; 31: Keyboard; 32: Mouse; 40: Storage medium; 100: Data generation program.

Claims

1. A data generation program that generates image data for confirming the state of teeth, wherein the data generation program causes a computer to execute: a display step of displaying an oral cavity image that three-dimensionally represents the oral cavity including the teeth on a display; an image overlapping step of overlapping a virtual plane on the oral cavity image based on user input or machine learning; and an image changing step of changing the oral cavity image according to the distance between the virtual plane and the occlusal surface of the teeth shown in the oral cavity image.

2. The data generation program according to claim 1, wherein the image changing step includes the steps of: causing the computer to move the position of the virtual plane in a given direction based on the user input, and changing the oral cavity image according to the distance that changes due to the position of the virtual plane.

3. The data generation program according to claim 2, wherein the given direction is a direction perpendicular to the virtual plane.

4. The data generation program according to claim 1 or 2, wherein the image overlapping step includes the steps of: causing the computer to display the oral cavity images observed from multiple viewpoints on the display based on the user input, and overlapping the virtual plane on the oral cavity image observed from a specific viewpoint selected from the multiple viewpoints.

5. The data generation program according to claim 1 or 2, wherein the data generation program further includes the step of: causing the computer to change at least one of the color and transparency of the virtual plane based on the user input.

6. The data generation program according to claim 1 or 2, wherein the image changing step includes the step of: causing the computer to attach a heat map corresponding to the distance to the occlusal surface of the teeth shown in the oral cavity image.

7. The data generation program according to claim 6, wherein the data generation program further includes the step of: causing the computer to divide the occlusal surface of the teeth to which the heat map is attached into the dental arch of the upper jaw and the dental arch of the lower jaw and display them on the display.

8. The data generation program according to claim 1 or 2, wherein the data generation program further includes the step of: causing the computer to set a point specified by the user on the occlusal surface of the teeth shown in the oral cavity image and attach a numerical value corresponding to the distance to the point.

9. The data generation program according to claim 1 or 2, wherein the virtual plane is a plane or a curved surface.

10. The data generation program according to claim 9, wherein the curved surface is composed of a part of the surface of a sphere set in the oral cavity image, and the data generation program includes the step of: causing the computer to set the center position and diameter or radius of the sphere based on the user input.

11. The data generation program according to claim 10, wherein The image change step includes the following steps: causing the computer to move the center position of the sphere along a given plane set in the oral cavity image based on the user input, and changing the oral cavity image according to the distance changed due to the center position of the sphere.

12. The data generation program according to claim 1 or 2, wherein, the oral cavity image is generated based on at least one of optical scanner data captured by an optical scanner including position information of each point of a point group representing the surface of the teeth and CT data obtained by performing CT (computer tomography) imaging on the teeth.

13. The data generation program according to claim 1 or 2, wherein, the data generation program further includes the following step: causing the computer to segment according to each anatomical element shown in the oral cavity image.

14. A data generation method, which is a data generation method for generating image data for confirming the state of teeth, wherein, the data generation method includes the following steps as processing executed by a computer: a display step of displaying an oral cavity image representing the oral cavity including the teeth in three dimensions on a display; an image overlapping step of overlapping a virtual plane on the oral cavity image based on user input or machine learning; and an image change step of changing the oral cavity image according to the distance between the virtual plane and the occlusal surface of the teeth shown in the oral cavity image.

15. A data generation device, which is a data generation device for generating image data for confirming the state of teeth, wherein, the data generation device includes: a display for displaying an image; an input device for receiving user input; and a control device for controlling the display based on the user input, the control device displays an oral cavity image representing the oral cavity including the teeth in three dimensions on the display, overlaps a virtual plane on the oral cavity image based on the user input or machine learning, and changes the oral cavity image according to the distance between the virtual plane and the occlusal surface of the teeth shown in the oral cavity image.

16. An X-ray imaging device, which is an X-ray imaging device for generating image data for confirming the state of teeth, wherein, the X-ray imaging device includes: an imaging unit for imaging the oral cavity including the teeth; a display for displaying an image; an input device for receiving user input; and a control device for controlling the display based on the user input, the control device displays an oral cavity image representing the oral cavity imaged by the imaging unit in three dimensions on the display, overlaps a virtual plane on the oral cavity image based on the user input or machine learning, and changes the oral cavity image according to the distance between the virtual plane and the occlusal surface of the teeth shown in the oral cavity image.

Images